如何在MATLAB中的进程之间共享内存? [英] How can I share memory between processes in MATLAB?

问题描述

有没有办法在同一台计算机上的MATLAB进程之间共享内存?

Is there any way to share memory between MATLAB processes on the same computer?

我正在多核计算机上运行几个MATLAB进程(如果重要，请运行Windows).它们都使用相同的巨大输入数据.最好在内存中只有一个副本.

I am running several MATLAB processes on a multi-core computer (running Windows, if it matters). They all use the same gigantic input data. It would be nice to only have a single copy of it in memory.

不幸的是，每个过程都需要访问整个巨大的输入数据，因此无法分割数据并解决问题.

Unfortunately each process needs access to the whole gigantic input data, so there is no way to divide the data and conquer the problem.

推荐答案

如果进程仅读取数据，但不修改，那么我相信你可以将输入数据放入一个大文件，并打开每个进程并从该文件读取.每个进程都有其自己的文件位置指示器，可以将其移动到文件中的任何位置以读取所需的数据.我测试了两个MATLAB进程同时从一个文件读取一百万次左右，并且一切正常.我只使用了基本文件I/O命令(在下面列出).看来您也可以使用 MEMMAPFILE 进行此操作，作为 Fooz先生在他的回答中提到(和SCFrench在评论中) (假设您具有MATLAB版本R2008a或更高版本).

If the processes only ever read the data, but do not modify it, then I believe you can place your input data into one large file and have each process open and read from that file. Each process will have it's own file position indicator that it can move anywhere in the file to read the data it needs. I tested having two MATLAB processes reading simultaneously from a file a million or so times each and everything seemed to work fine. I only used basic file I/O commands (listed below). It appears you could also do this using MEMMAPFILE, as Mr Fooz mentioned in his answer (and SCFrench in a comment), assuming you have MATLAB version R2008a or newer.

以下是您可能会使用的一些文件I/O命令:

Here are some of the file I/O commands that you will likely use for this:

• FOPEN :每个过程将调用FOPEN并返回将在所有后续调用中使用的文件标识符.您可以以 binary 或 text 模式打开文件:

• FOPEN: Each process will call FOPEN and return a file identifier it will use in all subsequent calls. You can open a file in either binary or text mode:

fid = fopen('data.dat','r');   % Binary mode
fid = fopen('data.txt','rt');  % Text mode

FREAD :二进制模式下，FREAD将从文件中读取数据:

FREAD: In binary mode, FREAD will read data from the file:

A = fread(fid,20,'double');  % Reads 20 double-precision values

FSCANF :在文本中模式下，FSCANF将从文件中读取数据并格式化数据:

FSCANF: In text mode, FSCANF will read and format data from the file:

A = fscanf(fid,'%d',4);  % Reads 4 integer values

FGETL / FGETS :在文本模式下，这些内容将从文件.

FGETL/FGETS: In text mode, these will read whole lines from the file.

FTELL :这将告诉您当前文件位置指示符(从文件开头算起):

FTELL: This will tell you the current file position indicator in bytes from the beginning of the file:

ftell(fid)
ans =
     8    % The position indicator is 8 bytes from the file beginning

FSEEK :这将将文件位置指示器设置到文件中的所需位置:

FSEEK: This will set the file position indicator to a desired position in the file:

fseek(fid,0,-1);  % Moves the position indicator to the file beginning

FCLOSE :每个过程将必须关闭其对文件的访问权限(很容易忘记这样做):

FCLOSE: Each process will have to close its access to the file (it's easy to forget to do this):

fclose(fid);

此解决方案可能需要输入文件具有易于遍历的结构良好的格式(即仅一个大矩阵).如果它具有很多可变长度的字段，那么从文件中的正确位置读取数据可能会非常棘手.

This solution will likely require that the input file has a well-structured format that is easy to traverse (i.e. just one large matrix). If it has lots of variable length fields then reading data from the correct position in the file could get very tricky.

如果流程还必须修改数据，这可能会变得更加困难.通常，您不希望文件/内存位置同时被多个进程写入，或者不希望一个进程同时另一个进程从同一位置读取而被一个进程写入，因为这会导致不必要的行为.在这种情况下，您将必须限制对文件的访问，以便一次只能对一个进程进行操作.其他过程将不得不等到第一个完成.在这种情况下，每个进程必须运行的示例代码版本为:

If the processes have to also modify the data, this could get even more difficult. In general, you don't want a file/memory location being simultaneously written to by multiple processes, or written to by one process while another is reading from the same location, since unwanted behavior can result. In such a case, you would have to limit access to the file such that only one process at a time is operating on it. Other processes would have to wait until the first is done. A sample version of code that each process would have to run in such a case is:

processDone = false;
while ~processDone,
  if file_is_free(),  % A function to check that other processes are not
                      %   accessing the file
    fid = fopen(fileName,'r+');  % Open the file
    perform_process(fid);        % The computation this process has to do
    fclose(fid);                 % Close the file
    processDone = true;
  end
end

像这样的

同步机制("锁")有时可能会有很高的开销，从而降低了代码的整体并行效率.

Synchronization mechanisms like these ("locks") can sometimes have a high overhead that reduces the overall parallel efficiency of the code.

这篇关于如何在MATLAB中的进程之间共享内存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！