如何在 MATLAB 中的进程之间共享内存? [英] How can I share memory between processes in MATLAB?

问题描述

有没有办法在同一台计算机上的 MATLAB 进程之间共享内存?

Is there any way to share memory between MATLAB processes on the same computer?

我正在多核计算机上运行多个 MATLAB 进程(运行 Windows，如果重要的话).它们都使用相同的巨大输入数据.如果内存中只有一个副本就好了.

I am running several MATLAB processes on a multi-core computer (running Windows, if it matters). They all use the same gigantic input data. It would be nice to only have a single copy of it in memory.

不幸的是，每个进程都需要访问整个庞大的输入数据，因此无法分割数据并解决问题.

Unfortunately each process needs access to the whole gigantic input data, so there is no way to divide the data and conquer the problem.

推荐答案

如果进程只读取数据，但不修改，那么我相信你可以将您的输入数据放入一个大文件中，并打开每个进程并从该文件中读取.每个进程都有它自己的文件位置指示器，它可以移动文件中的任何位置来读取它需要的数据.我测试了让两个 MATLAB 进程同时从一个文件中读取一百万次左右，一切似乎都运行良好.我只使用了基本的文件 I/O 命令(如下所列).看来您也可以使用 MEMMAPFILE 来执行此操作，正如 Mr Fooz 在他的回答中提到的(以及 SCFrench 在评论中提到的)，假设您有 MATLAB 版本 R2008a 或更新版本.

If the processes only ever read the data, but do not modify it, then I believe you can place your input data into one large file and have each process open and read from that file. Each process will have it's own file position indicator that it can move anywhere in the file to read the data it needs. I tested having two MATLAB processes reading simultaneously from a file a million or so times each and everything seemed to work fine. I only used basic file I/O commands (listed below). It appears you could also do this using MEMMAPFILE, as Mr Fooz mentioned in his answer (and SCFrench in a comment), assuming you have MATLAB version R2008a or newer.

以下是您可能会使用的一些文件 I/O 命令:

Here are some of the file I/O commands that you will likely use for this:

• FOPEN:每个流程将调用 FOPEN 并返回一个文件标识符，它将在所有后续调用中使用.您可以以二进制或文本模式打开文件:

• FOPEN: Each process will call FOPEN and return a file identifier it will use in all subsequent calls. You can open a file in either binary or text mode:

fid = fopen('data.dat','r');   % Binary mode
fid = fopen('data.txt','rt');  % Text mode

FREAD:二进制模式，FREAD 将从文件中读取数据:

FREAD: In binary mode, FREAD will read data from the file:

A = fread(fid,20,'double');  % Reads 20 double-precision values

FSCANF:在文本中模式，FSCANF 将从文件中读取和格式化数据:

FSCANF: In text mode, FSCANF will read and format data from the file:

A = fscanf(fid,'%d',4);  % Reads 4 integer values

FGETL/FGETS:在文本模式下，这些将从文件.

FGETL/FGETS: In text mode, these will read whole lines from the file.

FTELL:这将告诉您从文件开头开始的当前文件位置指示符(以字节为单位):

FTELL: This will tell you the current file position indicator in bytes from the beginning of the file:

ftell(fid)
ans =
     8    % The position indicator is 8 bytes from the file beginning

FSEEK:这将将文件位置指示器设置为文件中所需的位置:

FSEEK: This will set the file position indicator to a desired position in the file:

fseek(fid,0,-1);  % Moves the position indicator to the file beginning

FCLOSE:每个流程将不得不关闭它对文件的访问(很容易忘记这样做):

FCLOSE: Each process will have to close its access to the file (it's easy to forget to do this):

fclose(fid);

此解决方案可能要求输入文件具有易于遍历的结构良好的格式(即只有一个大矩阵).如果它有很多可变长度的字段，那么从文件中的正确位置读取数据可能会变得非常棘手.

This solution will likely require that the input file has a well-structured format that is easy to traverse (i.e. just one large matrix). If it has lots of variable length fields then reading data from the correct position in the file could get very tricky.

如果流程还必须修改数据，这可能会变得更加困难.通常，您不希望多个进程同时写入一个文件/内存位置，或者在另一个进程从同一位置读取时写入一个文件/内存位置，因为可能会导致不需要的行为.在这种情况下，您必须限制对文件的访问，以便一次只有一个进程在对其进行操作.其他进程必须等到第一个进程完成.在这种情况下，每个进程必须运行的代码示例版本是:

If the processes have to also modify the data, this could get even more difficult. In general, you don't want a file/memory location being simultaneously written to by multiple processes, or written to by one process while another is reading from the same location, since unwanted behavior can result. In such a case, you would have to limit access to the file such that only one process at a time is operating on it. Other processes would have to wait until the first is done. A sample version of code that each process would have to run in such a case is:

processDone = false;
while ~processDone,
  if file_is_free(),  % A function to check that other processes are not
                      %   accessing the file
    fid = fopen(fileName,'r+');  % Open the file
    perform_process(fid);        % The computation this process has to do
    fclose(fid);                 % Close the file
    processDone = true;
  end
end

像这样的同步机制(locks")有时可能会有很高的降低代码整体并行效率的开销.

Synchronization mechanisms like these ("locks") can sometimes have a high overhead that reduces the overall parallel efficiency of the code.

这篇关于如何在 MATLAB 中的进程之间共享内存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！