在节点子进程之间共享数据 [英] Share data between node child processes

问题描述

我有一个在多个节点子进程中运行的小节点脚本，并且我无权访问父进程。

I have a small node script that gets ran in several node child processes, and I do not have access to the parent process.

该脚本的目标是非常简单，可以从数组中随机返回一个元素。但是，返回的元素不能被任何其他子进程使用。

The goal of the script is pretty simple, to return one element at random from an array. However, the element returned cannot be used by any of the other child processes.

我能想到的唯一解决方案是使用redis或数据库，因为这样一个小脚本，我想避免这种情况。

The only solutions I can think of involve using redis or a database, and because this is such a tiny script, I would like to avoid that.

这里是我希望代码看起来像的一个例子：

Here is an example of what I would like my code to look like:

var accounts = [acc1, acc2, acc3]
function() {
  var usedAccounts = sharedStore.get('usedAccounts')
  var unusedAccounts = filter(accounts, usedAccounts)
  var account = getRandomAccount(unusedAccounts)
  usedAccounts.push(account)
  sharedStore.set('usedAccounts', usedAccounts)
  return account
}

到目前为止，我想到的解决方案不起作用因为同级进程最初都将分配给 usedAccounts 的空列表。

So far, the solutions I've thought of don't work because the sibling processes initially all get an empty list assigned to usedAccounts.

推荐答案

您需要解决两个问题：

1. 如何共享数据


2. 如何确保所有共享进程之间的数据一致。

如何在多个节点进程之间共享数据。

鉴于您的约束，不想使用外部服务（例如Redis或其他数据库服务），而nodejs没有使用诸如共享内存之类的简便方法，一种可行的解决方案是在所有进程之间使用共享文件。每个进程都可以读取和写入共享文件，并使用该文件来获取 userAccount 数据。

Given your constraints with not wanting to use an external service (like Redis or another database service), and that nodejs doesn't have an easy way to use something like shared memory, a possible solution is to use a shared file between all the processes. Each process can read and write to a shared file, and use that that to get it's userAccount data.

文件可以采用JSON格式，如下所示：

The file could be JSON formatted and look something like this:

[
      {
       "accountName":"bob",
       "accountUsed":false
      },
      {
       "accountName":"alice",
       "accountUsed":true
      }
]

这只是一个userAccount对象的数组，也有一个标志指示如果正在读取数据。

This would just be an array of userAccount objects, that also have a flag that indicate if the data is being read.

您的应用将：

GetAccountData（） ：

1. 打开文件


2. 将文件读入内存


3. 遍历数组


4. 查找第一个可用的用户帐户


5. 设置 accountUsed 标记为true


6. 将更新后的数组写回到文件中


7. 关闭文件。

1. Open the file
2. Read the file into memory
3. Iterate over the array
4. Find the first userAccount that is available
5. Set the accountUsed flag to true
6. Write the updated array back to the file
7. Close the file.

Wi具有多个进程读取和写入单个资源的并发问题是一个众所周知的并发问题，称为 Reader-Writers问题。

With having multiple processes reading and writing to a single resource is a well understood problem with concurrency called the Readers-Writers Problem.

如何确保所有共享过程中数据的一致性。

为确保数据一致，您需要确保一次只有一个进程才能从头到尾一次从上到下运行算法。

To ensure data is consistent, you need to ensure that only one process can run the algorithm from above from start to finish at a time.

操作系统可能提供文件的独占锁定，但是我的nodejs没有对此的原生支持。

Operating Systems may provide exclusive locking of a file, but I nodejs has no native support for that.

常见的机制是使用锁定文件，并使用它的存在是为了保护对上面数据文件的访问。如果无法获取锁，则应等待一段时间，然后尝试重新获取锁。

A common mechanism would be to use a lockfile, and use it's existence to guard access to the datafile above. If it can't acquire a lock, it should wait a period of time, then attempt to reacquire the lock.

要获取锁：

1. 检查锁定文件是否存在。


2. 如果锁定文件存在


3. 设置计时器（ setInterval ）来获取锁


4. 如果锁定文件不存在


5. 创建锁定文件


6. 如果锁定文件创建失败（因为存在-与其他进程竞争条件）


7. 设置计时器（ setInterval ）以获取锁


8. 如果锁定文件创建成功


9. 执行 GetAccountData（）;


10. 删除锁定文件

1. Check if the lockfile exists.
2. If lockfile exists
3. Set a timer (setInterval) to acquire the lock
4. If the lockfile doesn't exist
5. Create the lockfile
6. If the lockfile creation fails (because it exists--race condition with another process)
7. Set a timer (setInterval) to acquire the lock
8. If the lockfile creation succeeds
9. Do GetAccountData();
10. Remove lockfile

此解决方案应该可以工作，但并非没有效果。使用像锁一样的同步原语可能会导致应用程序死锁。如果不正确检查锁的创建，也使用计时器定期获取锁是浪费的，并且会导致竞争。

This solution should work, but it's not without kludge. Using a synchronizing primative like a lock can cause your application to deadlock. Also using a timer to periodically acquire the lock is wasteful and can cause a race condition if not properly checking lock creation.

如果您的应用在删除锁文件之前崩溃了，那么您可能会陷入僵局。为了防止这种情况，您可能需要放置最后一个未处理的异常处理程序，以删除由进程创建的锁文件。

If your app crashes before it removes the lockfile, then you may create a deadlock situation. To guard against that, you might want to put a final unhandled exception handler to remove the lockfile if it was created by the process.

您还需要确保自己只能保持足够长的锁来完成串行工作。保持锁的时间更长，将导致性能问题，并增加出现死锁的可能性。

You will need to also make sure you only hold the lock long enough to do your serial work. Holding the lock for longer, will cause performance issues, and increase the likelihood of a deadlock.

这篇关于在节点子进程之间共享数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！