Node.js / Express和并行队列 [英] Node.js/Express and parallel queues
问题描述
我们正在构建一个具有Node.js服务器和Express的基础设施。
We are building an infrastructure which features a Node.js server and Express.
在服务器中，发生的情况如下：
In the server, what is happening is as follow:
- 服务器接受来自客户端的传入HTTP请求。
- 服务器生成两个文件（此操作可以是相对较长 ，也意味着0.1秒左右）
- 服务器将生成的文件（每个约20-200 KB）上传到外部CDN
- 服务器响应客户端，这包括CDN上文件的URI
- The server accepts an incoming HTTP request from client.
- Server generates two files (this operation can be "relatively long", meaning also 0.1 seconds or so)
- Server uploads the generated files (~20-200 KB each) to an external CDN
- Server responds to client, and this includes the URI of the file on the CDN
目前，服务器正在为每个请求顺序执行此操作，很好（Node / Express可以自动处理并发请求）。然而，随着我们计划增长，并发请求的数量可能会增加，我们认为我们实现一个处理请求的队列会更好。否则，我们可能冒着同时运行太多任务和与CDN打开太多连接的风险。快速响应客户端不是一件相关的事情。
Currently the server is doing this sequentially for each request, and this works quite well (Node/Express can handle concurrent requests automatically). However, as we plan to grow, the number of concurrent requests may grow higher, and we believe it would be better for us to implement a queue for processing requests. Otherwise, we may risk having too many tasks running at the same time and too many open connections to the CDN. Responding to the client quickly is not a relevant thing.
我在想的是在Node服务器上单独包含一个 em>工人（2-3，但我们会做测试来确定正确的同时操作数）。
所以，新的流程将如下所示：
What I was thinking about is to have a separate part in the Node server that contains a few "workers" (2-3, but we will do tests to determine the correct number of simultaneous operations). So, the new flow would look something like:
- 服务器在接受客户端的请求后，添加一个操作一个队列。
- 有2-3（待测试）的工作人员将元素从队列中执行并执行所有操作（生成文件并将其上传到CDN） / li>
- 当工作人员处理了操作（如果它在队列中保持相对较长的时间没关系），它会通知Node服务器（回调），服务器响应客户（同时一直在等待）。
你怎么看待这种方法？你认为这是正确的吗？
What do you think of this approach? Do you believe it is the correct one?
最重要的是，如何在Node / Express中实现？
Mostly important, HOW could this be implemented in Node/Express?
谢谢你的时间
推荐答案
（回答我自己的问题）
根据堆栈溢出问题，我的案例中的一个解决方案是使用 Caolan McMahon的异步模块。
According to this question on Stack Overflow a solution in my case would be to implement a queue using Caolan McMahon's async module.
主应用程序将创建作业并将其推入队列，对可以运行的并发作业数量有限制。这允许同时处理任务，但是对极限进行严格的控制。它像Mac OSX上的Cocoa的NSOperationQueue一样工作。
The main application will create jobs and push them into a queue, which has a limit on the number of concurrent jobs that can run. This allows processing tasks concurrently but with a strict control on the limit. It works like Cocoa's NSOperationQueue on Mac OSX.
这篇关于Node.js / Express和并行队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
