Mongo更新速度 [英] Mongo Update Speed
问题描述
我有一个脚本，当它在本地系统上运行时，运行速度很快，但是在生产系统上运行时，运行速度慢得多.我已经用PHP事件探查器确认，MongoCollection对象的update方法中发生了减速.我已将PHP软件(二进制和模块)从生产系统复制到本地系统，以确保它不是由于机器/网络的差异而引起的.在我的本地系统上运行生产PHP软件的速度也很慢.因此，问题不在机器或网络上.由于它是相同的脚本，因此与我的脚本也没有关系.这指出了三种可能性之一:
I have a script that when run on my local system runs quickly but when running on the production system it runs much slower. I have confirmed with a PHP profiler that the slow-down is happening in the update method on the MongoCollection object. I have copied the PHP software (binary and modules) from the production system to my local system to make sure it isn't due to a difference in machine/network. Running the production PHP software on my local system also runs slow. So the problem is not machine or network. Since it is the same script it is also not something with my script. This points to one of three possibilities:
- PHP版本中的差异
- Mongo驱动程序版本中的差异
- 配置上的差异
有问题的脚本会遍历集合中的所有记录，并为每个记录进行更新.该更新将写关注设置为0，因为速度比知道成功执行要重要得多. PHP探查器在调用更新后就停止提供信息(因为更新是用C实现的).因此，我转到strace旁边，看看系统调用是否可以帮助解释速度的差异.发出更新的循环在快速系统上具有以下strace输出:
The script in question iterates through all records in the collection and makes an update for each one. The update has the write concern set to 0 as speed is more important than knowing it was executed successfully. The PHP profiler stopped providing information once it called update (since update is implemented in C). So I turned next to strace to see if the system calls could help explain the difference in speed. The loop where the updates are being issued has the following strace output on the fast system:
sendto(3, "\202\0\0\0\206\0\0\0\0\0\0\0\321\7\0\0\0\0\0\0properties_2"..., 130, MSG_DONTWAIT, NULL, 0) = 130
write(1, ".", 1) = 1
sendto(3, "\202\0\0\0\207\0\0\0\0\0\0\0\321\7\0\0\0\0\0\0properties_2"..., 130, MSG_DONTWAIT, NULL, 0) = 130
write(1, ".", 1) = 1
sendto(3, "\202\0\0\0\210\0\0\0\0\0\0\0\321\7\0\0\0\0\0\0properties_2"..., 130, MSG_DONTWAIT, NULL, 0) = 130
write(1, ".", 1) = 1
我正在输出."每次更新之间，因此我有反馈来指示更新进行的速度.在较慢的PHP上运行相同的脚本时，我看到以下内容:
I am outputting "." between each update so I have feedback to indicate how fast the updates are going. When running the same script on the slower PHP I see the following:
sendto(3, "\357\0\0\0v\0\0\0\0\0\0\0\324\7\0\0\0\0\0\0properties_2"..., 239, MSG_DONTWAIT, NULL, 0) = 239
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP}], 1, 30000) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "1\0\0\0009\0.\nv\0\0\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, MSG_DONTWAIT, NULL, NULL) = 49
write(1, ".", 1) = 1
sendto(3, "\357\0\0\0w\0\0\0\0\0\0\0\324\7\0\0\0\0\0\0properties_2"..., 239, MSG_DONTWAIT, NULL, 0) = 239
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP}], 1, 30000) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "1\0\0\0\201\0.\nw\0\0\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, MSG_DONTWAIT, NULL, NULL) = 49
write(1, ".", 1) = 1
sendto(3, "\357\0\0\0x\0\0\0\0\0\0\0\324\7\0\0\0\0\0\0properties_2"..., 239, MSG_DONTWAIT, NULL, 0) = 239
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP}], 1, 30000) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "1\0\0\0\320\0.\nx\0\0\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, MSG_DONTWAIT, NULL, NULL) = 49
write(1, ".", 1) = 1
sendto(3, "\357\0\0\0y\0\0\0\0\0\0\0\324\7\0\0\0\0\0\0properties_2"..., 239, MSG_DONTWAIT, NULL, 0) = 239
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP}], 1, 30000) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "1\0\0\0\365\0.\ny\0\0\0\1\0\0\0\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192, MSG_DONTWAIT, NULL, NULL) = 49
write(1, ".", 1) = 1
请注意，我们已经添加了系统调用.似乎正在检查对我的答复，这表明它未使用写关注0.这两个PHP安装之间的主要区别是快速安装使用1.4.5驱动程序，而慢安装使用1.5.3驱动程序.
Notice that we have added system calls. It looks like it is checking the reply which to me indicates it's not using write concern 0. The major difference between the two PHP installations is the fast one is using driver 1.4.5 while the slow one is using 1.5.3.
我比较了两个版本的更新代码.在1.4.5 中，它似乎只是发送消息然后返回.另一方面，查看1.5.3 它会发送消息然后得到答复.如果写入条件为0，我看不到跳过检查答复的任何信息.如果您遵循提取答复的代码，则可以看到它最终在何处调用了这两个额外的系统调用.
I compared the update code for both versions. In 1.4.5 it seems to just send the message and return. On the other hand looking at 1.5.3 it sends the message then gets a reply. I don't see anything about skipping checking the reply if the write condition is 0. If you follow the code that extracts the reply you can see where it eventually calls these two additional system calls.
能更好地理解这一点的人可以帮助我弄清楚如何使我的代码在生产中快速运行.在快速PHP安装(mongo驱动程序1.4.5)上，脚本仅需2-3分钟即可执行.在慢速系统(mongo驱动程序1.5.3)上，由于厌倦了等待，我在30分钟后将其杀死.谁知道完成一项工作需要多长时间.
Can someone who understand this better help me figure out how to get my code running fast in production. On the fast PHP install (mongo driver 1.4.5) the script executes in just 2-3 minutes. On the slow system (mongo driver 1.5.3) I killed it after 30 minutes because I was tired of waiting. Who knows how long it would have taken to get fully done.
推荐答案
(注意:经过一些额外的研究后更新了原始答案)
(Note: updated original answer after some extra research)
新的写入操作命令随2.6一起提供，因此在任何受支持的驱动程序(PHP 1.5+)和MongoDB服务器(2.6+)之间使用，意味着w=0写入的新语义正在发挥作用.这意味着服务器在发送响应之前等待操作完成(也就是说，w=0和w=1之间的唯一区别是w=0省略了错误详细信息).驾驶员仍会等待该响应，然后再从呼叫返回(即不再失火忘记).
The new write operation commands which came in with 2.6 and hence are in use between any supported driver (PHP 1.5+) and MongoDB server (2.6+) mean that the new semantics of w=0 writes are in play. That means that the server waits for the operation to complete before sending a response (that is, the only difference between w=0 and w=1 is that w=0 omits the error details). The driver still waits for that response before returning from the call (i.e. is no longer fire and forget).
您也可以在MongoDB shell中看到它，而解决该问题的官方方法是使用新的
You can see this in the MongoDB shell itself also, and the official way around it is to use the new Bulk API. Although I know the 1.5 driver will fall back to legacy write operations when connecting to a 2.4 and below server, there is no way to force that behavior in the PHP driver.
这篇关于Mongo更新速度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
