Cloud PubSub重复消息触发的Cloud Functions [英] Cloud Functions triggered by Cloud PubSub duplicate messages

问题描述

我正在尝试使用Cloud Functions作为由PubSub触发的异步后台工作程序，并做一些更长的工作(以分钟为单位). 完整的代码在 https://github.com/zdenulo/cloud-functions-pubsub

I'm experimenting with using Cloud Functions as async background worker triggered by PubSub and doing a bit longer work (in order of minutes). The complete code is here https://github.com/zdenulo/cloud-functions-pubsub

我的原型将数据插入到BigQuery中，并等待几分钟(以模仿更长的任务). 我正在发布100条关于PubSub主题的消息(间隔为1秒).

My prototype inserts data into BigQuery and waits for a few minutes (to mimic longer task). I am publishing 100 messages to PubSub topic (with 1 second interval).

强调了PubSub可以多次传递同一条消息，但是令我惊讶的是，每100条消息中有10条到40条重复. CF的响应时间为5、6、7分钟.在4分钟的回复中，我没有发现重复的内容.
我已经在相同的时间间隔内进行了多次测试.接收第一条消息和第二条消息之间的时间差范围为〜30至〜600秒.

It's emphasized that PubSub can deliver more than once the same message, but I was surprised that from 10 to 40 out of 100 are duplicated. Response time for CF was 5, 6, 7 minutes. For 4 minutes response, I didn't notice duplicates.
I've done multiple tests for the same time intervals. Time difference between receiving first and second message ranges from ~30 to ~600 seconds.

在文档 https://cloud.google.com/pubsub/docs/troubleshooting"Cloud Pub/Sub可以发送重复的消息.例如，当您在确认截止日期到期之前不确认消息时，Cloud Pub/Sub将重新发送该消息."对于Cloud Functions Subscription，确认截止日期为600秒(10分钟)，因此，根据我的理解，这不应该是原因.

In documentation https://cloud.google.com/pubsub/docs/troubleshooting is mentioned "Cloud Pub/Sub can send duplicate messages. For instance, when you do not acknowledge a message before its acknowledgement deadline has expired, Cloud Pub/Sub resends the message." For Cloud Functions Subscription, acknowledge deadline is 600 seconds (10 minutes), so based on my understanding that shouldn't be the reason.

也许我的测试用例是特定的，或者还有别的.
感谢您提供有关如何处理这种情况的建议，如果这是正常现象，或者如何防止重复发生(数据流除外).

Maybe the test case I have is specific or maybe there is something else.
I would be grateful for advice on how to handle such a situation and if this is normal or how to do it to prevent duplicates (excluding Dataflow).

推荐答案

有一个问题会影响在2019年1月之前部署的Cloud Functions，这会导致运行超过5分钟的函数的重复触发器的比率增加.请尝试删除并重新部署您的功能以解决该问题.

There is an issue impacting Cloud Functions deployed before January 2019 that causes an increased rate of duplicate triggers for functions that take more than 5 minutes to run. Please try deleting and re-deploying your function to resolve the issue.

这篇关于Cloud PubSub重复消息触发的Cloud Functions的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！