谷歌云 pubsub node.js 客户端与谷歌云功能不兼容 [英] google cloud pubsub node.js client not compatible with google cloud functions

问题描述

我们有一个使用 2 个发布订阅主题/订阅对的架构:

We have an architecture using 2 pubsub topic/subscription pairs:

• 主题 T1 由 cronjob 定期触发(例如每 5 分钟一次).订阅 S1 是我们云功能的触发器.
• 主题 T2 用作我们的一项服务发布的后台作业的队列.云函数在每次执行时读取订阅 S2，以服务排队的后台作业.

• Topic T1 is triggered by a cronjob periodically (every 5 minutes for example). Subscription S1 is the trigger for our cloud function.
• Topic T2 serves as a queue for background jobs that are published by one of our services. Subscription S2 is read by the cloud function on each execution to service the queued background jobs.

这使我们能够控制后台作业的服务频率，而与它们何时添加到队列无关.

This allows us to control the frequency the background jobs are serviced independent of when they are added to the queue.

云函数(由 S1 触发)通过 拉动.它决定哪些后台作业已准备就绪，并在成功为作业提供服务后，它会确认相关的消息.未准备好或失败的作业不会被确认以供以后提供服务.

The cloud function (triggered by S1) reads messages from S2 by pulling. It decides which background jobs are ready and upon successfully servicing the job, it ACK's the associated messages. Jobs not ready or failed are not ACK'ed to be serviced later.

我们在使用来自 google 的官方 node.js pubusb 客户端时遇到问题:

We have issues using the official node.js pubusb client from google:

1. 有时 ACK 的消息会重新出现(似乎无限).我们验证了消息在 ACK 截止日期之前得到确认，并通过调查我们的日志确定我们正在调用 ack().
2. 有时在第一次执行后(重新部署函数后)，后续执行永远不会收到新消息.我们可以通过验证堆栈驱动程序中未确认的消息计数或重新部署函数并查看消息得到服务来验证订阅 S2 中的消息是否排队.

1. Sometimes ACK'ed messages re-appear (seeming infinitely). We verified the messages are acked before the ACK deadline and are sure we are calling ack() by investigating our logs.
2. Sometimes after the first execution (after re-deploying the function), subsequent executions never receive new messages. We can verify the messages are queued in subscription S2 either by verifying the unacknowledged message count in stackdriver or by re-deploying the function and seeing the messages getting serviced.

我们认为这是 google 的 node.js pubsub 客户端的问题.云函数文档明确声明不启动后台活动.但是，查看 node.js pubsub 客户端源代码，它显然使用超时在后台服务确认.

We believe this is a problem with google's node.js pubsub client. The cloud function docs clearly state not start background activities. However, looking into the node.js pubsub client source, it clearly services acknowledgements in the background using timeouts.

google的node.js pubsub客户端不兼容google cloud的功能吗?Google 建议仅在客户端库不存在或不满足其他需求时才访问服务 API.在云功能中运行客户端是否是其他需求"，需要我们使用服务 API 编写自己的客户端?

Is google's node.js pubsub client not compatible with google cloud functions? Google recommends accessing the service API's only when a client library does not exist or does not meet other needs. Is running the client in a cloud function "other needs", requiring us to write our own client using the service API's?

作为一种解决方法"，我们尝试延迟执行 cloudfunction 的结束，以允许 node.js pubsub 客户端中的任何后台"进程完成，但这并没有始终消除我们的问题.似乎 pubsub 客户端对云功能不友好，并且无法从云功能执行之间的停止中恢复.

As a "workaround" we tried delaying the end of the execution of the cloudfunction to allow any "background" processes in the node.js pubsub client to complete, but this did not consistently eliminate our issue. It seems that pubsub client is not cloud function friendly and cannot recover from being stopped in between cloud function executions.

我写了 an我们博客上的文章详细描述了为什么我们以这种方式使用 PubSub，以及我们如何解决 node.js pubsub 客户端与云功能不兼容的事实.

I wrote an article on our blog that describes in detail why we used PubSub in this way and how we are working around the fact that node.js pubsub client is not compatible with cloud functions.

推荐答案

来自 node.js pubsub 客户端的开发者 确认不支持使用客户端从云函数中提取消息.

A developer from the node.js pubsub client confirmed that using the client to pull messages from a Cloud Function is not a supported use case.

替代方法是使用 服务 API.但是，当尝试从订阅中提取所有消息时，REST API它们自己的警告.

The alternative is to use the service APIs. However, the REST APIs have their own caveats when attempting to pull all messages from a subscription.

这篇关于谷歌云 pubsub node.js 客户端与谷歌云功能不兼容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！