监控谷歌发布/订阅提交作业的状态 [英] Monitoring the status of a google pub/sub submitted job

问题描述

我是Google Compute / Google App Engine平台的新手。我目前正在将使用芹菜异步任务的python烧瓶应用程序迁移到Google Compute / Google App Engine平台。但在文档中写道，我应该使用Google Pub / Sub而不是芹菜。在我的应用程序，每当我运行一个异步任务，我有一个页面来监测作业的状态使用相同的原则 http://blog.miguelgrinberg.com/post/using-celery-with-flask 。我已经检查了google pub / sub的文档，但是我不知道如何使用google pub / sub来实现相同的文档。任何人都可以帮助或指向我的方向来实现在谷歌发布/子。

解决方案

您可能能够请使用 psq 进行此操作，其设计看起来像是芹菜。从一般的Cloud Pub / Sub的角度来看，您可以按照以下步骤操作：

1. 为您的状态更新消息创建主题。
   
2. 在要监视其状态的异步任务中，定期发布带有状态的邮件。此讯息将采用您选择的某种格式，表示完成百分比或要显示的特定讯息。 为您的监控页面创建订阅，该页面将接收有关该主题的消息。


3. 在您的监控页面（或者将数据提供给监控页面的后台进程）中，拉取消息。
   
4. 处理消息并更新您的监控页面的作业状态。


5. 确认您收到的信息处理。
   
   
   

   在这个工作流程中要牢记几件事：
   
   ol>
   

6. Cloud Pub / Sub保证至少交付一次。这意味着您可能会多次收到相同的消息。


7. Cloud Pub / Sub不提供任何订购保证。因此，如果您定期发布状态更新，您的用户可能会收到这些错误。对于你的情况，你可能会希望你的消息在你的消息中包含某种时间戳或者严格增加的标识符，以便对每个任务的状态更新进行排序。如果您跟踪收到的最新状态更新，则可以忽略旧的消息并立即进行确认。

I am new to Google Compute/Google App Engine platform. I am currently migrating a python flask application using celery for async tasks to Google Compute/Google App Engine platform. However in the docs it's written I should use Google Pub/Sub instead of celery. In my application whenever I run an async task I have a page to monitor the status of the job using the same principle as http://blog.miguelgrinberg.com/post/using-celery-with-flask. I have checked the documents for google pub/sub, but I am at loss how to implement the same using google pub/sub. Can anybody help or point me to the right direction to implement the same in google pub/sub.

解决方案

You might be able to use psq for this, which is designed to look like celery. From a general Cloud Pub/Sub perspective, you would follow these steps:

1. Create a topic for your status update messages.
2. In the async task whose status you want to monitor, periodically publish a message with the status. This message will be of some format of your choosing that would indicate percentage completion or specific message to display.
3. Create a subscription for your monitoring page that will receive messages on the topic.
4. In your monitoring page (or a background process that will supply the data to your monitoring page), pull messages for the subscription.
5. Process the messages and update the state of your jobs for your monitoring page.
6. Ack the messages you pulled and processed.

A couple of things to keep in mind in this workflow:

1. Cloud Pub/Sub guarantees at-least-once delivery. That means you could potentially receive the same message more than once.
2. Cloud Pub/Sub does not provide any guarantees on ordering. Therefore, if you are periodically publishing status updates, your subscriber could potentially receive these out of order. For your case, you'll probably want your message to include some sort of timestamp or strictly-increasing identifier in your message to sequence your status updates per task. If you keep track of the most recent status update received, then you can disregard older messages and ack them immediately.

这篇关于监控谷歌发布/订阅提交作业的状态的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！