批量处理PubSub请求 [英] Batching PubSub requests

问题描述

用于批处理pubsub请求的NODEJS示例代码如下:

The NODEJS example code for batching pubsub requests looks like this:

// Imports the Google Cloud client library
const PubSub = require(`@google-cloud/pubsub`);

// Creates a client
const pubsub = new PubSub();

/**
 * TODO(developer): Uncomment the following lines to run the sample.
 */
// const topicName = 'your-topic';
// const data = JSON.stringify({ foo: 'bar' });
// const maxMessages = 10;
// const maxWaitTime = 10000;

// Publishes the message as a string, e.g. "Hello, world!" or JSON.stringify(someObject)
const dataBuffer = Buffer.from(data);

pubsub
  .topic(topicName)
  .publisher({
    batching: {
      maxMessages: maxMessages,
      maxMilliseconds: maxWaitTime,
    },
  })
  .publish(dataBuffer)
  .then(results => {
    const messageId = results[0];
    console.log(`Message ${messageId} published.`);
  })
  .catch(err => {
    console.error('ERROR:', err);
  });

对我来说，目前尚不清楚如何使用此示例同时发布多条消息.有人可以解释如何调整此代码，以便可以同时发布多条消息吗?

For me it is not clear how to publish multiple messages simultaneously using this example. Could someone explain how to adjust this code so it can be used to publish multiple messages simultaneously?

推荐答案

如果要批量处理邮件，则需要保持发布者的身份并多次调用publish.例如，您可以将代码更改为以下内容:

If you wanted to batch messages, then you'd need to keep hold of the publisher and call publish on it multiple times. For example, you could change the code to something like this:

// Imports the Google Cloud client library
const PubSub = require(`@google-cloud/pubsub`);

// Creates a client
const pubsub = new PubSub();


const topicName = 'my-topic';
const maxMessages = 10;
const maxWaitTime = 10000;
const data1 = JSON.stringify({ foo: 'bar1' });
const data2 = JSON.stringify({ foo: 'bar2' });
const data3 = JSON.stringify({ foo: 'bar3' });

const publisher = pubsub.topic(topicName).publisher({
    batching: {
      maxMessages: maxMessages,
      maxMilliseconds: maxWaitTime,
    },
  })

function handleResult(p) {
  p.then(results => {
    console.log(`Message ${results} published.`);
  })
  .catch(err => {
    console.error('ERROR:', err);
  });
}

// Publish three messages
handleResult(publisher.publish(Buffer.from(data1)));
handleResult(publisher.publish(Buffer.from(data2)));
handleResult(publisher.publish(Buffer.from(data3)));

邮件的批处理由maxMessages和maxMilliseconds属性处理.前者表示要包含在批处理中的最大邮件数.后者表示等待发布批处理的最大毫秒数.这些属性与发布延迟权衡较大的批次(可以更有效).如果要快速发布许多消息，则maxMilliseconds属性不会有太大影响；一旦准备好发送十条消息，客户端库就会向Cloud Pub/Sub服务发出发布请求.但是，如果发布是零星的或速度较慢，则可能会在出现十条消息之前发送一批消息.

Batching of messages is handled by the maxMessages and maxMilliseconds properties. The former indicates the maximum number of messages to include in a batch. The latter indicates the maximum number of milliseconds to wait to publish a batch. These properties trade off larger batches (which can be more efficient) with publish latency. If you are publishing many messages rapidly, then the maxMilliseconds property won't have much effect; as soon as ten messages are ready to go, the client library will make a publish request to the Cloud Pub/Sub service. However, if publishing is sporadic or slow, then a batch of messages might be sent before there are ten messages.

在上面的示例代码中，我们在三则消息上调用publish.这不足以填充一批并将其发送.因此，在第一次调用publish的10,000毫秒后，这三则消息将批量发送到Cloud Pub/Sub.

In the example code above, we call publish on three messages. This is not enough to fill up a batch and send it. Therefore, 10,000 milliseconds after the first call to publish, the three messages will be sent as a batch to Cloud Pub/Sub.

这篇关于批量处理PubSub请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！