读取或写入主题时记录错误消息 [英] Logging Error message while reading or writing to Topics

问题描述

在读取或写入主题时如何记录错误消息.我们将使用Apache Beam API来读取或写入主题.因此，我产生了任何异常，我们该如何记录它.我可以将数据发送到其他主题吗?

How do you log error messages while reading or writing to topic. We would be using Apache Beam API to read or write to topic. So I any exception is generated how do we log it. Can I send my data to other topic?

PubsubIO.writeMessages() PubsubIO.readMessages()

PubsubIO.writeMessages() PubsubIO.readMessages()

我可以编写此DoFn并添加调试日志吗? log.debug(将json消息发布到pubsub主题"); PubsubIO.Write消息= PubsubIO.writeMessages().to(pipelineOptions.getPubsubEnpEventTopic()); log.debug(邮件已发布到pubsub");

Can I write this DoFn and add debug logs log.debug("Publishing json message to pubsub topic"); PubsubIO.Write message = PubsubIO.writeMessages().to(pipelineOptions.getPubsubEnpEventTopic()); log.debug("Message published to pubsub");

推荐答案

从pubsub读取时有两种方法"withCoderAndParseFn"，而在写入pubsub时有"withFormatFn"方法.这些功能对于在读取或写入时解析消息很有用.也许您可以通过在其中应用自己的逻辑来使用它们，例如登录到stackdriver或将错误数据推送到pubsub.

There are two methods 'withCoderAndParseFn' while reading from pubsub and 'withFormatFn' while writing to pubsub. These function are useful to parse messages while reading or writing. May be you can use them by applying your own logic in there like logging into stackdriver or pushing error data to pubsub.

更多信息，请点击这里 https://www.codota.com/web/assistant/code/rs/5c6563f8138b620001f3aa7c#L470

More you can find here https://www.codota.com/web/assistant/code/rs/5c6563f8138b620001f3aa7c#L470

PubsubIO.<t>read().withCoderAndParseFn(coder, new ParsePayloadUsingCoder<>(coder));

private static class ParsePayloadUsingCoder<t> extends SimpleFunction<pubsubmessage, t=""> {
    private Coder<t> coder;

    public ParsePayloadUsingCoder(Coder<t> coder) {
      this.coder = coder;
    }

    @Override
    public T apply(PubsubMessage input) {
      try {
        return CoderUtils.decodeFromByteArray(coder, input.getPayload());
      } catch (CoderException e) {
        throw new RuntimeException("Could not decode Pubsub message", e);
      }
    }
  }

这篇关于读取或写入主题时记录错误消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！