卡夫卡喷口的现场分组 [英] Field Grouping for a Kafka Spout

问题描述

可以对卡夫卡喷口发出的元组进行字段分组吗?如果是，那么Storm将如何了解Kafka记录中的字段?

Can field grouping be done on tuples emitted by a kafka spout? If yes, then how does Storm gets to know the fields in a Kafka record?

推荐答案

Kafka Spout像其他任何组件一样声明了其输出字段.我的解释是基于KafkaSpout的当前实现.

Kafka Spout declared its output fields like any other component. My explanation is based on current implementation of KafkaSpout.

在KafkaSpout.java类中，我们看到clarifyOutputFields方法，该方法调用KafkaConfig方案的getOutputFields()方法.

In KafkaSpout.java class we see declareOutputFields method that call getOutputFields() method of KafkaConfig Scheme.

@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
    declarer.declare(_spoutConfig.scheme.getOutputFields());
}

默认情况下，KafkaConfig使用

By default, KafkaConfig uses RawMultiScheme that implements this method in this way.

  @Override
  public Fields getOutputFields() {
    return new Fields("bytes");
  }

那是什么意思?如果您声明用fieldGrouping从KafkaSpout中读取元组的bolt，您就会知道每个包含等价字段字节"的元组都将由同一任务执行.如果要发出任何字段，则应根据需要实施新方案.

So what does it mean?, if you declared bolt which reads tuples from KafkaSpout with fieldGrouping you know that every tuple that contains equals field "bytes" is going to be executed by the same task. If you want to emit any field, you should implement new scheme for your needs.

这篇关于卡夫卡喷口的现场分组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！