Kafka Streams是否有超时要求处理多长时间? [英] Do Kafka Streams have a timeout on how long processing can take?

问题描述

我正在使用在 map/foreach 期间调用外部系统的Kafka Streams. map 或 foreach 可以花费多长时间?

I'm using Kafka Streams that call external systems during map/foreach. Is there any timeout on how long map or foreach can take?

在长时间(例如几小时)内是否有任何需要阻止的警告?

Are there any caveats to blocking for a long time (say hours)?

推荐答案

没有超时适用于 map/foreach .

但是，由于Kafka Streams在内部使用 KafkaConsumer 和 KafkaProducer ，因此它们的所有超时都适用(例如， max.poll.interval.ms ).您当然可以对其进行相应的配置，但是不建议长时间阻止对外部系统的调用.

However, as Kafka Streams uses KafkaConsumer and KafkaProducer internally, all their timeouts apply (eg. max.poll.interval.ms). You can of course configure them accordingly, but it's not recommended to do long blocking calls to external systems.

将具有很多优点，即使用Kafka Connect将外部服务中的数据加载到主题中，并在应用程序中以 KTable 读取此数据，然后执行流表联接，而不是一张地图.这种方法使您的Streams应用程序与外部系统脱钩，从而使整体体系结构更加健壮.

It would have many advantages, to load the data from you external service into a topic using Kafka Connect, and read this data as a KTable in your application a do a stream-table join instead of a map. This approach decouples your Streams application from the external system and thus makes the overall architecture more robust.

这篇关于Kafka Streams是否有超时要求处理多长时间?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！