JDBC Sink配置选项batch.size [英] JDBC Sink Configuration Options batch.size

问题描述

来自 https://docs.confluent.io/3.1.1/connect/connect-jdbc/docs/sink_config_options.html#jdbc-sink-configuration-options

Specifies how many records to attempt to batch together for insertion into the destination table, when possible.

Type: int
Default: 3000
Valid Values: [0,…]
Importance: medium

因此，这是来自Confluent网站.

So, this is from Confluent site.

重要性为中，默认值为3000.如果我要KAFKA，该怎么办 即使有人说，每30秒更改一次，只有27条KAFKA消息 为主题?什么是在计算机上进行处理的默认设置? 每个经过的时间?我们都知道这样做是可以解决的，因为我们可以运行许多示例，仅将1条记录从mySQL传递到SQLServer，但是我找不到用于基于时间的处理的参数值.我可以影响它吗?

Importance is medium, default is 3000. What if I want the KAFKA changes every 30 secs even if there are say, only 27 KAFKA messages for the topic? What is default setting in which processing occurs on a per elapsed time basis? We all know this is catered for as we can run many examples just passing 1 records from, say mySQL to SQLServer, but I cannot find the parameter value for time based processing. Can I influence it?

https://github.com/confluentinc/kafka-connect-jdbc /issues/290 也注意到了这一点.那里有一些有趣的东西.

https://github.com/confluentinc/kafka-connect-jdbc/issues/290 Noted this as well. Some interesting stuff there.

推荐答案

我认为您应该关注尽可能"一词.

I think you should focus on the words "when possible"

consumer.max.poll.records将始终从Kafka抓取那么多记录.轮询完成后，JDBC接收器将根据需要构建任意数量的批处理，直到在consumer.max.poll.interval.ms

consumer.max.poll.records will always grab up to that many records from Kafka. Once a poll is complete, the JDBC sink will build as many batches as needed until the next consumer poll is called within consumer.max.poll.interval.ms

这篇关于JDBC Sink配置选项batch.size的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！