无法在CentOS 7上启动kafka经纪人 [英] unable to bring up kafka broker on CentOS 7

问题描述

Kafka的新手，我尝试在Centos服务器上安装kafka并出现以下错误，这是全新安装，但我没有进行任何配置即可将数据推送到kafka生产者

New to Kafka ,I tried installing kafka on Centos server with the below error ,it's a fresh installation and I haven't configured anything to push data to the kafka producer

[2016-12-12 18:20:18,531] INFO zookeeper state changed (SyncConnected)     (org.I0Itec.zkclient.ZkClient)
[2016-12-12 18:20:18,601] INFO Loading logs. (kafka.log.LogManager)
[2016-12-12 18:20:18,609] INFO Logs loading complete. (kafka.log.LogManager)
[2016-12-12 18:20:18,676] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2016-12-12 18:20:18,678] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2016-12-12 18:20:18,734] INFO Awaiting socket connections on 172.16.23.13:9092. (kafka.network.Acceptor)
[2016-12-12 18:20:18,770] INFO [Socket Server on Broker 0], Started 1 acceptor threads (kafka.network.SocketServer)
[2016-12-12 18:20:18,796] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:20:18,798] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:20:18,876] INFO Creating /controller (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:20:18,887] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:20:18,888] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2016-12-12 18:20:18,980] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:20:18,981] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:20:18,995] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.GroupCoordinator)
[2016-12-12 18:20:18,996] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.GroupCoordinator)
[2016-12-12 18:20:19,001] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 10 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2016-12-12 18:20:19,016] INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2016-12-12 18:20:19,017] INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2016-12-12 18:20:19,024] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2016-12-12 18:20:19,053] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:20:19,056] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:20:19,058] INFO Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT -> EndPoint(172.16.23.13,9092,PLAINTEXT) (kafka.utils.ZkUtils)
[2016-12-12 18:20:19,079] INFO Kafka version : 0.10.0.0 (org.apache.kafka.common.utils.AppInfoParser)
[2016-12-12 18:20:19,079] INFO Kafka commitId : b8642491e78c5a13 (org.apache.kafka.common.utils.AppInfoParser)
[2016-12-12 18:20:19,080] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
[2016-12-12 18:20:19,082] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2016-12-12 18:20:20,170] ERROR Processor got uncaught exception. (kafka.network.Processor)
java.lang.OutOfMemoryError: Java heap space
    at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
    at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
    at    org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:93)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:323)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
    at kafka.network.Processor.poll(SocketServer.scala:472)
    at kafka.network.Processor.run(SocketServer.scala:412)
    at java.lang.Thread.run(Thread.java:745)
[2016-12-12 18:20:20,180] WARN Unexpected error from /172.16.23.13; closing connection (org.apache.kafka.common.network.Selector)
java.lang.NullPointerException
    at org.apache.kafka.common.network.NetworkReceive.complete(NetworkReceive.java:67)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:136)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:323)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
    at kafka.network.Processor.poll(SocketServer.scala:472)
    at kafka.network.Processor.run(SocketServer.scala:412)
    at java.lang.Thread.run(Thread.java:745)    

如果增加内存，则会出现以下错误.

[2016-12-12 18:18:20,789] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2016-12-12 18:18:20,844] INFO Awaiting socket connections on 172.16.23.13:9092. (kafka.network.Acceptor)
[2016-12-12 18:18:20,848] INFO [Socket Server on Broker 0], Started 1 acceptor threads (kafka.network.SocketServer)
[2016-12-12 18:18:20,905] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:18:20,908] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:18:20,988] INFO Creating /controller (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:18:20,999] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:18:21,000] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2016-12-12 18:18:21,097] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:18:21,099] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-12 18:18:21,112] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.GroupCoordinator)
[2016-12-12 18:18:21,113] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.GroupCoordinator)
[2016-12-12 18:18:21,117] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 10 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2016-12-12 18:18:21,133] INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2016-12-12 18:18:21,134] INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2016-12-12 18:18:21,141] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2016-12-12 18:18:21,168] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:18:21,172] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2016-12-12 18:18:21,174] INFO Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT -> EndPoint(172.16.23.13,9092,PLAINTEXT)  (kafka.utils.ZkUtils)
[2016-12-12 18:18:21,194] INFO Kafka version : 0.10.0.0 (org.apache.kafka.common.utils.AppInfoParser)
[2016-12-12 18:18:21,194] INFO Kafka commitId : b8642491e78c5a13 (org.apache.kafka.common.utils.AppInfoParser)
[2016-12-12 18:18:21,195] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
[2016-12-12 18:18:21,200] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2016-12-12 18:18:30,013] WARN Unexpected error from /172.16.23.13; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1347375956 larger than 104857600)
    at  org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:91)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:323)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:283)
    at kafka.network.Processor.poll(SocketServer.scala:472)
    at kafka.network.Processor.run(SocketServer.scala:412)
    at java.lang.Thread.run(Thread.java:745)

推荐答案

某些其他进程正在将数据发送到kafka.在我的情况下，因为kapacitor也侦听9092端口，所以influxdb正在将数据发送到kafka端口.因此influxdb会将数据发送到kafka.一旦我停止了influxdb，就再也没有例外了.

Some other process is sending data to kafka. In my case influxdb was sending data to kafka port because kapacitor also listens on 9092 port. so influxdb is sending data to kafka. Once i stopped influxdb no more exceptions.

这篇关于无法在CentOS 7上启动kafka经纪人的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！