Kafka 不会从 PEM 证书开始 [英] Kafka won't start with PEM certificate

问题描述

我发现 Kafka 2.7.0 支持 PEM 证书，因此我决定尝试使用 DigiCert SSL 证书设置代理.我使用了新选项，并且我在 KIP-651.但我收到错误:

I found that Kafka 2.7.0 supports PEM certificates and I decided to try setting up the broker with DigiCert SSL certificate. I used new options and I did everything like in example in KIP-651. But I get the error:

[2021-01-20 17:54:55,787] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.config.ConfigException: Invalid value javax.net.ssl.SSLHandshakeException: no cipher suites in common for configuration A client SSLEngine created with the provided settings can't connect to a server SSLEngine created with those settings.
        at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:98)
        at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:72)
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
        at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:97)
        at kafka.network.Processor.<init>(SocketServer.scala:790)
        at kafka.network.SocketServer.newProcessor(SocketServer.scala:415)
        at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors$1(SocketServer.scala:288)
        at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:287)
        at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1(SocketServer.scala:254)
        at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1$adapted(SocketServer.scala:251)
        at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
        at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
        at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:251)
        at kafka.network.SocketServer.startup(SocketServer.scala:125)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:303)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
        at kafka.Kafka$.main(Kafka.scala:82)
        at kafka.Kafka.main(Kafka.scala)

openssl x509 -in certificate.pem -text:

Certificate:
    ...
    Signature Algorithm: ecdsa-with-SHA384
        ...
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)

我尝试使用不同版本的 Oracle JDK 和 OpenJDK，但没有效果.

I've tried using different versions of Oracle JDK and OpenJDK, but no effect.

我也尝试使用选项设置证书:

I also tried to set certificate with options:

ssl.keystore.type=PEM
ssl.keystore.location=/opt/kafka/certs/certificate.pem
ssl.key.password=null

我又遇到了一个新错误:

And I got а new error:

[2021-01-23 20:33:21,552] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.errors.InvalidConfigurationException: Failed to load PEM SSL keystore /opt/kafka/certs/certificate.pem
Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
Caused by: java.io.IOException: overrun, bytes = 111
    at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:92)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.privateKey(DefaultSslEngineFactory.java:512)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.createKeyStoreFromPem(DefaultSslEngineFactory.java:462)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.<init>(DefaultSslEngineFactory.java:435)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.load(DefaultSslEngineFactory.java:412)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:349)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedPemStore.<init>(DefaultSslEngineFactory.java:405)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:293)
    at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161)
    at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:136)
    at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:93)
    at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:72)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
    at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:97)
    at kafka.network.Processor.<init>(SocketServer.scala:790)
    at kafka.network.SocketServer.newProcessor(SocketServer.scala:415)
    at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors$1(SocketServer.scala:288)
    at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:287)
    at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1(SocketServer.scala:254)
    at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1$adapted(SocketServer.scala:251)
    at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
    at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
    at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:251)
    at kafka.network.SocketServer.startup(SocketServer.scala:125)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:303)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:44)
    at kafka.Kafka$.main(Kafka.scala:82)
    at kafka.Kafka.main(Kafka.scala)

但是如果将此证书转换为 jks:

But if convert this certificate to jks:

openssl pkcs12 -export -in certificate.pem -out certificate.p12
keytool -importkeystore -srckeystore certificate.p12 -srcstoretype pkcs12 -destkeystore certificate.jks

echo 'ssl.keystore.location=/opt/kafka/certs/certificate.jks' >>server.properties
echo 'ssl.keystore.password=password' >>server.properties

Broker 工作正常.

Broker works correctly.

这可能是一个错误还是我做错了什么?完整日志这里

Could this be a bug or am I doing anything wrong? Full log here

推荐答案

我认为这可能是因为您使用的私钥是使用 PBES2 方案加密的.您可以使用 OpenSSL 转换原始密钥并改用 PBES1:

I think this might be because the private key you are using is encrypted with a PBES2 scheme. You can use OpenSSL to convert the original key and use PBES1 instead:

openssl pkcs8 -in old_kafka.key -passout "pass:password" -topk8 -v1 PBE-SHA1-3DES -out kafka.key

您可以找到有关哪些 PKCS#5 加密算法可用的更多信息OpenSSL 的 PKCS#8 手册页中的 PBES1 和 PBES2.

You can find more info of which PKCS#5 encryption algorithms are available with PBES1 and PBES2 in OpenSSL's PKCS#8 man page.

此外，这个问题中提到了 OpenJDK 中的 PBES2 支持.

Also, PBES2 support in OpenJDK is mentioned in this issue.

这篇关于Kafka 不会从 PEM 证书开始的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！