与客户端证书SSL重新协商导致服务器缓冲区溢出 [英] SSL Renegotiation with Client Certificate causes Server Buffer Overflow

问题描述

我有codeD其中使用客户端证书通过HTTPS连接到一个Apache Web服务器并执行文件到服务器的HTTP PUT一个Java客户端应用程序。它正常工作与小文件，但路数崩溃。

Apache服务器日志显示以下内容：

  ...
OpenSSL的：握手：完成
...
更改客户端验证类型将迫使重新谈判
...
填充缓存，最大大小131072字节
...
请求体超过了SSL缓存最大尺寸（131072）
不能缓冲邮件正文以允许SSL重新协商继续进行
...
OpenSSL的：I / O错误，5个字节有望在BIO阅读
（104）连接被对方​​复位：SSL输入滤波器读取失败。
（32）破碎管：core_output_filter：写数据到网络
连接关闭儿童20标准关机
 

在客户端上的响应是：

  java.io.IOException异常：服务器返回的HTTP响应code：401网址
 

我不熟悉这个过程，所以我不知道，如果再谈判是必要的，或在这里如果有什么我可以做，以prevent它。或许我可以让客户等到重新谈判是应用程序发送数据之前完成？下面是客户端code的摘录（除去操作错误）：

 网​​址URL =新的URL（我的网址放在这里）;
        CON =（HttpsURLConnection的）url.openConnection（）;
        con.setSSLSocketFactory（getMyCustomClientCertSocketFactory（））;
        con.setRequestMethod（PUT）;
        con.setDoOutput（真）;
        con.connect（）;
        作家=新OutputStreamWriter（con.getOutputStream（））;
        writer.write（XML）;
        writer.close（）;        parseServerResponse（con.getInputStream（））;
 

我想也许我需要使用较低级别的API一样的SSLSocket和利用HandshakeCompletedListener任何？

我也想知道，如果在Apache SSLVerifyDepth指令有什么关系，为什么重新谈判正在发生。我和值2得到了指令，在每个目录范围内（只有一个上传目录）和Apache手册说，这一下：

  

在每个目录上下文中强制SSL重新协商与
  重新配置客户端验证深度HTTP请求被读取后，
  但在此之前的HTTP响应被发送

由于这里要求的是Java调试输出：

  keyStore在：
密钥仓库类型是：JKS
密钥库提供的是：
初始化密钥库
类型SunX509的初始化的KeyManager
信任存储：C：\\ Program Files文件\\的Java \\ jdk1.6.0_35 \\ JRE \\ lib \\ security中\\ cacerts中
的trustStore类型：JKS
的trustStore供应商是：
初始化信任
添加为受信任的证书：
 ...
SecureRandom的触发播种
完成播种的SecureRandom
***
密钥别名：对发现的关键
链[0] = [
[
...
]
***
SecureRandom的触发播种
完成播种的SecureRandom
允许不安全的重新谈判：假的
允许旧Hello报文：真
在初始握手：真
是安全的重新谈判：假的
%%无缓存的客户端会话
***的ClientHello，使用TLSv1
RandomCookie：...
会话ID：{}
密码套件：[SSL_RSA_WITH_RC4_128_MD5，SSL_RSA_WITH_RC4_128_SHA，TLS_RSA_WITH_AES_128_CBC_SHA，TLS_DHE_RSA_WITH_AES_128_CBC_SHA，TLS_DHE_DSS_WITH_AES_128_CBC_SHA，SSL_RSA_WITH_3DES_EDE_CBC_SHA，SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA，SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA，SSL_RSA_WITH_DES_CBC_SHA，SSL_DHE_RSA_WITH_DES_CBC_SHA，SSL_DHE_DSS_WITH_DES_CBC_SHA，SSL_RSA_EXPORT_WITH_RC4_40_MD5，SSL_RSA_EXPORT_WITH_DES40_CBC_SHA，SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA，SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA，TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
COM pression方法：{0}
***
主要写：使用TLSv1握手，长度= 75
主要写：客户端的SSLv2 Hello报文，长度= 101
主要阅读：使用TLSv1握手，长度= 81
*** ServerHello，使用TLSv1
RandomCookie：...
会话ID：...
密码套件：TLS_RSA_WITH_AES_128_CBC_SHA
COM pression方法：0
扩展renegotiation_info，renegotiated_connection：其中;空＆GT;
***
%%创建：[会话-1，TLS_RSA_WITH_AES_128_CBC_SHA]
** TLS_RSA_WITH_AES_128_CBC_SHA
主要阅读：使用TLSv1握手，长度= 4392
***证书链
链[0] = [
[
...
证书扩展：8
[1]：的ObjectId：1.3.6.1.5.5.7.1.1临界= FALSE
AuthorityInfoAccess [
  [
   accessMethod：...
   accessLocation：URIName：...
   accessMethod：...
   accessLocation：URIName：...
][2]：的ObjectId：2.5.29.35临界= FALSE
执行authorityKeyIdentifier [
KeyIdentifier [
...
]
]
[3]：的ObjectId：2.5.29.19临界= FALSE
BasicConstraints：
  CA：FALSE
  pathLen不能：未定义
]
[4]：的ObjectId：2.5.29.31临界= FALSE
CRLDistributionPoints [
  [DistributionPoint：
     [URIName：...
]]
[5]：的ObjectId：2.5.29.32临界= FALSE
CertificatePolicies [
  [CertificatePolicyId：...
[一个PolicyQualifierInfo：
  qualifierID：...
  预选赛：...
]]]
]
[6]：的ObjectId：2.5.29.37临界= FALSE
ExtendedKeyUsages [
  serverAuth
  clientAuth
]
[7]的ObjectId：2.5.29.15临界= TRUE
密钥使用[
  电子签名
  Key_Encipherment
]
[8]的ObjectId：2.5.29.17临界= FALSE
SubjectAlternativeName [
  DNSNAME：...
]
]
  算法：[SHA1withRSA]
  签名：
...
]
...
***
主要阅读：使用TLSv1握手，长度= 4
*** ServerHelloDone
*** ClientKeyExchange，RSA preMasterSecret，使用TLSv1
主要写：使用TLSv1握手，长度= 518
会议凯基：
preMaster揭秘：
...
连接凯基：
客户端随机数：
...
服务器随机数：
...
主密钥：
...
客户端的MAC写揭秘：
...
服务器的MAC写揭秘：
...
客户端写密钥：
...
服务器写密钥：
...
客户端写四：
...
服务器写四：
...
主要写：使用TLSv1更改密码规范，长度= 1
***完成
verify_data：{18，162，18，251，82，111，87，133，53，240，114，155}
***
主要写：使用TLSv1握手，长度= 48
主要阅读：使用TLSv1更改密码规范，长度= 1
主要阅读：使用TLSv1握手，长度= 48
***完成
verify_data：{46，206，8，40，63，252，99，190，251，183，110，201}
***
%%缓存客户端会话：会话1，TLS_RSA_WITH_AES_128_CBC_SHA]
主要写：使用TLSv1应用程序数据，长度= 256
主要写：使用TLSv1应用程序数据，长度= 32
主要写：使用TLSv1应用程序数据，长度= 16416
主要写：使用TLSv1应用程序数据，长度= 16416
...
主要写：使用TLSv1应用程序数据，长度= 16416
主要写：使用TLSv1应用程序数据，长度= 16416
主要写：使用TLSv1应用程序数据，长度= 512
主要阅读：使用TLSv1应用程序数据，长度= 304
 

由于这里要求的是getMyCustomClientCertSocketFactory源（获得证书和密钥从PEM文件）：

 公共静态SSLSocketFactory的getMyCustomClientCertSocketFactory（字符串pemPath，
        布尔verifyPeer）
        抛出抛出：NoSuchAlgorithmException，FileNotFoundException异常，IOException异常，
        KeyStoreException，CertificateException，UnrecoverableKeyException，
        KeyM​​anagementException，InvalidKeySpecException {
    的SSLContext上下文= SSLContext.getInstance（TLS）;    字节[] = certAndKey IOUtil.fileToBytes（新文件（pemPath））;
    字节[] = certBytes parseDERFromPEM（certAndKey，
            ----- BEGIN CERTIFICATE -----，----- END CERTIFICATE -----）;
    字节[] = keyBytes parseDERFromPEM（certAndKey，
            ----- BEGIN PRIVATE KEY -----，----- END PRIVATE KEY -----）;    x509证书证书= generateX509CertificateFromDER（certBytes）;
    RSAPrivateKey键= generateRSAPrivateKeyFromDER（keyBytes）;    密钥库密钥库= KeyStore.getInstance（JKS）;
    keystore.load（NULL）;
    keystore.setCertificateEntry（认证别名，证书）;
    keystore.setKeyEntry（键别名，重点的changeit.toCharArray（）
            新证书[] {证书}）;    的KeyManagerFactory KMF = KeyManagerFactory.getInstance（SunX509）;
    kmf.init（密钥库的changeit.toCharArray（））;    的KeyManager [] =公里kmf.getKeyManagers（）;    的TrustManager [] TM = NULL;    如果（！verifyPeer）{
        TM =新的TrustManager [] {新TrustyTrustManager（）};
    }    context.init（公里，TM，NULL）;    返回context.getSocketFactory（）;
}
 

解决方案

这似乎是内置的Sun Java的HttpsURLConnection的设备无法处理与客户端证书的场景在服务器友好的方式（大HTTP PUT即不会溢出服务器SSL重新谈判缓冲）。

我要检查什么卷曲在做，看看什么是服务器友好的意思，而原来有一个HTTP 1.1头名为期待，这卷曲值为100继续（见规格的 http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html ＃sec14.20 ）。这个头基本上是说：我已经有了一个巨大的有效载荷，但在此之前我把它请让我知道如果你能处理它。这使端点时间重新协商客户端证书发送有效载荷之前。

在太阳HttpURLConnection的实施看来这头是不允许的，而实际上是在限制标题列表;这意味着，即使你用HttpUrlConnection.setRequestProperty方法标题实际上没有发送到服务器设置。您可以覆盖受限的标题与系统属性sun.net.http.allowRestrictedHeaders，但随后的客户端只是因为Sun实现套接字异常崩溃，不知道如何处理这部分协议。

有趣的是，似乎在OpenJDK的Java实现不支持这个头。此外，Apache的HTTP客户端库支持这个头（ http://hc.apache.org/ ）;我实现了一个测试程序与Apache HTTP客户端库，它可以使用客户端证书和期望头成功地执行和大文件的HTTP PUT请求。

要回顾一下，该解决方案是：

1. 设置Apache的SSLRenegBufferSize指令数量巨大（如64MB）。默认值是128K。该解决方案可以创建服务风险拒绝


2. 配置始终需要客户端证书，而不是在其中只有少数目录需要它的主机。这将避免重新谈判。这不是我的方案一个很好的选择，因为大多数用户都是匿名或用户名/密码认证。只有对文件的程序上传单个上传目录。我们必须创建自己的SSL证书一个新的虚拟主机只为这一个目录。


3. 使用它支持HTTP 1.1期望的头一个客户端。不幸的是在Sun Java不支持这个开箱。必须使用第三方如的Apache HTTP组件客户端库或使用Java套接字API推出自己的解决方案。


4. 利用HTTP 1.1永久连接（流水线与保活）由最初发布不具有大​​的有效载荷的HTTP请求，但会导致发生重新谈判，然后重新用于HTTP PUT的连接。理论上，客户端应该能够发出的上传目录的HTTP头或选项，然后重复使用相同的连接做PUT。为了这个工作持久连接池很可能只需要包含一个连接，以避免吸一连接，然后正在发行另一个用于PUT。但是，它似乎并不像HttpURLConnection的类将保持/重用涉及客户端证书或SSL，因为我一直无法获得该解决方案的工作持久连接。见（<一个href=\"http://stackoverflow.com/questions/9943351/httpsurlconnection-and-keep-alive\">HttpsUrlConnection并保持活动）。

I've coded a Java client application which connects to an Apache web server over HTTPS using a client certificate and performs an HTTP PUT of a file to the server. It works fine with small files, but crashes with large ones.

The Apache server log shows the following:

...
OpenSSL: Handshake: done
...
Changed client verification type will force renegotiation
...
filling buffer, max size 131072 bytes
...
request body exceeds maximum size (131072) for SSL buffer
could not buffer message body to allow SSL renegotiation to proceed
...    
OpenSSL: I/O error, 5 bytes expected to read on BIO
(104)Connection reset by peer: SSL input filter read failed.
(32)Broken pipe: core_output_filter: writing data to the network
Connection closed to child 20 with standard shutdown

The response on the client is:

java.io.IOException: Server returned HTTP response code: 401 for URL

I'm not familiar with this process so I'm not sure if renegotiation is necessary here or if there is something I can do to prevent it. Or perhaps I can have the client wait until the renegotiation is complete before sending application data? Here is an excerpt of the client code (error handling removed):

        URL url = new URL("my url goes here");
        con = (HttpsURLConnection) url.openConnection();
        con.setSSLSocketFactory(getMyCustomClientCertSocketFactory());
        con.setRequestMethod("PUT");
        con.setDoOutput(true);
        con.connect();
        writer = new OutputStreamWriter(con.getOutputStream());
        writer.write(xml);
        writer.close();

        parseServerResponse(con.getInputStream());

I'm thinking maybe I need to use a lower level API like SSLSocket and leverage the HandshakeCompletedListener?

I'm also wondering if the Apache SSLVerifyDepth directive has anything to do with why a renegotiation is occurring. I've got the directive in a per-directory context (only one upload directory) with value 2 and The Apache manual says this about it:

In per-directory context it forces a SSL renegotation with the reconfigured client verification depth after the HTTP request was read but before the HTTP response is sent.

As requested here is the Java debugging output:

keyStore is : 
keyStore type is : jks
keyStore provider is : 
init keystore
init keymanager of type SunX509
trustStore is: C:\Program Files\Java\jdk1.6.0_35\jre\lib\security\cacerts
trustStore type is : jks
trustStore provider is : 
init truststore
adding as trusted cert:
 ...
trigger seeding of SecureRandom
done seeding SecureRandom
***
found key for : key-alias
chain [0] = [
[
...
]
***
trigger seeding of SecureRandom
done seeding SecureRandom
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  ...
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: TLSv1 Handshake, length = 81
*** ServerHello, TLSv1
RandomCookie:  ...
Session ID:  ...
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
%% Created:  [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
** TLS_RSA_WITH_AES_128_CBC_SHA
main, READ: TLSv1 Handshake, length = 4392
*** Certificate chain
chain [0] = [
[
...
Certificate Extensions: 8
[1]: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: ...
   accessLocation: URIName: ...
   accessMethod: ...
   accessLocation: URIName: ...
]

[2]: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
...
]
]
[3]: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:false
  PathLen: undefined
]
[4]: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
  [DistributionPoint:
     [URIName: ...
]]
[5]: ObjectId: 2.5.29.32 Criticality=false
CertificatePolicies [
  [CertificatePolicyId: ...
[PolicyQualifierInfo: [
  qualifierID: ...
  qualifier: ...
]]  ]
]
[6]: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
  clientAuth
]
[7]: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
]
[8]: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: ...
]
]
  Algorithm: [SHA1withRSA]
  Signature:
...
]
...
***
main, READ: TLSv1 Handshake, length = 4
*** ServerHelloDone
*** ClientKeyExchange, RSA PreMasterSecret, TLSv1
main, WRITE: TLSv1 Handshake, length = 518
SESSION KEYGEN:
PreMaster Secret:
...
CONNECTION KEYGEN:
Client Nonce:
...
Server Nonce:
...
Master Secret:
...
Client MAC write Secret:
...
Server MAC write Secret:
...
Client write key:
...
Server write key:
...
Client write IV:
...
Server write IV:
...
main, WRITE: TLSv1 Change Cipher Spec, length = 1
*** Finished
verify_data:  { 18, 162, 18, 251, 82, 111, 87, 133, 53, 240, 114, 155 }
***
main, WRITE: TLSv1 Handshake, length = 48
main, READ: TLSv1 Change Cipher Spec, length = 1
main, READ: TLSv1 Handshake, length = 48
*** Finished
verify_data:  { 46, 206, 8, 40, 63, 252, 99, 190, 251, 183, 110, 201 }
***
%% Cached client session: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]
main, WRITE: TLSv1 Application Data, length = 256
main, WRITE: TLSv1 Application Data, length = 32
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 16416
...
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 16416
main, WRITE: TLSv1 Application Data, length = 512
main, READ: TLSv1 Application Data, length = 304 

As requested here is the getMyCustomClientCertSocketFactory source (obtains certificate and key from a PEM file):

public static SSLSocketFactory getMyCustomClientCertSocketFactory(String pemPath,
        boolean verifyPeer)
        throws NoSuchAlgorithmException, FileNotFoundException, IOException,
        KeyStoreException, CertificateException, UnrecoverableKeyException,
        KeyManagementException, InvalidKeySpecException {
    SSLContext context = SSLContext.getInstance("TLS");

    byte[] certAndKey = IOUtil.fileToBytes(new File(pemPath));
    byte[] certBytes = parseDERFromPEM(certAndKey,
            "-----BEGIN CERTIFICATE-----", "-----END CERTIFICATE-----");
    byte[] keyBytes = parseDERFromPEM(certAndKey,
            "-----BEGIN PRIVATE KEY-----", "-----END PRIVATE KEY-----");

    X509Certificate cert = generateX509CertificateFromDER(certBytes);
    RSAPrivateKey key = generateRSAPrivateKeyFromDER(keyBytes);

    KeyStore keystore = KeyStore.getInstance("JKS");
    keystore.load(null);
    keystore.setCertificateEntry("cert-alias", cert);
    keystore.setKeyEntry("key-alias", key, "changeit".toCharArray(),
            new Certificate[]{cert});

    KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
    kmf.init(keystore, "changeit".toCharArray());

    KeyManager[] km = kmf.getKeyManagers();

    TrustManager[] tm = null;

    if (!verifyPeer) {
        tm = new TrustManager[]{new TrustyTrustManager()};
    }

    context.init(km, tm, null);

    return context.getSocketFactory();
}

解决方案

It would seem that the HttpsUrlConnection facility built into Sun Java cannot handle the large HTTP PUT with client certificate scenario in a server friendly way (i.e. without overflowing the servers SSL renegotiate buffer).

I examined what curl was doing to see what "server friendly meant", and it turns out there is an HTTP 1.1 header named "Expect", which curl sends with value "100-continue" (see spec http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.20). This header essentially says "I've got a huge payload, but before I send it please let me know if you can handle it". This gives the endpoints time to renegotiate the client certificate before the payload is sent.

In the Sun HttpUrlConnection implementation it seems this header is not allowed, and is actually in the restricted headers list; meaning even if you set it with the HttpUrlConnection.setRequestProperty method the header is not actually sent to the server. You can override the restricted headers with the system property sun.net.http.allowRestrictedHeaders, but then the client just crashes with a socket exception since the Sun implementation doesn't know how to handle this part of the protocol.

Interestingly it seems that the OpenJDK implementation of Java does support this header. Also, the Apache HTTP Client library supports this header (http://hc.apache.org/); I've implemented a test program with the Apache HTTP client library and it can successfully perform and HTTP PUT request of a large file using a client certificate and the Expect header.

To recap, the solutions are:

1. Set the Apache SSLRenegBufferSize directive to a huge number (like 64MB). The default is 128K. This solution may create a denial of service risk
2. Configure a host that always requires client certificates, as opposed to one in which only a few directories require it. This will avoid renegotiation. This isn't a good option in my scenario because the majority of users are anonymous or username/password authenticated. There is only a single upload directory for programmatic upload of files. We would have to create a new virtual host with its own SSL certificate just for this one directory.
3. Use a client which supports the HTTP 1.1 Expect header. Unfortunately the Sun Java does not support this out of the box. Must use third party such as Apache HTTP Component Client library or roll your own solution using Java socket API.
4. Leverage HTTP 1.1 persistent connections (pipelining with keep-alive) by initially issuing an HTTP request that doesn't have a big payload, but causes the renegotiation to occur, then reuse the connection for the HTTP PUT. In theory the client should be able to issue an HTTP HEAD or OPTIONS on the upload directory and then reuse the same connection to do the PUT. In order for this to work the persistent connection pool would probably need to only contain one connection to avoid "priming" one connection and then being issued another for the PUT. However, it doesn't seem like the HttpUrlConnection class will keep/reuse persistent connections involving client certificates or SSL because I've been unable to get this solution to work. See (HttpsUrlConnection and keep-alive).

这篇关于与客户端证书SSL重新协商导致服务器缓冲区溢出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！