如何通过HTTPS POST正确发送二进制数据？ [英] How to correctly send binary data over HTTPS POST?

问题描述

我从客户端（Debian 6.0.3）发送二进制数据到服务器（Windows Server 2003）。要绕过大多数防火墙，我使用 HTTPS POST 。客户端和服务器是使用 Boost.Asio 和 OpenSSL 实现的。首先，我实现了最简单的可能版本，工作正常。

HTTP标头：

  POST / HTTP / 1.1 
 User-Agent：我的自定义客户端v.1 
 
 [二进制数据] 
  

（ [binary data] >

然后，在另一台客户端计算机上失败（连接到同一台服务器）。该行为不稳定。连接始终建立良好（端口443）。大多数时候我通过SSL握手精细，但服务器接收没有数据（几乎没有数据，有时一包或两个实际上接收）。有时我收到SSL握手错误短读。有时我收到无效的数据。

客户端连接到服务器，握手，发送HTTP POST头，然后无限发送二进制数据，直到出现错误。对于测试，我使用自定义生成的SSL证书。

服务器代码：

 命名空间ssl = boost :: asio :: ssl; 
 ssl :: context context（io_service，ssl :: context :: sslv23）; 
 context.set_options（ssl :: context :: default_workarounds | ssl :: context :: no_sslv2）; 
 context.use_certificate_chain_file（server.pem）; 
 context.use_private_key_file（server.pem，boost :: asio :: ssl :: context :: pem）; 
 
 ssl :: stream< tcp :: socket> socket（io_service，context）; 
 
 //标准连接接受
 
 socket.async_handshake（ssl :: stream_base :: server，...）; 
 ... 
 boost :: asio :: async_read_until（socket，POST_header，\r\\\
\r\\\
，...）; 
 ... 
  

客户代码：

  ssl :: context context（io_service，ssl :: context :: sslv23）; 
 context.load_verify_file（server.crt）; 
 socket.reset（new ssl :: stream< tcp :: socket>（io_service，context））; 
 socket-> set_verify_mode（ssl :: verify_none）; 
 
 //标准连接
 
 socket.async_handshake（ssl :: stream_base :: client，...）; 
 ... 
  

（错误处理与相关代码一起省略）

如您所见，这是最简单的SSL连接。哪里不对？原因是防火墙吗？

我在同一个443端口尝试了简单的TCP w / o SSL，这很好。

编辑：

尝试添加Content-Type：application / octet-stream p>

编辑2：

通常我收到HTTP POST标头。然后我发送数据块为 chunk-size（4字节）chunk（chunk-size字节）... 。服务器接收 chunk-size 很好，但没有什么。客户端不会通知服务器问题（没有错误），并继续发送数据。有时候，服务器可以接收一两块，有时它会收到无​​效的 chunk-size ，但大多数时候只是没有。

编辑3：

比较客户端和服务器上捕获的流量，没有发现任何差异。

解决方案

我从这个问题开始就被误导了。缩小到令人惊讶的细节：

如果我在Boost v.1.48中使用Boost.Asio多缓冲区（此时是最近的一个），通过SSL套接字发送失败， 。示例：

  //要发送的数据，协议为[数据包大小：4字节] [数据包：packet_size字节] 
 std :: vector< char> packet = ...; 
 uint32_t packet_size = packet.size（）; 
 //准备缓冲区
 boost :: array< boost :: asio :: const_buffer，2> bufs = {{boost :: asio :: buffer（& packet_size，sizeof（packet_size）），boost :: asio :: buffer（packet）}}; 
 //通过单个调用发送多个缓冲区
 boost :: asio :: async_write（socket，bufs，...）; 
  

分别发送 packet_size 和 packet 在此示例中解决问题。我远远没有调用任何可疑的行为作为一个bug，特别是如果它与Boost库相关。但这个真的看起来像一个错误。试用Boost v.1.47 - 工作正常。尝试与通常的TCP套接字（不是SSL一） - 工作正常。在Linux和Windows上也是如此。

我会在Asio邮件列表中找到有关此问题的任何报告，如果没有找到则会报告。

解决方案

我从这个问题的开始被误导了。缩小到令人惊讶的细节：

如果我在Boost v.1.48中使用Boost.Asio多缓冲区（此时是最近的一个），通过SSL套接字发送失败， 。示例：

  //要发送的数据，协议为[数据包大小：4字节] [数据包：packet_size字节] 
 std :: vector< char> packet = ...; 
 uint32_t packet_size = packet.size（）; 
 //准备缓冲区
 boost :: array< boost :: asio :: const_buffer，2> bufs = {{boost :: asio :: buffer（& packet_size，sizeof（packet_size）），boost :: asio :: buffer（packet）}}; 
 //通过单个调用发送多个缓冲区
 boost :: asio :: async_write（socket，bufs，...）; 
  

分别发送 packet_size 和 packet 在此示例中解决问题。我远远没有调用任何可疑的行为作为一个bug，特别是如果它与Boost库相关。但这个真的看起来像一个错误。试用Boost v.1.47 - 工作正常。尝试与通常的TCP套接字（不是SSL一） - 工作正常。在Linux和Windows上也是如此。

我会在Asio邮件列表中找到有关此问题的任何报告，如果没有找到，则会报告。

I send binary data from client (Debian 6.0.3) to server (Windows Server 2003). To bypass most firewalls I use HTTPS POST. Client and server are implemented using Boost.Asio and OpenSSL. First I implemented the simplest possible version and it worked fine.

HTTP Header:

POST / HTTP/1.1
User-Agent: my custom client v.1

[binary data]

([binary data] is not base64 encoded if this matters)

Then, on another client machine it failed (connected to the same server machine). The behavior is not stable. Connection always is established fine (port 443). Most time I pass SSL handshake fine but server receive no data (almost no data, sometimes a packet or two are actually received). Sometimes I receive SSL handshake error "short read". Sometimes I receive invalid data.

Client connects to server, handshakes, sends HTTP POST header and then infinitely sends binary data until something wrong hapenned. For test I use custom generated SSL certificate.

Server code:

namespace ssl = boost::asio::ssl;
ssl::context context(io_service, ssl::context::sslv23);
context.set_options(ssl::context::default_workarounds | ssl::context::no_sslv2);
context.use_certificate_chain_file("server.pem");
context.use_private_key_file("server.pem", boost::asio::ssl::context::pem);

ssl::stream<tcp::socket> socket(io_service, context);

// standard connection accepting

socket.async_handshake(ssl::stream_base::server, ...);
...
boost::asio::async_read_until(socket, POST_header, "\r\n\r\n", ...);
...

Client code:

ssl::context context(io_service, ssl::context::sslv23);
context.load_verify_file("server.crt");
socket.reset(new ssl::stream<tcp::socket>(io_service, context));
socket->set_verify_mode(ssl::verify_none);

// standard connection

socket.async_handshake(ssl::stream_base::client, ...);
...

(error handling is omitted along with not relevant code)

As you can see, it's the simplest possible SSL connection. What is wrong? Can the reason be a firewall?

I tried simple TCP w/o SSL over the same 443 port, this works fine.

EDIT:

Tried adding "Content-Type: application/octet-stream", doesn't help.

EDIT 2:

Usually I receive HTTP POST header fine. Then I send data chunks as chunk-size(4 bytes)chunk(chunk-size bytes).... Server receives chunk-size fine, but then nothing. Client doesn't notify server problems (no errors) and continue to send data. Sometimes server can receive chunk or two, sometimes it receives invalid chunk-size, but most time just nothing.

EDIT 3:

Compared captured traffic on client and server, didn't find any differences.

Solution

I was misled from the start with this problem. Narrowed it down to surprising details:

Sending over SSL socket fails if I use Boost.Asio multi-buffers in Boost v.1.48 (the most recent one at this moment). Example:

// data to send, protocol is [packet size: 4 bytes][packet: packet_size bytes]
std::vector<char> packet = ...;
uint32_t packet_size = packet.size();
// prepare buffers
boost::array<boost::asio::const_buffer, 2> bufs = {{boost::asio::buffer(&packet_size, sizeof(packet_size)), boost::asio::buffer(packet)}};
// send multi buffers by single call
boost::asio::async_write(socket, bufs, ...);

Sending separately packet_size and packet in this example works around the problem. I'm far from calling any suspicious behavior as a bug, especially if it's related with Boost libraries. But this one really looks like a bug. Tried on Boost v.1.47 - works fine. Tried with usual TCP socket (not SSL one) - works fine. The same on both Linux and Windows.

I'm going to find any reports about this problem in Asio mailing list and will report it if nothing found.

解决方案

I was misled from the start with this problem. Narrowed it down to surprising details:

Sending over SSL socket fails if I use Boost.Asio multi-buffers in Boost v.1.48 (the most recent one at this moment). Example:

// data to send, protocol is [packet size: 4 bytes][packet: packet_size bytes]
std::vector<char> packet = ...;
uint32_t packet_size = packet.size();
// prepare buffers
boost::array<boost::asio::const_buffer, 2> bufs = {{boost::asio::buffer(&packet_size, sizeof(packet_size)), boost::asio::buffer(packet)}};
// send multi buffers by single call
boost::asio::async_write(socket, bufs, ...);

Sending separately packet_size and packet in this example works around the problem. I'm far from calling any suspicious behavior as a bug, especially if it's related with Boost libraries. But this one really looks like a bug. Tried on Boost v.1.47 - works fine. Tried with usual TCP socket (not SSL one) - works fine. The same on both Linux and Windows.

I'm going to find any reports about this problem in Asio mailing list and will report it if nothing found.

这篇关于如何通过HTTPS POST正确发送二进制数据？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！