如何使用 Win32 TransmitFile() 传输大于 2,147,483,646 字节(~2 GiB)的文件? [英] How would one transfer files larger than 2,147,483,646 bytes (~2 GiB) with Win32 TransmitFile()?

问题描述

引自 TransmitFile 的 MSDN 条目:

使用一次 TransmitFile 函数调用可以传输的最大字节数为 2,147,483,646，即 32 位整数减 1 的最大值.单次调用中发送的最大字节数包括发送的任何数据在 lpTransmitBuffers 参数指向的文件数据之前或之后加上 nNumberOfBytesToWrite 参数中指定的值，用于发送文件数据的长度.如果应用程序需要传输大于 2,147,483,646 字节的文件，则可以多次调用 TransmitFile 函数，每次调用传输的文件不超过 2,147,483,646 字节.对于大于 2,147,483,646 字节的文件，将 nNumberOfBytesToWrite 参数设置为零也将失败，因为在这种情况下，TransmitFile 函数将使用文件的大小作为要传输的字节数的值.

The maximum number of bytes that can be transmitted using a single call to the TransmitFile function is 2,147,483,646, the maximum value for a 32-bit integer minus 1. The maximum number of bytes to send in a single call includes any data sent before or after the file data pointed to by the lpTransmitBuffers parameter plus the value specified in the nNumberOfBytesToWrite parameter for the length of file data to send. If an application needs to transmit a file larger than 2,147,483,646 bytes, then multiple calls to the TransmitFile function can be used with each call transferring no more than 2,147,483,646 bytes. Setting the nNumberOfBytesToWrite parameter to zero for a file larger than 2,147,483,646 bytes will also fail since in this case the TransmitFile function will use the size of the file as the value for the number of bytes to transmit.

好的.使用 TransmitFile 发送大小为 2*2,147,483,646 字节(~ 4 GiB)的文件然后必须至少分为两部分(例如，2 GiB + 2 GiB 在对 TransmitFile 的两次调用中).但是，究竟如何做到这一点，同时最好在两者之间保持底层 TCP 连接处于活动状态?

Alright. Sending a file of size 2*2,147,483,646 bytes (~ 4 GiB) with TransmitFile would then have to be divided into two parts at minimum (e.g. 2 GiB + 2 GiB in two calls to TransmitFile). But how exactly would one go about doing that, while preferably also keeping the underlying TCP connection alive in between?

当文件的大小确实小于等于 2,147,483,646 字节时，可以这样写:

When the file is indeed <=2,147,483,646 bytes in size, one could just write:

HANDLE fh = CreateFile(filename, GENERIC_READ, FILE_SHARE_READ, NULL, 
 OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN, NULL);   

TransmitFile(SOCK_STREAM_socket, fh, 0, 0, NULL, NULL, TF_DISCONNECT);

让 Windows 处理所有较低级别的事情(缓存，将数据分块以进行高效传输等.但是，与类似的 Linux sendfile() 系统调用，在调用中没有立即明显的偏移参数(虽然第五个参数，LPOVERLAPPED lpOverlapped 可能正是我'我正在寻找).我想我可以一起破解一些东西，但我也在从真正了解这些东西的人那里寻找优雅的、良好实践的 Win32 解决方案.

to let Windows handle all the lower-level stuff (caching, chunking the data up into pieces for efficient transmission etc. However, unlike the comparable Linux sendfile() syscall, there is no immediately obvious offset argument in the call (although the fifth argument, LPOVERLAPPED lpOverlapped probably is exactly what I'm looking for). I suppose I could hack something together, but I'm also looking for a graceful, good practice Win32 solution from someone who actually knows about this stuff.

您可以使用 lpOverlapped 参数通过设置 OVERLAPPED 结构的 Offset 和 OffsetHigh 成员来指定文件内的 64 位偏移量，在该偏移量处开始文件数据传输.如果 lpOverlapped 是一个 NULL 指针，则数据的传输总是从文件中的当前字节偏移量开始.

You can use the lpOverlapped parameter to specify a 64-bit offset within the file at which to start the file data transfer by setting the Offset and OffsetHigh member of the OVERLAPPED structure. If lpOverlapped is a NULL pointer, the transmission of data always starts at the current byte offset in the file.

那么，由于缺乏网络上现成的最小示例，完成此类任务需要哪些调用?

So, for lack of a minimal example readily available on the net, which calls are necessary to accomplish such a task?

推荐答案

根据评论设法解决了.

因此，如果 LPOVERLAPPED lpOverlapped 是空指针，则调用在文件的当前文件偏移开始传输(很像 Linux sendfile() 系统调用及其 off_t *offset 参数).这个偏移量(指针)可以用 SetFilePointerEx 操作，所以可以这样写:

So, if LPOVERLAPPED lpOverlapped is a null pointer, the call starts transmission at the current file offset of the file (much like the Linux sendfile() syscall and its off_t *offset parameter). This offset (pointer) can be manipulated with SetFilePointerEx, so one could write:

#define TRANSMITFILE_MAX ((2<<30) - 1)

LARGE_INTEGER total_bytes;
memset(&total_bytes, 0, sizeof(total_bytes));

while (total_bytes < filesize) {
     DWORD bytes = MIN(filesize-total_bytes, TRANSMITFILE_MAX);
     if (!TransmitFile(SOCK_STREAM_socket, fh, bytes, 0, NULL, NULL, 0))
     { /* error handling */ }

     total_bytes.HighPart += bytes;
     SetFilePointerEx(fh, total_bytes, NULL, FILE_BEGIN);
}

closesocket(SOCK_STREAM_socket);

完成任务.不是很优雅的imo，但它有效.

to accomplish the task. Not very elegant imo, but it works.

这篇关于如何使用 Win32 TransmitFile() 传输大于 2,147,483,646 字节(~2 GiB)的文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！