使用原始套接字重新传输大数据包 [英] Retransmitting large packets with raw sockets

问题描述

问题:在原始套接字上， recvfrom 捕获的字节数超过了 sendto 可以发送的字节数，从而阻止了我重新传输大于MTU的数据包./p>

背景:我正在编写一个将捕获并重新传输数据包的应用程序.基本上，主机A将数据发送到X，将其记录下来，然后将其转发到所有Linux机器B.我正在使用原始套接字，因此我可以捕获所有数据，并使用 socket(AF_PACKET，SOCK_RAW，htons(ETH_P_ALL))创建.

然后，有代码等待并读取传入的数据包:

  const int buffer_size = 2048;uint8_t * buffer =新的uint8_t [buffer_size];sockaddr_ll addr = {0};socklen_t addr_len = sizeof(addr);int receive_bytes = recvfrom(_raw_socket，buffer，buffer_size，0，(struct sockaddr *)& addr，& addr_len); 

随后进行数据包处理，并通过再次发送数据包完成循环:

  struct sockaddr_ll addr;memset(& addr，0，sizeof(struct sockaddr_ll));addr.sll_family = htons(AF_PACKET);addr.sll_protocol = eth_hdr-> type;addr.sll_ifindex = interface().id();addr.sll_halen = HardwareAddress :: byte_size;memcpy(&(addr.sll_addr)，eth_hdr-> dest_mac，HardwareAddress :: byte_size);//尝试发送数据包if(sendto(raw_socket()，data，length，0，(struct sockaddr *)& addr，sizeof(addr))< 0) 

问题是我不希望收到大于以太网MTU(1500字节)的数据包，而且我不应该这样做，因为我使用的是分别处理每个数据包的原始套接字.但是有时候我会收到比MTU大的数据包.我以为代码中可能有错误，但Wireshark确认如图所示，因此必须在较低级别进行一些重组，例如网络控制器本身.

好吧，那么我认为没有一种方法可以仅针对一个应用程序禁用此功能，并且我无法更改主机配置，因此可能会增加缓冲区大小.但是问题是，当我用大于MTU大小的任何东西调用 sendto 时(实际上是1514B，因为eth标头)，我得到了 80:消息太长 errno.这就是上面提到的问题-我无法发送与收到的相同的数据包.有什么可能的解决方案?而我需要什么缓冲区大小才能始终捕获整个数据包?

编辑:我刚刚使用 ethtool -k interf 检查了机器，并在所有机器上都打开了 tcp-segmentation-offload:on 它们，所以看来这实际上是NIC重新组装的片段.但是我想知道为什么 sendto 不能表现为 recvfrom .如果可以自动重组数据包，为什么不分段?

旁注:应用程序需要发送那些数据包.使用iptables等设置转发功能无效.

解决方案

您的网卡可能已启用分段卸载，这意味着硬件可以在TCP分段到达操作系统或您的代码之前重新组装它们.

您可以通过运行 ethtool -k 检查情况是否如此.透明地捕获TCP流量并以如此低的级别重新传输通常比它值得的麻烦更多(通常最好在应用程序层执行此操作，终止TCP连接并为您的主机建立一个新的TCP连接B)，如果您的网卡弄乱了数据包，则无法捕获并重新发送数据包.您需要:

• 关闭通用细分卸载
• 关闭通用接收卸载
• 关闭tcp-segmentation-offload
• 如果您还处理UDP，请关闭udp-fragmentation-offload
• 如果您的数据包是VLAN封装的，请关闭rx-vlan-offload/tx-vlan-offload
• 可能会关闭rx-checksumming和tx-checksumming.如果两者都起作用已启用，或已损坏.RAW套接字(如果启用)，具体取决于您的内核版本和网卡类型.

可以使用 ethtool -K 命令打开/关闭这些命令，确切的语法在ethtool联机帮助页中进行了描述.

Problem: On raw sockets, recvfrom can capture more bytes than sendto can send, preventing me from retransmitting packets larger than MTU.

Background: I'm programming an application that will capture and retransmit packets. Basically host A sends data to X that logs them and forwards them to B, all Linux machines. I'm using raw socket so I can capture all data and it's created with socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL)).

Then, there's code waiting for and reading incoming packets:

const int buffer_size = 2048;
uint8_t* buffer = new uint8_t[buffer_size];
sockaddr_ll addr = {0};
socklen_t addr_len = sizeof(addr);
int received_bytes = recvfrom(_raw_socket, buffer, buffer_size, 0, (struct sockaddr*)&addr, &addr_len);

Packet processing follows and the loop is finished with sending packet out again:

struct sockaddr_ll addr;
memset(&addr, 0, sizeof(struct sockaddr_ll));
addr.sll_family = htons(AF_PACKET);
addr.sll_protocol = eth_hdr->type;
addr.sll_ifindex = interface().id();
addr.sll_halen = HardwareAddress::byte_size;
memcpy(&(addr.sll_addr), eth_hdr->dest_mac, HardwareAddress::byte_size);

// Try to send packet
if(sendto(raw_socket(), data, length, 0, (struct sockaddr*)&addr, sizeof(addr)) < 0)

The problem is that I don't expect to receive packets that are larger than Ethernet MTU (1500 bytes) and I shouldn't since I'm using raw sockets that process each packet individually. But sometimes I do receive packets larger than MTU. I thought it might be error in my code but Wireshark confirms that as shown in the image, so there must be some reassembly going on at lower level like network controller itself.

Well, ok then I don't think there's a way to disable this for just one application and I can't change the host configuration, so I might increase buffer size. But the problem is that when I call sendto with anything larger than MTU size (actually 1514B, becuase of eth header) I get 80: Message too long errno. And that's the problem stated above - I can't send out the same packet I received. What could be possible solution for this? And what buffer size would I need to always capture whole packet?

EDIT: I have just checked on the machines with ethtool -k interf and got tcp-segmentation-offload: on on all of them, so it seems that it's really NIC reassembling fragments. But I wonder why sendto doesn't behave as recvfrom. If the packets can be automatically reassembled, why not fragmented?

A side note: The application needs to send those packets. Setting up forwarding with iptables etc. won't work.

解决方案

Your network card probably has segmentation offload enabled, which means the hardware can re-assemble TCP segments before they reach the OS or your code.

You can check whether that is the case by running ethtool -k. While transparently capturing TCP traffic and re-transmitting it at such a low level is often more trouble than it is worth(one are often better off doing this at the application layer, terminate the TCP connection and set up a new TCP connection towards your host B), you cannot capture and re-send packets if your network card has messed with the packets. You need to:

• Turn off generic-segmentation-offload
• Turn off generic-receive-offload
• Turn off tcp-segmentation-offload
• Turn off udp-fragmentation-offload if you are also dealing with UDP
• Turn off rx-vlan-offload/tx-vlan-offload if your packets are VLAN encapsulated
• Possibly turn off rx-checksumming and tx-checksumming. It either works if both are enabled, or it's broken wrt. RAW sockets if enabled, depending on your kernel version and type of network card.

These can be turned on/off with the ethtool -K command, the exact syntax is described in the ethtool manpage.

这篇关于使用原始套接字重新传输大数据包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！