如何使用 tcp 连接在 go (golang) 中编写代理 [英] How to write a proxy in go (golang) using tcp connections

问题描述

如果这些问题中的一些对专业网络程序员来说是显而易见的，我事先表示歉意.我已经研究并阅读了有关网络编码的信息，但我仍然不清楚如何做到这一点.

I apologize before hand if some of these questions might be obvious for expert network programmers. I have researched and read about coding in networking and it is still not clear to me how to do this.

假设我想用一些 TCP 客户端和一些 TCP 服务器之间的连接编写一个 tcp 代理(在 go 中).像这样:

Assume that I want to write a tcp proxy (in go) with the connection between some TCP client and some TCP server. Something like this:

首先假设这些连接是半永久性的(会在很长一段时间后关闭)并且我需要数据按顺序到达.

First assume that these connection are semi-permanent (will be closed after a long long while) and I need the data to arrive in order.

我想要实现的想法如下:每当我收到来自客户端的请求时，我想将该请求转发到后端服务器并等待(并且什么都不做)直到后端服务器响应我(代理)，然后将该响应转发给客户端(假设在普通情况下将保持两个 TCP 连接).

The idea that I want to implement is the following: whenever I get a request from the client, I want to forward that request to the backend server and wait (and do nothing) until the backend server responds to me (the proxy) and then forward that response to the client (assume that both TCP connection will be maintained in the common case).

有一个主要问题我不知道如何解决.当我将请求从代理转发到服务器并获得响应时，如果我事先不知道从服务器发送到的数据的格式，我如何知道服务器何时向我发送了我需要的所有信息代理(即我不知道来自服务器的响应是否采用 的形式)类型-长度-值方案 我也不知道` 是否表示来自服务器的消息的结尾).有人告诉我，只要我从 tcp 连接读取的大小为零或小于我预期的读取大小，我就应该假设我从服务器连接获取所有数据.但是，这对我来说似乎不正确.一般而言，它可能不正确的原因如下:

There is one main problem that I am not sure how to solve. When I forward the request from the proxy to the server, and get the response, how do I know when the server has sent me all the information that I need if I do not know beforehand the format of the data being sent from the server to the proxy (i.e. I don't know if the response from the server is of the form of type-length-value scheme nor do I know if ` indicates the end of the message form the server). I was told that I should assume that I get all the data from the server connection whenever my read size from the tcp connection is zero or smaller than the read size that I expected. However, this does not seem correct to me. The reason it might not be correct in general is the following:

假设服务器出于某种原因一次只向其套接字写入一个字节，但对真实"客户端的响应总长度要长得多.因此，是否有可能当代理读取连接到服务器的 tcp 套接字时，代理只读取一个字节，并且如果它循环得足够快(在接收更多数据之前进行读取)，那么读取零并且不正确得出的结论是它得到了客户端想要接收的所有消息?

Assume that the server for some reason is only writing to its socket one byte at a time but the total length of the response to the "real" client is much much much longer. Therefore, isn't it possible that when the proxy reads the tcp socket connected to the server, that the proxy only reads one byte and if it loops fast enough (to do a read before it receives more data), then read zero and incorrectly concludes that It got all the message that the client intended to receive?

解决此问题的一种方法可能是在每次从套接字读取后等待，这样代理的循环速度不会超过获取字节的速度.我担心的原因是，假设有一个网络分区，我无法再与服务器通话.但是，它与我断开连接的时间不足以使 TCP 连接超时.因此，是否有可能我再次尝试从 tcp 套接字读取到服务器(比我获取数据更快)并读取零并错误地得出它的所有数据，然后将其打包发送到客户端?(请记住，我想保持的承诺是，当我写入客户端连接时，我只将整个消息发送给客户端.因此，如果代理去考虑正确的行为，在它之后再次读取连接是非法的已经写入客户端，并在稍后发送丢失的块，可能在不同请求的响应期间).

One way to fix this might be to wait after each read from the socket, so that the proxy doesn't loop faster than it gets bytes. The reason that I am worried is, assume there is a network partition and i can't talk to the server anymore. However, it is not disconnected from me long enough to timeout the TCP connection. Thus, isn't it possible that I try to read from the tcp socket to the server again (faster than I get data) and read zero and incorrectly conclude that its all the data and then send it pack to the client? (remember, the promise I want to keep is that I only send whole messages to the client when i write to the client connection. Thus, its illegal to consider correct behaviour if the proxy goes, reads the connection again at a later time after it already wrote to the client, and sends the missing chunk at a later time, maybe during the response of a different request).

我编写的代码在 go-playground 中.

我喜欢用来解释为什么我认为这种方法不起作用的类比如下:

The analogy that I like to use to explain why I think this method doesn't work is the following:

假设我们有一个杯子，代理每次从服务器读取数据时都会喝掉一半的杯子，但服务器一次只放 1 茶匙.因此，如果代理喝水的速度比获得茶匙的速度快，它可能会过早地达到零并得出结论，它的套接字是空的，可以继续前进！如果我们想保证每次都发送完整的消息，这是错误的.或者，这个类比是错误的，TCP 的一些魔法"使它起作用，或者假设直到套接字为空的算法是完全错误的.

Say we have a cup and the proxy is drinking half the cup every time it does a read from the server, but the server only puts 1 teaspoon at a time. Thus, if the proxy drinks faster than it gets teaspoons it might reach zero too soon and conclude that its socket is empty and that its ok to move on! Which is wrong if we want to guarantee we are sending full messages every time. Either, this analogy is wrong and some "magic" from TCP makes it work or the algorithm that assumes until the socket is empty is just plain wrong.

一个问题<处理类似问题的/a> 建议阅读直到 EOF.但是，我不确定为什么这是正确的.阅读 EOF 是否意味着我收到了缩进的消息?每次有人将一大块字节写入 tcp 套接字时都会发送一个 EOF(即我担心如果服务器一次写入一个字节，它会发送 1 个 EOF> 每字节)?然而，EOF 可能是 TCP 连接真正工作方式的一些魔法"?发送 EOF 会关闭连接吗?如果它不是我想使用的方法.此外，我无法控制服务器可能正在做什么(即我不知道它想多久写入一次套接字以将数据发送到代理，但是，假设它以某种标准"写入套接字是合理的/正常写入套接字的算法").我只是不相信从服务器的套接字读取直到 EOF 是正确的.为什么会呢?我什么时候可以读到EOF?EOF 是数据的一部分还是在 TCP 标头中?

A question that deals with a similar problems here suggests to read until EOF. However, I am unsure why that would be correct. Does reading EOF mean that I got the indented message? Is an EOF sent each time someone writes a chunk of bytes to a tcp socket (i.e. I am worried that if the server writes one byte at a time, that it sends 1 EOF per bytes)? However, EOF might be some of the "magic" of how a TCP connection really works? Does sending EOF's close the connection? If it does its not a method that I want to use. Also, I have no control of what the server might be doing (i.e. I do not know how often it wants to write to the socket to send data to the proxy, however, its reasonable to assume it writes to the socket with some "standard/normal writing algorithm to sockets"). I am just not convinced that reading till EOF from the socket from server is correct. Why would it? When can I even read to EOF? Are EOFs part of the data or are they in the TCP header?

此外，我写的关于将 wait 放在超时时间之后的想法，这在最坏的情况下有效还是仅在平均情况下有效?我也在想，我意识到如果 Wait() 调用比超时时间长，那么如果你返回到 tcp 连接并且它没有任何东西，那么继续前进是安全的.但是，如果它没有任何东西并且我们不知道服务器发生了什么，那么我们就会超时.因此关闭连接是安全的(因为无论如何超时都会这样做).因此，我认为如果 Wait 调用至少与超时一样长，则此过程确实有效！人们怎么看?

Also, the idea that I wrote about putting a wait just epsilon bellow the time-out, would that work in the worst-case or only on average? I was also thinking, I realized that if the Wait() call is longer than the time-out, then if you return to the tcp connection and it doesn't have anything, then its safe to move on. However, if it doesn't have anything and we don't know what happened to the server, then we would time out. So its safe to close the connection (because the timeout would have done that anyway). Thus, I think if the Wait call is at least as long as the timeout, this procedure does work! What do people think?

我也对一个答案感兴趣，该答案可以证明为什么该算法适用于某些情况.比如我在想，即使服务器一次只写一个字节，如果部署的场景是一个紧凑的数据中心，那么平均来说，因为延迟真的很小，等待调用几乎可以肯定，那么就不会了这个算法不行吗?

I am also interested in an answer that can justify maybe why this algorithm work on some cases. For example, I was thinking, even if the server only write a byte at a time, if the scenario of deployment is a tight data centre, then on average, because delays are really small and the wait call is almost certainly enough, then wouldn't this algorithm be fine?

另外，我写的代码有没有陷入死锁"的风险?

Also, are there any risks of the code I wrote getting into a "deadlock"?

package main

import (
    "fmt"
    "net"
)

type Proxy struct {
    ServerConnection *net.TCPConn
    ClientConnection *net.TCPConn
}

func (p *Proxy) Proxy() {
    fmt.Println("Running proxy...")
    for {
        request := p.receiveRequestClient()
        p.sendClientRequestToServer(request)
        response := p.receiveResponseFromServer() //<--worried about this one.
        p.sendServerResponseToClient(response)
    }
}

func (p *Proxy) receiveRequestClient() (request []byte) {
    //assume this function is a black box and that it works.
    //maybe we know that the messages from the client always end in 
 or they
    //they are length prefixed.
    return
}

func (p *Proxy) sendClientRequestToServer(request []byte) {
    //do
    bytesSent := 0
    bytesToSend := len(request)
    for bytesSent < bytesToSend {
        n, _ := p.ServerConnection.Write(request)
        bytesSent += n
    }
    return
}

// Intended behaviour: waits until ALL of the response from backend server is obtained.
// What it does though, assumes that if it reads zero, that the server has not yet
// written to the proxy and therefore waits. However, once the first byte has been read,
// keeps writting until it extracts all the data from the server and the socket is "empty".
// (Signaled by reading zero from the second loop)
func (p *Proxy) receiveResponseFromServer() (response []byte) {
    bytesRead, _ := p.ServerConnection.Read(response)
    for bytesRead == 0 {
        bytesRead, _ = p.ServerConnection.Read(response)
    }
    for bytesRead != 0 {
        n, _ := p.ServerConnection.Read(response)
        bytesRead += n
        //Wait(n) could solve it here?
    }
    return
}

func (p *Proxy) sendServerResponseToClient(response []byte) {
    bytesSent := 0
    bytesToSend := len(request)
    for bytesSent < bytesToSend {
        n, _ := p.ServerConnection.Write(request)
        bytesSent += n
    }
    return
}

func main() {
    proxy := &Proxy{}
    proxy.Proxy()
}

推荐答案

除非您使用特定的更高级别协议，否则不会从客户端读取消息"以中继到服务器.TCP 是一种流协议，您所能做的就是来回传输字节.

Unless you're working with a specific higher-level protocol, there is no "message" to read from the client to relay to the server. TCP is a stream protocol, and all you can do is shuttle bytes back and forth.

好消息是这非常简单，这个代理的核心部分是:

The good news is that this is amazingly easy in go, and the core part of this proxy will be:

go io.Copy(server, client)
io.Copy(client, server)

这显然缺少错误处理，并且没有完全关闭，但清楚地显示了核心数据传输的处理方式.

This is obviously missing error handling, and doesn't shut down cleanly, but clearly shows how the core data transfer is handled.

这篇关于如何使用 tcp 连接在 go (golang) 中编写代理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！