为什么Go HTTPS客户端不能重用连接？ [英] Why is Go HTTPS Client not reusing connections?

问题描述

我有一个http客户端，它创建与主机的多个连接。我想设置它可以设置给特定主机的最大连接数。
请求中没有这样的选项。运输。
我的代码看起来像

  package main 
 
 import（
fmt 
net / http
net / url
）
 
 const（
 endpoint_url_fmt =https://blah.com/api1 ？％s
）
 
 
 func main（）{
 
 transport：= http.Transport {DisableKeepAlives：false} 
 $ b $ outParams：= url.Values {} 
 outParams.Set（method，write）
 outParams.Set（message，BLAH）
 
代表{
 //作为URI的一部分进行编码。 
 outboundRequest，err：= http.NewRequest（
GET，
 fmt.Sprintf（endpoint_url_fmt，outParams.Encode（）），
 nil 
）
 outboundRequest.Close = false 
 _，err = transport.RoundTrip（outboundRequest）
 if err！= nil {
 fmt.Println（err）
} 
 } 
 
} 
  

我希望这会创建1个连接。正如我在for循环中调用它。但是这样可以创建无限数量的连接。

使用请求库的类似python代码只创建一个连接。

 ＃！/ usr / bin / env python 
导入请求
 endpoint_url_fmt =https://something.com/restserver.php
 params = {} 
 params ['method'] ='write'
 params ['category'] = category_errors_scuba 
 params ['message'] =blah
 while真：
r = requests.get（endpoint_url_fmt，params = params）
  

由于某些原因， go代码不会重用http连接。

编辑：
go代码需要关闭主体才能重新使用连接。

  resp，err = transport.RoundTrip（outboundRequest）
 resp.Close（）//允许连接被重用
  

解决方案

基于OP的进一步说明。默认客户端确实重新使用连接。

务必关闭响应。

完成读取操作后，呼叫者应关闭resp.Body。如果resp.Body没有关闭，则客户端的底层RoundTripper（通常为Transport）可能无法重新使用持久的到服务器的TCP连接以进行后续保持活动请求。

另外，在调用Close（）之前，我发现我还需要阅读，直到响应完成。
$ b

例如

  res，_：= client.Do（req）
 io.Copy（ioutil.Discard，res.Body ）
 res.Body.Close（）
  

为确保http.Client连接重用，请确保做两件事：

• 阅读到响应完成（即 ioutil.ReadAll（resp.Body） code $）


• 调用 Body.Close（）

---

旧的答案，对速率限制很有用，但不是OP后面的内容：

我不认为通过golang 1.1 http API可以设置最大连接。这意味着如果你不小心的话，你可以用大量的TCP连接在自己的脚下进行拍摄（直到你用完文件描述符或其他任何东西）。

也就是说， 您可以通过time.Tick限制某个特定主机（以及出站请求和连接）的呼叫例程的速度。

例如：

  importtime
 
 requests_per_second：= 5 
 throttle： = time.Tick（1000000000 / requests_per_second）
 
 for i：= 0;我< 16; i + = 1 {
 <-throttle 
 go serveQueue（）
} 
  

I have an http client which creates multiple connections to the host. I want to set a maximum number of connections it can set to a particular host. There are no such options in go's request.Transport. My code looks like

package main 

import (
  "fmt"
  "net/http"
  "net/url"
)

const (
  endpoint_url_fmt      = "https://blah.com/api1?%s"
)


func main() {

  transport := http.Transport{ DisableKeepAlives : false }

  outParams := url.Values{}
  outParams.Set("method", "write")
  outParams.Set("message", "BLAH")

  for {
    // Encode as part of URI.
    outboundRequest, err := http.NewRequest(
      "GET",
      fmt.Sprintf(endpoint_url_fmt, outParams.Encode()),
      nil
    )
    outboundRequest.Close = false
    _ , err = transport.RoundTrip(outboundRequest)
    if err != nil {
      fmt.Println(err)
    }
  }

}

I would expect this to create 1 connection. As I am calling it in a for-loop. But this keeps creating an infinite number of connections.

Where as similar python code using the requests library creates only one connection.

#!/usr/bin/env python
import requests
endpoint_url_fmt      = "https://something.com/restserver.php"
params = {}
params['method'] = 'write'
params['category'] = category_errors_scuba
params['message'] = "blah"
while True:
  r = requests.get(endpoint_url_fmt, params = params)

For some reason the go code is not reusing http connections.

EDIT : The go code needs the body to be closed to reuse the connection.

 resp , err = transport.RoundTrip(outboundRequest)
 resp.Close() //  This allows the connection to be reused

解决方案

Based on further clarification from the OP. The default client does reuse connections.

Be sure to close the response.

Callers should close resp.Body when done reading from it. If resp.Body is not closed, the Client's underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent "keep-alive" request.

Additionally, I've found that I also needed to read until the response was complete before calling Close().

e.g.

res, _ := client.Do(req)
io.Copy(ioutil.Discard, res.Body)
res.Body.Close()

To ensure http.Client connection reuse be sure to do two things:

• Read until Response is complete (i.e. ioutil.ReadAll(resp.Body))
• Call Body.Close()

---

Old answer, useful for rate limiting, but not what the OP was after:

I don't think setting max connections is possible via the golang 1.1 http APIs. This means you can shoot yourself in the foot with tons of TCP connections (until you run out of file descriptors or whatever) if you aren't careful.

That said, you could limit the rate at which you call the go routine for a particular host (and therefore outbound requests and connections) via time.Tick.

For example:

import "time"

requests_per_second := 5
throttle := time.Tick(1000000000 / requests_per_second)

for i := 0; i < 16; i += 1 {
  <-throttle  
  go serveQueue()
}

这篇关于为什么Go HTTPS客户端不能重用连接？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！