golang http超时和goroutines积累 [英] golang http timeout and goroutines accumulation

查看:103
本文介绍了golang http超时和goroutines积累的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我使用goroutines实现http.Get超时,然后我发现这个数字一直在稳步上升,当它达到1000左右时,程序将退出

代码:

 包主
$ b导入(
错误
io / ioutil
记录

净/ http
运行时
时间

$ b $ //超时拨号
func timeoutDialler(超时time.Duration)func(network,addr string)(net.Conn,error){
return func(network,addr string)( net.Conn,错误){
return net.DialTimeout(network,addr,timeout)
}
}

func timeoutHttpGet(url string)([] byte ,错误){
//改变拨号程序添加超时支持&&禁用保持活动
tr:=& http.Transport {
Dial:timeoutDialler(3 * time.Second),
DisableKeepAlives:true,
}

client:=& http.Client {Transport:tr}

类型响应struct {
resp [] byte
err error
}

ch:= make(chan Response,0)
defer func(){
close(ch)
ch = nil
}()

去func(){
resp,err:= client.Get(url)
if err!= nil {
ch < - Response {[] byte {}, err}
return
}
defer resp.Body.Close()

body,err:= ioutil.ReadAll(resp.Body)
if err!= nil {
ch< - Response {[] byte {},err}
返回
}

tr.CloseIdleConnections()
ch < - Response {body,err}
}()

select {
case< -time.After(5 * time.Second):
return [] byte {},errors.New(timeout)
case response:=< -ch :
return response.resp,response.err
}
}

func处理程序(w http.ResponseWriter,r * http.Request){
_,err:= timeoutHttpGet(http://google.com)
if err!= nil {
log.Println(err)
return
}


func main(){
为{
log.Println(runtime.NumGoroutine())
时间去func(){
。睡眠(500 * time.Millisecond)
}
}()

s:=& http.Server {
地址::8888,
ReadTimeout:15 * time.Second,
WriteTimeout:15 * time.Second,
}

http.HandleFunc(/,处理函数)
log.Fatal (s.ListenAndServe())
}

http://play.golang.org/p/SzGTMMmZkI

解决方案

使用1而不是0初始化你的陈:


$ b

ch:= make(chan Response,1) code>



然后移除关闭的延期块,并删除



请参阅: http://blog.golang.org/go-concurrency-patterns-timing-out - 和



以下是我认为正在发生的事情:


  1. 5s超时后,timeoutHttpGet返回

  2. defer语句运行,关闭ch然后将其设置为零

  3. 它开始执行的去程序实际上 获取完成并尝试将数据发送给ch

  4. ,但ch为零,因此不会收到任何内容,从而阻止该语句结束,从而阻止去程序完成
  5. li>

我假设你设置 ch = nil ,因为在你之前,遇到运行时恐慌,因为当您尝试写入封闭的频道时会发生这种情况,如规范

给出1的缓冲区意味着取指去程序可以发送给它而不需要接收器。如果处理程序由于超时而返回,则所有内容都将在稍后进行垃圾回收。


I use goroutines achieve http.Get timeout, and then I found that the number has been rising steadily goroutines, and when it reaches 1000 or so, the program will exit

Code:

package main

import (
        "errors"
        "io/ioutil"
        "log"
        "net"
        "net/http"
        "runtime"
        "time"
)

// timeout dialler
func timeoutDialler(timeout time.Duration) func(network, addr string) (net.Conn, error) {
        return func(network, addr string) (net.Conn, error) {
                return net.DialTimeout(network, addr, timeout)
        }
}

func timeoutHttpGet(url string) ([]byte, error) {
        // change dialler add timeout support && disable keep-alive
        tr := &http.Transport{
                Dial:              timeoutDialler(3 * time.Second),
                DisableKeepAlives: true,
        }

        client := &http.Client{Transport: tr}

        type Response struct {
                resp []byte
                err  error
        }

        ch := make(chan Response, 0)
        defer func() {
                close(ch)
                ch = nil
        }()

        go func() {
                resp, err := client.Get(url)
                if err != nil {
                        ch <- Response{[]byte{}, err}
                        return
                }
                defer resp.Body.Close()

                body, err := ioutil.ReadAll(resp.Body)
                if err != nil {
                        ch <- Response{[]byte{}, err}
                        return
                }

                tr.CloseIdleConnections()
                ch <- Response{body, err}
        }()

        select {
        case <-time.After(5 * time.Second):
                return []byte{}, errors.New("timeout")
        case response := <-ch:
                return response.resp, response.err
        }
}

func handler(w http.ResponseWriter, r *http.Request) {
        _, err := timeoutHttpGet("http://google.com")
        if err != nil {
                log.Println(err)
                return
        }
}

func main() {
        go func() {
                for {
                        log.Println(runtime.NumGoroutine())
                        time.Sleep(500 * time.Millisecond)
                }
        }()

        s := &http.Server{
                Addr:         ":8888",
                ReadTimeout:  15 * time.Second,
                WriteTimeout: 15 * time.Second,
        }

        http.HandleFunc("/", handler)
        log.Fatal(s.ListenAndServe())
}

http://play.golang.org/p/SzGTMMmZkI

解决方案

Init your chan with 1 instead of 0:

ch := make(chan Response, 1)

And remove the defer block that closes and nils ch.

See: http://blog.golang.org/go-concurrency-patterns-timing-out-and

Here is what I think is happening:

  1. after the 5s timeout, timeoutHttpGet returns
  2. the defer statement runs, closing ch and then setting it to nil
  3. the go routine it started to do the actual fetch finishes and attempts to send its data to ch
  4. but ch is nil, and so won't receive anything, preventing that statement from finishing, and thus preventing the go routine from finishing

I assume you are setting ch = nil because before you had that, you would get run-time panics because that's what happens when you attempt to write to a closed channel, as described by the spec.

Giving ch a buffer of 1 means that the fetch go routine can send to it without needing a receiver. If the handler has returned due to timeout, everything will just get garbage collected later on.

这篇关于golang http超时和goroutines积累的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆