Go http标准库中的内存泄漏？ [英] Memory leak in Go http standard library?

问题描述

让Go二进制文件实现一个http服务器：

  package main 
 
 import（
net / http
）
 
 func main（）{
 http.ListenAndServe（：8080，零）
} 
  

它将以〜850 kb左右的内存开始。通过您的网络浏览器发送一些请求。观察它迅速上升到1 MB。如果你等待，你会发现它永远不会下降。现在用Apache Bench（使用下面的脚本）锤击它，并看到你的内存使用量不断增加。经过一段时间后，它最终会稳定在8.2 MB左右。

编辑：它似乎并没有停止在8.2，而是显着下降。它目前在9.2，并且仍在上涨。

总之，为什么会发生这种情况？我使用了这个shell脚本：

  while [true] 
 do 
 ab -n 1000 -c 100 http://127.0.0.1:8080/ 
 sleep 1 
 end 
  

在试图深入到底时，我试图调整设置。我试着用 r.Close = true 来关闭，以阻止Keep-Alive。没有什么似乎工作。

我最初在尝试确定在编写程序时是否存在内存泄漏时发现了这个问题。它有很多http处理程序和I / O调用。在检查完我关闭了所有数据库连接后，我一直看到它的内存使用率上升。我的节目在 433 MB 附近稳定。

以下是Goenv的输出：

  GOARCH =amd64
 GOBIN =
 GOCHAR =6
 GOEXE =
 GOHOSTARCH =amd64 
 GOHOSTOS =darwin
 GOOS =darwin
 GOPATH =/ Users / mark / Documents / Programming / Go
 GORACE =
 GOROOT = / usr / local / go
 GOTOOLDIR =/ usr / local / go / pkg / tool / darwin_amd64
 TERM =dumb
 CC =clang
 GOGCCFLAGS = -  g -O2 -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fno-common
 CXX =clang ++
 CGO_ENABLED =1
  




解决方案

从堆 pprof 你在评论中提供了，看起来你正在通过 gorilla / sessions 和 gorilla / context 泄漏内存（几乎400MB）。

请参阅此ML主题： https://groups.google.com/f orum /＃！msg / gorilla-web / clJfCzenuWY / N_Xj9-5Lk6wJ 和GH问题： https://github.com/gorilla/sessions/issues/15

这是一个非常快速泄漏的版本：

 包主
 
导入（
fmt
 //github.com/gorilla/上下文
github.com/gorilla/sessions
net / http
）
 
 var（
 cookieStore = sessions.NewCookieStore（[ ] byte（cookie-secret））
）
 
 func main（）{
 http.HandleFunc（/，defaultHandler）
 http.ListenAndServe（ ：8080，无）
} 
 
 func defaultHandler（w http.ResponseWriter，r * http.Request）{
 cookieStore.Get（r，leak-test ）
 fmt.Fprint（w，你好）
} 
  

这里是一个清理，并有一个相对静态的RSS：

  package main 
 
 import（
fmt
github .com / gorilla / context
github.com/gorilla/sessions
net / http
）
 
 var（
 cookieStore = session.NewCookieStore（[] byte（cookie-secret））
）
 
 func main（）{
 http.HandleFunc（/，defaultHandler）
 http.ListenAndServe（：8080，context.ClearHandler（http.DefaultServeMux））
} 
 
 func defaultHandler（w http.ResponseWriter，r * http.Request）{
 cookieStore.Get（r，leak-test）
 fmt.Fprint（w，Hi there）
} 
  

Have a Go binary implement an http server:

package main

import (
    "net/http"
)

func main() {
    http.ListenAndServe(":8080", nil)
}

It will start with ~850 kb or so of memory. Send it some requests via your web browser. Observe it quickly rises to 1 mb. If you wait, you'll see it never goes down. Now hammer it with Apache Bench (using the script below) and see your memory usage continually increase. After sometime it will eventually plateau at around 8.2 MB or so.

Edit: It doesn't seem to stop at 8.2, rather it slows down significantly. It's currently at 9.2 and still rising.

In short, why is this happening? I used this shell script:

while [ true ]
do
    ab -n 1000 -c 100 http://127.0.0.1:8080/
    sleep 1
end

While trying to get to the bottom of this, I've tried to tweak the settings. I've tried closing using r.Close = true to prevent Keep-Alive. Nothing seems to work.

I found this originally while trying to determine if there was a memory leak in a program I'm writing. It has a lot of http handlers and I/O calls. After checking I had closed all my database connections I kept seeing it's memory usage rise. My program to plateau at around 433 MB.

Here's the output of Goenv:

GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mark/Documents/Programming/Go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
TERM="dumb"
CC="clang"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fno-common"
CXX="clang++"
CGO_ENABLED="1"

解决方案

From the heap pprof you have provided in comments, it looks like you are leaking memory via gorilla/sessions and gorilla/context (almost 400MB).

Refer to this ML thread: https://groups.google.com/forum/#!msg/gorilla-web/clJfCzenuWY/N_Xj9-5Lk6wJ and this GH issue: https://github.com/gorilla/sessions/issues/15

Here is a version that leaks extremely quickly:

package main

import (
    "fmt"
    // "github.com/gorilla/context"
    "github.com/gorilla/sessions"
    "net/http"
)

var (
    cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
    http.HandleFunc("/", defaultHandler)
    http.ListenAndServe(":8080", nil)
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
    cookieStore.Get(r, "leak-test")
    fmt.Fprint(w, "Hi there")
}

Here is one that cleans up and has a relatively static RSS:

package main

import (
    "fmt"
    "github.com/gorilla/context"
    "github.com/gorilla/sessions"
    "net/http"
)

var (
    cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
    http.HandleFunc("/", defaultHandler)
    http.ListenAndServe(":8080", context.ClearHandler(http.DefaultServeMux))
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
    cookieStore.Get(r, "leak-test")
    fmt.Fprint(w, "Hi there")
}

这篇关于Go http标准库中的内存泄漏？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！