无法释放曾经被bytes.Buffer占用的内存 [英] Cannot free memory once occupied by bytes.Buffer

问题描述

我接收类型为[]字节的 compressedbytes 中的压缩ASCII文本字节。我面临的问题是，下面的过程占用了大量的内存，在函数达到其结束时不会释放内存，并在程序的整个运行时保持占用。

  b：= bytes.NewReader（compressedbytes）
r，err：= zlib.NewReader（b）
 if err！= nil {
 panic（err ）
} 
 cleartext，err = ioutil.ReadAll（r）
 if err！= nil {
 panic（err）
} 
  

我注意到正在使用的类型是 bytes.Buffer ，并且此类型有 Reset（）和 Truncate（）函数，但它们都不允许释放曾经占用的内存。 / p>

文档 Reset（）声明如下：
$ b

Reset将缓冲区重置为空，但它保留了未来写入使用的底层存储。重置与截断（0）相同。

如何取消设置缓冲区并再次释放内存？
我的程序在运行过程中需要大约50MB的内存，需要2小时。当我导入zlib压缩的字符串时，程序需要200 MB内存。

感谢您的帮助。

===更新

我甚至为解压创建了一个单独的函数，并使用 runtime.GC（）



  //解压缩zlib压缩字节
 func unpack（packedData [] byte）[] byte {
b：= bytes.NewReader（packedData）
r，err：= zlib.NewReader（b）
 if err！= nil {
 panic（err）
} 
 cleartext，err：= ioutil.ReadAll（r）
 if err！= nil {
 panic（err）
} 
 r.Close（）
 return cleartext 
} 
  




解决方案

Go是一种垃圾收集语言，这意味着变量分配和使用的内存会在垃圾收集器无法访问时自动释放（如果您有另一个指向该变量的指针，仍然被视为可达）。

释放内存并不意味着它会返回到操作系统。释放内存意味着内存可以被回收，如果需要的话可以重新用于其他变量。所以从操作系统来看，你不会看到内存会立即减少，只是因为某些变量变得无法访问，垃圾收集器检测到它并释放内存使用。

Go但是，如果运行时间没有被使用一段时间（通常大约5分钟），它会将内存返回给操作系统。如果在此期间内存使用量增加（并且可能会再次缩小），则内存很可能不会返回到操作系统。

如果您等待一段时间并且未分配内存内存再次释放内存将最终返回到操作系统（显然不是所有的，但未使用的大块将是）。如果您不能等待发生这种情况，可以致电 debug。 FreeOSMemory（） 强制执行此行为：

FreeOSMemory强制执行垃圾回收，然后尝试尽可能多地将内存返回给操作系统。 （即使未被调用，运行时会逐渐将内存返回到操作系统的后台任务中。）

检查此类旧的，但真正内容丰富的问题​​+答案：

Go 1.3垃圾收集器不释放服务器内存回到系统

I receive bytes of compressed ASCII text in compressedbytes of type []byte. The problem I face is that the following procedure occupies a lot of memory that does not get freed after the function reaches its end and remains occupied during the whole runtime of the program.

    b := bytes.NewReader(compressedbytes)
    r, err := zlib.NewReader(b)
    if err != nil {
        panic(err)
    }
    cleartext, err = ioutil.ReadAll(r)
    if err != nil {
        panic(err)
    }

I noticed that the type in use is bytes.Buffer and this type has the Reset() and Truncate() functions but none of them allows to free the memory that is once occupied.

The documentation of Reset() states the following:

Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).

How can I unset the buffer and free the memory again? My program needs about 50MB of memory during the run that takes 2h. When I import strings that are zlib compressed the program needs 200 MB of memory.

Thanks for your help.

=== Update

I even created a separate function for the decompression and call the garbage collector manually with runtime.GC() after the program returns from that function without success.

// unpack decompresses zlib compressed bytes
func unpack(packedData []byte) []byte {
    b := bytes.NewReader(packedData)
    r, err := zlib.NewReader(b)
    if err != nil {
        panic(err)
    }
    cleartext, err := ioutil.ReadAll(r)
    if err != nil {
        panic(err)
    }
    r.Close()
    return cleartext
}

解决方案

Some things to clear. Go is a garbage collected language, which means that memory allocated and used by variables is automatically freed by the garbage collector when those variables become unreachable (if you have another pointer to the variable, that still counts as "reachable").

Freed memory does not mean it is returned to the OS. Freed memory means the memory can be reclaimed, reused for another variable if there is a need. So from the operating system you won't see memory decreasing right away just because some variable became unreachable and the garbage collector detected this and freed memory used by it.

The Go runtime will however return memory to the OS if it is not used for some time (which is usually around 5 minutes). If the memory usage increases during this period (and optionally shrinks again), the memory will most likely not be returned to the OS.

If you wait some time and not allocate memory again, freed memory will be returned to the OS eventually (obviously not all, but unused "big chunks" will be). If you can't wait for this to happen, you may call debug.FreeOSMemory() to force this behavior:

FreeOSMemory forces a garbage collection followed by an attempt to return as much memory to the operating system as possible. (Even if this is not called, the runtime gradually returns memory to the operating system in a background task.)

Check out this kind of old but really informative question+answers:

Go 1.3 Garbage collector not releasing server memory back to system

这篇关于无法释放曾经被bytes.Buffer占用的内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！