一旦被bytes.Buffer占用就无法释放内存 [英] Cannot free memory once occupied by bytes.Buffer
问题描述
我以 []byte 类型的 compressedbytes 接收压缩 ASCII 文本字节.我面临的问题是以下过程占用了大量内存，在函数结束后没有释放，并在整个程序运行时一直被占用.
I receive bytes of compressed ASCII text in compressedbytes of type []byte. The problem I face is that the following procedure occupies a lot of memory that does not get freed after the function reaches its end and remains occupied during the whole runtime of the program.
b := bytes.NewReader(compressedbytes)
r, err := zlib.NewReader(b)
if err != nil {
panic(err)
}
cleartext, err = ioutil.ReadAll(r)
if err != nil {
panic(err)
}
我注意到正在使用的类型是 bytes.Buffer 并且这种类型具有 Reset() 和 Truncate() 函数但没有其中允许释放曾经被占用的内存.
I noticed that the type in use is bytes.Buffer and this type has the Reset() and Truncate() functions but none of them allows to free the memory that is once occupied.
Reset() 的文档指出以下:
Reset 将缓冲区重置为空，但它保留了底层存储以供将来写入使用.Reset 与 Truncate(0) 相同.
Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).
如何取消设置缓冲区并再次释放内存?我的程序在需要 2 小时的运行期间需要大约 50MB 的内存.当我导入 zlib 压缩的字符串时，程序需要 200 MB 内存.
How can I unset the buffer and free the memory again? My program needs about 50MB of memory during the run that takes 2h. When I import strings that are zlib compressed the program needs 200 MB of memory.
感谢您的帮助.
=== 更新
我什至为解压创建了一个单独的函数，并在程序从该函数返回但没有成功后使用 runtime.GC() 手动调用垃圾收集器.
I even created a separate function for the decompression and call the garbage collector manually with runtime.GC() after the program returns from that function without success.
// unpack decompresses zlib compressed bytes
func unpack(packedData []byte) []byte {
b := bytes.NewReader(packedData)
r, err := zlib.NewReader(b)
if err != nil {
panic(err)
}
cleartext, err := ioutil.ReadAll(r)
if err != nil {
panic(err)
}
r.Close()
return cleartext
}
推荐答案
有些事情需要澄清.Go 是一种垃圾收集语言，这意味着当这些变量无法访问时，垃圾收集器会自动释放变量分配和使用的内存(如果您有另一个指向该变量的指针，仍然算作可达").
Some things to clear. Go is a garbage collected language, which means that memory allocated and used by variables is automatically freed by the garbage collector when those variables become unreachable (if you have another pointer to the variable, that still counts as "reachable").
释放的内存并不意味着它会返回给操作系统.释放内存意味着内存可以被回收，如果需要，可以重新用于另一个变量.因此，从操作系统来看，您不会看到内存立即减少，因为某些变量无法访问并且垃圾收集器检测到并释放了它使用的内存.
Freed memory does not mean it is returned to the OS. Freed memory means the memory can be reclaimed, reused for another variable if there is a need. So from the operating system you won't see memory decreasing right away just because some variable became unreachable and the garbage collector detected this and freed memory used by it.
但是，如果一段时间(通常大约 5 分钟)没有使用，Go 运行时会将内存返回给操作系统.如果在此期间内存使用量增加(并且可选地再次收缩)，则内存很可能不会返回给操作系统.
The Go runtime will however return memory to the OS if it is not used for some time (which is usually around 5 minutes). If the memory usage increases during this period (and optionally shrinks again), the memory will most likely not be returned to the OS.
如果您等待一段时间后不再分配内存，则释放的内存最终将返回给操作系统(显然不是全部，但未使用的大块"会).如果您迫不及待地等待这种情况发生，您可以调用 debug.FreeOSMemory() 强制这种行为:
If you wait some time and not allocate memory again, freed memory will be returned to the OS eventually (obviously not all, but unused "big chunks" will be). If you can't wait for this to happen, you may call debug.FreeOSMemory() to force this behavior:
FreeOSMemory 强制进行垃圾回收，然后尝试将尽可能多的内存返回给操作系统.(即使不调用此方法，运行时也会在后台任务中逐渐将内存返还给操作系统.)
FreeOSMemory forces a garbage collection followed by an attempt to return as much memory to the operating system as possible. (Even if this is not called, the runtime gradually returns memory to the operating system in a background task.)
看看这种古老但信息量很大的问题+答案:
Check out this kind of old but really informative question+answers:
这篇关于一旦被bytes.Buffer占用就无法释放内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
