Docker 容器不断增长 [英] Docker container keeps growing
问题描述
我有一个循环的 pyhton 脚本
I have a pyhton script that on a loop
- 将视频块从 AWS S3 下载到/filename
- 按顺序对文件进行排序并将它们连接起来.
- 将整个处理过的视频文件上传到 AWS S3
- 删除文件夹/文件名
然后继续循环直到 AWS SQS 队列为空.
Then continues on a loop until the AWS SQS queue is empty.
脚本很好用!我已经运行了几个月.硬盘空间会有所不同,但不会达到 5% 左右,具体取决于视频大小.
Scripts works great! I have run it for months. The hard drive space varies but never gets about 5%, depending on size of the video.
我决定把这个脚本放在一个 docker 容器中并运行 docker-compose 这样我就可以一次运行一堆.
I decided to put this script in a docker container and run docker-compose so I could run a bunch of them at a time.
问题是硬盘已满!我知道 5 运行磁盘上的空间将被租用,但是当我完成处理文件时,将删除.
The problem is the hard drive fills up! I know with 5 running the space on the disk will be hire, but when I'm done processing the file get delete.
但是用docker,好像是缓存什么的.我执行到每个容器中,它们运行良好.删除旧文件等.
But with docker, seems to be a cache or something. I exec into each container and they are running fine. Delete old files and all.
不知道在 docker 容器中运行和作为服务运行之间的区别对 HD 有什么影响.
No clue what the difference between in a docker container and running as a service would have the impact on the HD.
任何方向都会很棒.
补充一下.当我rm"docker 容器时,硬盘空间被释放.我运行 docker ps -s 并且容器上的空间并不疯狂.就像当你在 docker 容器中rm"一个文件时,它从来没有真正 rm 它.
To add to this. When I "rm" the docker containers the hard drive space frees up. I run a docker ps -s and the space on the containers is not crazy. Just seems like when you "rm" a file inside the docker container it never really rm it.
推荐答案
如果您将图像下载到未从主机映射的目录中,则 docker 容器将不会释放已使用的磁盘空间,直到容器被移除 -- 在容器中所做的任何事情都是短暂的,但 HOST 不知道容器内发生的事情的状态.
If you're downloading the image to a directory NOT volumed mapped from the host, the docker container will not release the used disk space until the container is removed--anything done in the container is ephemeral, but the HOST doesn't know the state of what's going on inside the container.
从这个意义上说,它很像一个虚拟机映像,由一个根据需要增长但从不缩小的文件支持.Docker 有一个用于运行容器跟踪更改的目录.在主机上,您可以在 /var/lib/docker/containers/
In this sense it's a lot like a virtual machine image, backed by a file that just grows as needed, but never shrinks. Docker has a directory for a running container tracking changes. On the host you can find the files backing the running container in /var/lib/docker/containers/<id>
如果您需要容器共享磁盘空间,我建议您将一个共享卷从主机映射到每个 docker 容器映像中进行共享.
If you need your containers to share disk space, I'd recommend you map a shared volume from the host into each docker container images to share.
试试下面的方法
docker run -ti -v /host/dir:/container/dir ubuntu bash
以上将以终端交互模式运行 ubuntu 镜像,并将主机目录 /host/dir
挂载到正在运行的容器内.容器写入 /container/dir
的任何内容都将出现在主机 /host/dir
中,并且安装它的任何其他容器也会看到更改.
The above would run the ubuntu image in terminal interactive mode and mounting the host's directory /host/dir
inside the running container. Anything the container writes to /container/dir
will appear in the hosts /host/dir
and any other containers mounting it will see the changes as well.
请记住,在共享卷中所做的任何事情都会被挂载它的所有容器看到,因此在从中添加和删除文件/目录时要小心!
Just remember anything done in the shared volume is seen by all containers that mount it, so be careful when adding and deleting files/directories from it!
这篇关于Docker 容器不断增长的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!