为Python项目构建Docker映像时如何避免重新安装软件包？ [英] How to avoid reinstalling packages when building Docker image for Python projects?

问题描述

我的Dockerfile类似于

  FROM my / base 
 
添加。 / srv 
运行点安装-r requirements.txt 
运行python setup.py安装
 
 ENTRYPOINT [run_server] 
  / pre> 
 
 
每次我构建一个新的图像时，都必须重新安装依赖关系，这在我的区域可能很慢。
 
 
 
我想到的一种方式是缓存已安装的软件包是覆盖 my / base 具有较新图像的图像：
  docker build -t new_image_1。 
 docker tag new_image_1我/ base 
  
所以下一次使用这个Dockerfile构建我/基本已经安装了一些软件包。
 
 
 
但是这个解决方案有两个问题：
 
 
并不总是可以覆盖基本图像
 
 
随着较新的图像分层，基本图像越来越大越来越小
 
 
那么可以用什么更好的解决方案来解决这个问题？
 
 
 
 EDIT ##：
 
 
 
有关我机器上码头服务器的一些信息：
 ☁test docker版本
客户端版本：1.1 .2 
客户端API版本：1.13 
 Go版本（客户端）：go1.2.1 
 Git提交（客户端）：d84a070 
服务器版本：1.1.2 
服务器API版本：1.13 
 Go version（server）：go1.2.1 
 Git commit（server）：d84a070 
☁test docker info 
容器：0 
图片：56 
存储驱动程序：aufs 
根目录：/ var / lib / docker / aufs 
目录s：56 
执行驱动程序：native-0.2 
内核版本：3.13.0-29-generic 
警告：无交换限制支持
  
 
 
解决方案
尝试使用下面的Dockerfile构建。
  FROM my / base 
 
 WORKDIR / srv 
添加./requirements.txt /srv/requirements.txt 
 RUN pip install -r requirements.txt 
添加。 / srv 
 RUN python setup.py install 
 
 ENTRYPOINT [run_server] 
  
如果。（您的项目）中有一些更改，docker将跳过 pip install 缓存。
 
 
 
当您编辑requires.txt文件时，Docker只能在构建版本上运行 pip install 。
 
 
 
 
 
 
我写简单 Hello，World！程序
  $ tree 
。 
├──Dockerfile 
├──requirements.txt 
└──run.py 
 
 0个目录，3个文件
 
＃ Dockerfile 
 
 FROM dockerfile / python 
 WORKDIR / srv 
添加./requirements.txt /srv/requirements.txt 
运行点安装-r requirements.txt 
添加/ srv 
 CMD python /srv/run.py 
 
＃requirements.txt 
 pytest == 2.3.4 
 
＃run.py 
 print（Hello，World）
  
以下是输出。
 步骤1：WORKDIR / srv 
 --->运行在22d725d22e10 
 ---> 55768a00fd94 
删除中间容器22d725d22e10 
步骤2：ADD ./requirements.txt /srv/requirements.txt 
 ---> 968a7c3a4483 
删除中间容器5f4e01f290fd 
步骤3：运行点安装-r requirements.txt 
 --->运行在08188205e92b 
下载/解包pytest == 2.3.4（从-r requirements.txt（第1行））
运行setup.py（path：/tmp/pip_build_root/pytest/setup.py） egg_info for package pytest 
 .... 
清理... 
 ---> bf5c154b87c9 
删除中间容器08188205e92b 
步骤4：添加。 / srv 
 ---> 3002a3a67e72 
删除中间容器83defd1851d0 
步骤5：CMD python /srv/run.py 
 --->运行在11e69b887341 
 ---> 5c0e7e3726d6 
删除中间容器11e69b887341 
成功构建5c0e7e3726d6 
  
我只更新运行。 py，然后尝试重新构建。
 ＃run.py 
 print（Hello，Python）
  
以下是输出。 
 将构建上下文发送到Docker守护程序5.12 kB 
将构建上下文发送到Docker守护程序
步骤0：FROM dockerfile / python 
 ---> f86d6993fc7b 
步骤1：WORKDIR / srv 
 --->使用缓存
 ---> 55768a00fd94 
步骤2：ADD ./requirements.txt /srv/requirements.txt 
 --->使用缓存
 ---> 968a7c3a4483 
步骤3：运行点安装-r requirements.txt 
 --->使用缓存
 ---> bf5c154b87c9 
步骤4：添加。 / srv 
 ---> 9cc7508034d6 
删除中间容器0d7cf71eb05e 
步骤5：CMD python /srv/run.py 
 --->运行在f25c21135010 
 ---> 4ffab7bc66c7 
删除中间容器f25c21135010 
成功构建4ffab7bc66c7 
  
如上所示，docker使用构建缓存。而且我这次更新require.txt。
 ＃requirements.txt 
 
 pytest == 2.3。 4 
 ipython 
  
以下是输出。
 
 $ b $发送构建上下文到Docker守护程序5.12 kB 
将构建上下文发送到Docker守护程序
步骤0：FROM dockerfile / python 
 ---> ; f86d6993fc7b 
步骤1：WORKDIR / srv 
 --->使用缓存
 ---> 55768a00fd94 
步骤2：ADD ./requirements.txt /srv/requirements.txt 
 ---> b6c19f0643b5 
删除中间容器a4d9cb37dff0 
步骤3：运行点安装-r requirements.txt 
 --->运行在4b7a85a64c33 
下载/解包pytest == 2.3.4（从-r requirements.txt（第1行））
运行setup.py（path：/tmp/pip_build_root/pytest/setup.py） egg_info for package pytest 
 
下载/解包ipython（从-r requirements.txt（第2行））
下载/解压缩py> = 1.4.12（从pytest == 2.3.4- > -r requirements.txt（第1行））
运行setup.py（路径：/tmp/pip_build_root/py/setup.py）egg_info for package py 
 
安装收集的软件包： pytest，python，py 
运行setup.py安装pytest 
 
安装py.test脚本到/ usr / local / bin 
安装py.test-2.7脚本到/ usr / local / bin 
运行setup.py install for py 
 
成功安装pytest ipython py 
清理... 
 ---> 23a1af3df8ed 
删除中间容器4b7a85a64c33 
步骤4：添加。 / srv 
 ---> d8ae270eca35 
删除中间容器7f003ebc3179 
步骤5：CMD python /srv/run.py 
 --->运行在510359cf9e12 
 ---> e42fc9121a77 
删除中间容器510359cf9e12 
成功构建e42fc9121a77 
  

而docker不使用构建缓存。

 客户端版本：1.1.2 
客户端API版本： 1.13 
 Go版本（客户端）：go1.2.1 
 Git提交（客户端）：d84a070 
服务器版本：1.1.2 
服务器API版本：1.13 
转到版本（服务器）：go1.2.1 
 Git commit（server）：d84a070 
  

My Dockerfile is something like

FROM my/base

ADD . /srv
RUN pip install -r requirements.txt
RUN python setup.py install

ENTRYPOINT ["run_server"]

Every time I build a new image, dependencies have to be reinstalled, which could be very slow in my region.

One way I think of to cache packages that have been installed is to override the my/base image with newer images like this:

docker build -t new_image_1 .
docker tag new_image_1 my/base

So next time I build with this Dockerfile, my/base already has some packages installed.

But this solution has two problems:

1. It is not always possible to override a base image
2. The base image grow bigger and bigger as newer images are layered on it

So what better solution could I use to solve this problem?

EDIT##:

Some information about the docker on my machine:

☁  test  docker version
Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070
☁  test  docker info
Containers: 0
Images: 56
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 56
Execution Driver: native-0.2
Kernel Version: 3.13.0-29-generic
WARNING: No swap limit support

解决方案

Try to build with below Dockerfile.

FROM my/base

WORKDIR /srv
ADD ./requirements.txt /srv/requirements.txt
RUN pip install -r requirements.txt
ADD . /srv
RUN python setup.py install

ENTRYPOINT ["run_server"]

If there are some changes on .(your project), docker skip pip install line by using cache.

Docker only run pip install on build when you edit requirements.txt file.

---

I write simple Hello, World! program.

$ tree
.
├── Dockerfile
├── requirements.txt
└── run.py   

0 directories, 3 file

# Dockerfile

FROM dockerfile/python
WORKDIR /srv
ADD ./requirements.txt /srv/requirements.txt
RUN pip install -r requirements.txt
ADD . /srv
CMD python /srv/run.py

# requirements.txt
pytest==2.3.4

# run.py
print("Hello, World")

Below is output.

Step 1 : WORKDIR /srv
---> Running in 22d725d22e10
---> 55768a00fd94
Removing intermediate container 22d725d22e10
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> 968a7c3a4483
Removing intermediate container 5f4e01f290fd
Step 3 : RUN pip install -r requirements.txt
---> Running in 08188205e92b
Downloading/unpacking pytest==2.3.4 (from -r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/pytest/setup.py) egg_info for package pytest
....
Cleaning up...
---> bf5c154b87c9
Removing intermediate container 08188205e92b
Step 4 : ADD . /srv
---> 3002a3a67e72
Removing intermediate container 83defd1851d0
Step 5 : CMD python /srv/run.py
---> Running in 11e69b887341
---> 5c0e7e3726d6
Removing intermediate container 11e69b887341
Successfully built 5c0e7e3726d6

I update only run.py and try to build again.

# run.py
print("Hello, Python")

Below is output.

Sending build context to Docker daemon  5.12 kB
Sending build context to Docker daemon 
Step 0 : FROM dockerfile/python
---> f86d6993fc7b
Step 1 : WORKDIR /srv
---> Using cache
---> 55768a00fd94
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> Using cache
---> 968a7c3a4483
Step 3 : RUN pip install -r requirements.txt
---> Using cache
---> bf5c154b87c9
Step 4 : ADD . /srv
---> 9cc7508034d6
Removing intermediate container 0d7cf71eb05e
Step 5 : CMD python /srv/run.py
---> Running in f25c21135010
---> 4ffab7bc66c7
Removing intermediate container f25c21135010
Successfully built 4ffab7bc66c7

As you can see above, docker use build cache. And I update requirements.txt this time.

# requirements.txt

pytest==2.3.4
ipython

Below is output.

Sending build context to Docker daemon  5.12 kB
Sending build context to Docker daemon 
Step 0 : FROM dockerfile/python
---> f86d6993fc7b
Step 1 : WORKDIR /srv
---> Using cache
---> 55768a00fd94
Step 2 : ADD ./requirements.txt /srv/requirements.txt
---> b6c19f0643b5
Removing intermediate container a4d9cb37dff0
Step 3 : RUN pip install -r requirements.txt
---> Running in 4b7a85a64c33
Downloading/unpacking pytest==2.3.4 (from -r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/pytest/setup.py) egg_info for package pytest

Downloading/unpacking ipython (from -r requirements.txt (line 2))
Downloading/unpacking py>=1.4.12 (from pytest==2.3.4->-r requirements.txt (line 1))
  Running setup.py (path:/tmp/pip_build_root/py/setup.py) egg_info for package py

Installing collected packages: pytest, ipython, py
  Running setup.py install for pytest

Installing py.test script to /usr/local/bin
Installing py.test-2.7 script to /usr/local/bin
  Running setup.py install for py

Successfully installed pytest ipython py
Cleaning up...
---> 23a1af3df8ed
Removing intermediate container 4b7a85a64c33
Step 4 : ADD . /srv
---> d8ae270eca35
Removing intermediate container 7f003ebc3179
Step 5 : CMD python /srv/run.py
---> Running in 510359cf9e12
---> e42fc9121a77
Removing intermediate container 510359cf9e12
Successfully built e42fc9121a77

And docker doesn't use build cache. If it doesn't work, check your docker version.

Client version: 1.1.2
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): d84a070
Server version: 1.1.2
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): d84a070

这篇关于为Python项目构建Docker映像时如何避免重新安装软件包？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！