在GitHub Actions工作流程中缓存APT软件包 [英] Caching APT packages in GitHub Actions workflow

问题描述

我将以下Github Actions工作流程用于我的C项目.工作流在大约40秒内完成，但是其中一半以上的时间是通过安装valgrind软件包及其依赖项来花费的.

I use the following Github Actions workflow for my C project. The workflow finishes in ~40 seconds, but more than half of that time is spent by installing the valgrind package and its dependencies.

我相信缓存可以帮助我加快工作流程.我不介意再等几秒钟，但这似乎对GitHub的资源毫无意义的浪费.

I believe caching could help me speed up the workflow. I do not mind waiting a couple of extra seconds, but this just seems like a pointless waste of GitHub's resources.

name: C Workflow

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1

    - name: make
      run: make

    - name: valgrind
      run: |
        sudo apt-get install -y valgrind
        valgrind -v --leak-check=full --show-leak-kinds=all ./bin

运行sudo apt-get install -y valgrind将安装以下软件包:

Running sudo apt-get install -y valgrind installs the following packages:

• gdb
• gdbserver
• libbabeltrace1
• libc6-dbg
• libipt1
• valgrind

• gdb
• gdbserver
• libbabeltrace1
• libc6-dbg
• libipt1
• valgrind

我知道Actions支持特定目录的缓存(并且已经有一些关于此的已解答的SO问题和文章)，但是我不确定apt安装的所有不同软件包的最终结果.我认为/bin/或/usr/bin/不是受安装软件包影响的唯一目录.

I know Actions support caching of a specific directory (and there are already several answered SO questions and articles about this), but I am not sure where all the different packages installed by apt end up. I assume /bin/ or /usr/bin/ are not the only directories affected by installing packages.

是否存在一种优雅的方式来缓存已安装的系统软件包以供将来的工作流运行?

推荐答案

此答案的目的是演示如何使用github操作完成缓存.不一定要显示它确实显示的如何缓存valgrind，而是要表明并非/应该缓存所有内容，并且需要考虑缓存和还原缓存与重新安装依赖项之间的权衡./em>

The purpose of this answer is to show how caching can be done with github actions. Not necessarily to show how to cache valgrind, which it does show, but also to show that not everything can/should be cached, and the tradeoffs of caching and restoring a cache vs reinstalling the dependency needs to be taken into account.

您将使用 actions/cache 操作执行此操作.

You will make use of the actions/cache action to do this.

将其添加为一个步骤(在需要使用valgrind之前):

Add it as a step (before you need to use valgrind):

- name: Cache valgrind
  uses: actions/cache@v1.0.3
  id: cache-valgrind
  with:
      path: "~/valgrind"
      key: ${{secrets.VALGRIND_VERSION}}

下一步应该尝试安装缓存的版本(如果有的话)或从存储库中安装:

The next step should attempt to install the cached version if any or install from the repositories:

- name: Install valgrind
  env:
    CACHE_HIT: ${{steps.cache-valgrind.outputs.cache-hit}}
    VALGRIND_VERSION: ${{secrets.VALGRIND_VERSION}}
  run: |
      if [[ "$CACHE_HIT" == 'true' ]]; then
        sudo cp --verbose --force --recursive ~/valgrind/* /
      else
        sudo apt-get install --yes valgrind="$VALGRIND_VERSION"
        mkdir -p ~/valgrind
        sudo dpkg -L valgrind | while IFS= read -r f; do if test -f $f; then echo $f; fi; done | xargs cp --parents --target-directory ~/valgrind/
      fi

说明

将VALGRIND_VERSION密码设置为以下内容的输出:

Explanation

Set VALGRIND_VERSION secret to be the output of:

apt-cache policy valgrind | grep -oP '(?<=Candidate:\s)(.+)'

这将使您仅通过更改密钥的值即可在发布新版本时使缓存无效.

this will allow you to invalidate the cache when a new version is released simply by changing the value of the secret.

dpkg -L valgrind用于列出使用sudo apt-get install valgrind时安装的所有文件.

dpkg -L valgrind is used to list all the files installed when using sudo apt-get install valgrind.

我们现在可以使用此命令将所有依赖项复制到我们的缓存文件夹中:

What we can now do with this command is to copy all the dependencies to our cache folder:

dpkg -L valgrind | while IFS= read -r f; do if test -f $f; then echo $f; fi; done | xargs cp --parents --target-directory ~/valgrind/

---

此外

除了复制valgrind的所有组件外，还可能需要复制依赖项(在这种情况下，例如libc)，但是我不建议您沿此路径继续，因为依赖关系链从那里开始发展.确切地说，要使最终具有适合valgrind的运行环境，需要复制的依赖项如下:

---

Furthermore

In addition to copying all the components of valgrind, it may also be necessary to copy the dependencies (such as libc in this case), but I don't recommend continuing along this path because the dependency chain just grows from there. To be precise, the dependencies needed to copy to finally have an environment suitable for valgrind to run in is as follows:

• libc6
• libgcc1
• gcc-8-base

要复制所有这些依赖项，可以使用与上面相同的语法:

To copy all these dependencies, you can use the same syntax as above:

for dep in libc6 libgcc1 gcc-8-base; do
    dpkg -L $dep | while IFS= read -r f; do if test -f $f; then echo $f; fi; done | xargs cp --parents --target-directory ~/valgrind/
done

当首先安装valgrind所需的全部操作只是简单地运行sudo apt-get install valgrind时，所有这些工作是否真的值得麻烦?如果您的目标是加快构建过程，那么您还必须考虑还原(下载和提取)缓存所花费的时间，而不是再次运行命令以安装valgrind.

Is all this work really worth the trouble when all that is required to install valgrind in the first place is to simply run sudo apt-get install valgrind? If your goal is to speed up the build process, then you also have to take into consideration the amount of time it is taking to restore (downloading, and extracting) the cache vs simply running the command again to install valgrind.

最后，要恢复缓存(假设它存储在/tmp/valgrind中)，可以使用以下命令:

And finally to restore the cache, assuming it is stored at /tmp/valgrind, you can use the command:

cp --force --recursive /tmp/valgrind/* /

基本上，这会将所有文件从缓存复制到根分区.

Which will basically copy all the files from the cache unto the root partition.

除了上述过程外，我还有一个示例.现在，缓存的大小约为63MB(压缩)，并且仍然需要单独安装libc，这无法达到目的.

In addition to the process above, I also have an example of "caching valgrind" by installing and compiling it from source. The cache is now about 63MB (compressed) in size and one still needs to separately install libc which kind of defeats the purpose.

参考:

• https://askubuntu.com/a/408785
• https://unix.stackexchange.com/questions/83593/copy-specific-file-type-care-the-folder-structure

• https://askubuntu.com/a/408785
• https://unix.stackexchange.com/questions/83593/copy-specific-file-type-keeping-the-folder-structure

这篇关于在GitHub Actions工作流程中缓存APT软件包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！