Cache Brew使用travis ci构建 [英] Cache Brew builds with travis ci

问题描述

我有一个Travis CI osx构建，该构建具有brew依赖关系，必须从源代码构建.

I have a Travis CI osx build with a brew dependency that has to be built from source.

我知道Travis具有cache功能，但是它没有有关如何缓存brew构建或输出的任何文档.

I know that Travis has the cache feature, but it doesn't have any documentation on how to cache brew builds or outputs.

关于如何在travis中缓存brew程序包的任何想法?

Any idea on how to cache a brew package in travis?

推荐答案

此处存在3个独立的，松散相关的问题:

• 缓存下载的瓶子
• 缓存本地制造的瓶子
• 缓存自制的元数据

• Cache downloaded bottles
• Cache locally-built bottles
• Cache Homebrew metadata

您不一定需要全部三个，所以请遵循适合您需要的任何部分.

You don't necessarily need all three, so follow whichever sections fit your needs.

• 将$HOME/Library/Caches/Homebrew添加到Travis的缓存_{(实际上，应该使用brew --cache检索此路径，但是您不能在这里调用它)}

• Add $HOME/Library/Caches/Homebrew to Travis' cache _{(actually, this path is supposed to be retrieved with brew --cache but you can't call it here, can you)}

cache:
  directories:
    - $HOME/Library/Caches/Homebrew

在before_cache阶段运行brew cleanup-否则，随着新软件包版本的发布，缓存将无限期增长

Run brew cleanup at before_cache stage -- otherwise, the cache will grow indefinitely as new package versions are released

before_cache:
  - brew cleanup

完整代码太长，无法在此处列出因此给出算法.

The full code is too long to list it here so giving the algorithm.

这是上一节的补充.如果不使用它，则在安装时在 步骤中将本地瓶保存在Homebrew缓存之外的某个位置，然后将其添加到以下位置的缓存中在下面的启动时步骤中使用适当的名称.

This is in addition to the previous section. If using without it, save local bottles somewhere outside of Homebrew cache at on installing step and add them to the cache under appropriate names at the at startup step below.

• 安装时:

• 用brew deps递归检查包的依赖关系
  • 如果您的环境无法使用该包装瓶(在brew info <pkg>输出中没有(bottled))，请在--include-build
  中包括构建依赖项
  • Check package's dependencies with brew deps recursively
    • If a bottle for the package is not available for your environment (no (bottled) in brew info <pkg> output), include build dependencies with --include-build

    对于每个程序包和依赖项，

    For each of the packages and dependencies,

    • 如果已经安装(brew list --versions <pkg>成功)和最新版本(brew outdated缺少)，请跳过
    • 如果存在旧版本，请在以下步骤中
    • If it's already installed (brew list --versions <pkg> succeeds) and latest version (absent from brew outdated), skip it
    • If an older version is present, in the following steps, you'll need to install the new version alongside the old one:
      • brew unlink the old version if it's not keg-only (no [keg-only] in brew info output)
      • Invoke all brew install's with --force

      如果没有瓶子，

      • 构建并安装 _{(似乎没有任何官方方法来获取生成的瓶子和JSON文件的名称.我从brew bottle输出中获取了瓶子的名称，并从中推断出JSON文件的名称.)}

        _{(There doesn't seem to be any official way to get the names of the resulting bottle and JSON file. I took the bottle name from brew bottle output and inferred JSON file name from it.)}

        将瓶子信息添加到包装的公式中

        Add the bottle info into the package's formula

        brew bottle --merge --write <json file>
        

      • 将瓶文件以brew --cache <pkg>

        • _{仅在添加瓶子信息之后执行此操作-否则，您将获得到源包的路径.}
        • _{(Homebrew还会在$HOME/Library/Caches/Homebrew中对下载的文件进行符号链接.您不需要这样做.)}
        • _{Only do this after adding bottle info -- otherwise, you'll get a path to the source package instead.}
        • _{(Homebrew also makes symlinks to downloaded files in $HOME/Library/Caches/Homebrew. You don't need to do this.)}

        启动时:

        • 如果要去，请执行brew update
        • 浏览已保存的.json文件.对于每一个，检查本地瓶子是否仍然合适(通过比较版本和重建编号；您可以对此数据解析brew info --json=v1 <pkg>和brew info --json=v1 <bottle>的输出).
          • 删除缓存的瓶子和.json(如果没有)
            • _{由于此时您无法使用brew --cache到达瓶子的路径，因此需要独立保存.在撰写本文时，符号链接尚未保存在Travis的缓存中，因此我最终使用了保存路径的常规文件.}
            • Do brew update if you're going to
            • Go through saved .json files. For each one, check if the local bottle is still appropriate (by comparing versions and rebuild numbers; you can parse the output of brew info --json=v1 <pkg> and brew info --json=v1 <bottle> for this data).
              • Delete the cached bottle and the .json if not
                • _{Since you won't be able to get the path to your bottle with brew --cache at this point, you need to have saved it independently. Symlinks aren't being saved in Travis' cache as of this writing, so I ended up using regular files that held the paths.}
                • _{在不改变版本的情况下更改公式中的下载URL的可能性也不大-瓶子的预期缓存名称将更改，因为其中的哈希是下载URL的哈希.为此，请在添加信息后检查brew --cache <pkg>是否仍然指向您的瓶子.}
                • _{There's also an unlikely possibility that they change the download URL in the formula without bumping the version -- then the bottle's expected cached name will change because the hash in it is the hash of the download URL. To allow for this, check if brew --cache <pkg> still points to your bottle after adding the info.}

                在before_cache:

                At before_cache:

                • 如果您正在使用上一节中的brew cleanup，请在运行之前将本地构建的瓶子文件保存在缓存中的某个位置，因为cleanup可能会删除这次不需要的文件. cleanup之后，恢复已删除的内容.
                • If you're using brew cleanup from the previous section, save your locally-built bottle files from cache somewhere before running it because cleanup may delete those that weren't needed this time around. After cleanup, restore those that were deleted.

                ^{(同样，完整代码是太长，因此无法给出算法.)}
                如果您运行brew update --verbose _{(并确保.travis.yml中没有秘密变量或您的Travis项目设置-brew仅当stdout是tty时，才会打印许多状态消息)}- -您将看到真正构成Homebrew自更新操作的内容-因此应缓存的内容:

                ^{(Again, the full code is too long so giving the algorithm.)}
                If you run brew update --verbose _{(and make sure there are no secret variables in .travis.yml or your Travis project settings -- brew prints many status messages only if stdout is a tty)} -- you'll see what exactly consititutes a Homebrew selfupdate operation -- thus what you should cache:

                • 将(实际上，默认情况下为rebase)拉入实际上是git存储库的一些路径:
                  • /usr/local/Homebrew-自制软件本身
                  • /usr/local/Homebrew/Library/Taps/*/*-已安装的水龙头
                  • Pulling (actually, rebase'ing by default) into a few paths that are actually git repositories:
                    • /usr/local/Homebrew -- Homebrew itself
                    • /usr/local/Homebrew/Library/Taps/*/* -- installed taps
                    • _{将始终尝试将Taps/caskroom/homebrew-cask迁移到Taps/homebrew/homebrew-cask，并在Taps/homebrew/homebrew-cask/homebrew-cask处创建副本.如果被缓存，则该副本将在下次运行时导致错误:文件存在".}
                    • 将始终尝试将许多未提交的文件导入到Taps/homebrew/homebrew-versions
                    • _{will always try migrating Taps/caskroom/homebrew-cask to Taps/homebrew/homebrew-cask, creating a copy at Taps/homebrew/homebrew-cask/homebrew-cask. If cached, this copy will cause an "error: file exists" on the next run.}
                    • will always try to import lots of non-committed files into Taps/homebrew/homebrew-versions

                    因此，操作将是:

                    • 将/usr/local/Homebrew添加到Travis缓存

                    • Add /usr/local/Homebrew to Travis cache

                    • 添加/usr/local/Cellar 和/usr/local/opt 是一个坏主意:首先，它们太大，在创建时会导致超时并上传缓存；其次，这是不安全的'cuz postinstall脚本可能会影响系统的其他任意部分，因此应每次从(缓存)的瓶子中安装新的软件包版本，而不是缓存结果.无论如何，安装瓶子只需要几秒钟.
                    • adding /usr/local/Cellar and /usr/local/opt proved to be a bad idea: first, they are too large, causing a timeout while making and uploading cache; second, this is unsafe 'cuz postinstall scripts may affect other arbitrary parts of the system, so one should install new package versions from (cached) bottles each time rather than cache the result. Installing a bottle only takes a few seconds anyway.

                    在brew update之前:清理Homebrew代码库

                    Before brew update: clean up Homebrew codebase

                    • 删除Taps/caskroom/homebrew-cask目录(如果Taps/homebrew/homebrew-cask存在)
                    • 在/usr/local/Homebrew下找到所有git仓库(find -type d -name .git，得到结果dirname)并在每个文件中运行git clean -fxd以摆脱Travis的剩余物
                    • 也可以使用brew cleanup从剩菜中清除Homebrew缓存(如果与上一节结合使用，请参见此处以了解其他操作)-否则，brew update中的正在迁移缓存条目..." 阶段.
                    • Delete the Taps/caskroom/homebrew-cask dir if Taps/homebrew/homebrew-cask exists
                    • Find all git repos under /usr/local/Homebrew (find -type d -name .git, get dirname of the result) and run git clean -fxd in each to get rid of Travis' leftovers
                    • Clean up Homebrew cache from leftovers, too, with brew cleanup (if using in conjunction with the previous section, see there for additional operations) -- otherwise, you'll get lots of errors in brew update on the "Migrating cache entries..." stage.

                    在brew update:

                    At brew update:

                    • 改为使用brew update --merge －它会自动解决与带有瓶子信息的本地提交的任何可能的冲突
                    • Use brew update --merge instead -- it will auto-resolve any possible conflicts with your local commits with bottle info

                    重新添加本地瓶子时(如果与上一节结合使用):

                    When re-adding local bottles (if using in conjunction with the previous section):

                    • 如果瓶子信息已经存在，请不要将其重新添加到公式中
                    • 如果包装版本已更改且您的奶瓶信息已出现在公式中，请从公式中将其删除，并git commit结果.没有通用的方法，因此您必须使用脚本来解析和编辑公式文件，并从bottle do表中删除相应的行.公式文件的路径是使用brew formula <pkg>检索的.
                    • Don't re-add bottle info into the formula if it's already present there
                    • If package version has changed and your bottle info is present in the formula, remove it from the formula and git commit the result. There's no stock way to do that, so you'll have to parse and edit the formula file with a script and delete the corresponsing line from bottle do table. The path to the formula file is retrieved with brew formula <pkg>.

                    安装时:

                    • 如果使用第三方水龙头，请始终检查是否已安装该水龙头:

                    • If using 3rd-party taps, always check if you already have that tap installed:

                    brew tap | grep -qxF <tap> || brew tap <tap>
                    

                    • 由于符号链接未保存在Travis缓存中，因此可能不会记住引脚.但是，对它们进行检查也不会造成伤害:

                      • Since symlinks aren't saved in Travis cache, pins will likely not be remembered. But it won't hurt to check for them, too:

                        brew tap --list-pinned | grep -qxF <tap> || brew tap-pin <tap>
                        

                      在before_cache:

                      At before_cache:

                      • 删除Taps/homebrew/homebrew-cask/homebrew-cask(如果存在)
                      • Delete Taps/homebrew/homebrew-cask/homebrew-cask if it exists

                      这篇关于Cache Brew使用travis ci构建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！