pyinstaller 可执行文件的差异更新(修改嵌入式 PYZ-00.pyz) [英] differential update of pyinstaller executable (modify embedded PYZ-00.pyz)

查看:60
本文介绍了pyinstaller 可执行文件的差异更新(修改嵌入式 PYZ-00.pyz)的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我打算创建一个巨大的可执行目录并将其安装在某些设备上.

I'm planning to create a huge executable directory and install it on some devices.

想象一下,后来我在我的一个 Python 模块中发现了一个错误.有没有什么办法只传输/复制修改后的字节码,然后用新的字节码替换原来的字节码.

Imagine, that lateron I discover a bug in one of my python modules. Is there any way to transfer/copy only the modified byte code and replace the original byte code with the new one.

我想这样做的原因是,在我的上下文中带宽非常昂贵,我想远程修补代码.

The reason I want to do this is, that in my context bandwidth is very expensive and I'd like to patch the code remotely.

示例:我有一个包含两个文件的项目:prog.py:(以下三行)

Example: I have a project with two files: prog.py: (with following three lines)

import mod1
if __name__ == "__main__":
    mod1.hello()

mod1.py:(以下两行)

def hello():
    print("hello old world")

现在我使用 PYTHONHASHSEED=2 pyinstaller prog.py 创建我的目录并将其复制到我的设备

Now I use PYTHONHASHSEED=2 pyinstaller prog.py to create my directory which I copy to my device

现在我修改mod1.py:

def hello():
    print("hello new world")

然后我用 PYTHONHASHSEED=2 pyinstaller prog.py 重新编译完整目录(去皮和压缩)大小约为 10Mdist/prog/prog 文件大小约为 1M

and I recompile with PYTHONHASHSEED=2 pyinstaller prog.py The full directory has (tared and gzipped) a size of about 10M The file dist/prog/prog has a size of about 1M

使用 pyi-archive_viewer 我可以从我的可执行文件 dist/prog/prog 中提取 PYZ-00.pyzPYZ-00.pyz 中,我可以找到并提取仅使用 133 个字节的 mod1.

with pyi-archive_viewer I can extract PYZ-00.pyz out of my executable dist/prog/prog In PYZ-00.pyz I can find and extract mod1 which uses only 133 bytes.

现在,如果我将该文件复制到我的设备上,我该如何更新旧的dist/prog/prog 这样,它就有了新的 PYZ-00.pyz:mod1 字节码.

Now if I copy that file to my device, how could I update the old dist/prog/prog such, that it has the new PYZ-00.pyz:mod1 byte code.

我可以用什么代码来分解,我可以用什么代码在替换了一个特定的文件(模块)后重新组装?

What code could I use to decompose, what code could I use to reassemble after having replaced one specific file (module)?

替代方法:将 pyc 文件移动到 zip 文件启动性能不是那么重要.我也可以使用替代解决方案,其中没有创建 PYZ 文件并将其添加到可执行文件中,但是 dist 目录包含一个包含所有 .pyc 文件的 zip 文件

Alternative: Move pyc files to a zip file Startup performance is not that crucial. I could also live with an alternative solution, where no PYZ file is created and added to the executable, but where the dist directory contains a zip file with all the .pyc files

另一种选择:将 .pyc 文件复制到应用程序目录中这将导致 __file__ 具有与 PYZ 模式完全相同的值.性能方面可能没有那么好,并且会创建大量文件,但如果增量更新至关重要,那么也许可以选择一种处理方式.

Another alternative: copy .pyc files into application directory This would result in __file__ having exactly the same value as in the PYZ mode. Performance wise probably not that nice and creating a lot of files, but if incremental updates are crucial perhaps one option to handle it.

推荐答案

此解决方案既不能修补".PYZ 文件,也不能放置所有 .pyc> 将文件压缩为 zip 文件.

This solution is neither capable of 'patching' a .PYZ file nor capable of putting all .pyc files into a zip file.

但到目前为止,它是我找到的唯一可行的解​​决方案,适用于具有大量第三方依赖项的大型项目.

But so far it is the only viable solution I found so far, that works for huge projects with loads of third party dependencies.

这个想法是删除所有(或大部分文件从 .PYZ 文件)并将相应的 .pyc 文件复制到工作目录中.

The idea is to remove all (or most files from the .PYZ file) and copy the corresponding .pyc files into the working directory.

我会随着时间的推移加强和详细说明这个答案.我还在试验中:

I will enhance and elaborate this answer over time. I'm still experimenting:

我通过修改规范文件来实现这一点:

I achieve this by modyfing the spec file:

  • 确定spec文件所在的目录MYDIR
  • 创建一个目录,MYDIR/src,其中所有来自a.pure的文件都将被复制到
  • 将所有文件从 a.pure 复制到 MYDIR/src.(具有与模块名称相对应的子目录.例如,模块 mypackage.mod.common 将存储在 MYDIR/src/mypackage/mod/common.py 中)
  • 遍历文件并将它们编译为 .pyc 文件,然后删除 .py 文件.
  • 创建一个 PYZ 文件,其中仅包含未复制的文件.(在我的测试用例中,PYZ 中不保留 .pyc 文件)
  • 使用修改后的PYZ
  • 创建exe
  • 收集所有应该收集的文件以及来自 MYDIR/src 的所有文件(例如使用 a.datas + Tree(src")
  • determine the directory MYDIR where the spec file is located in
  • create a directory, MYDIR/src where all the files from a.pure shall be copied to
  • copy all files from a.pure to to MYDIR/src. (with subdirectories corresponding to the module's name. Module mypackage.mod.common would for example be stored in MYDIR/src/mypackage/mod/common.py)
  • iterate through files and compile them to a .pyc file and remove .py file afterwards.
  • create a PYZ file which contains only the files that are not copied. (in my test case, keep no .pyc file in the PYZ)
  • create exe with the modified PYZ
  • collect all files that should be collected plus also all files from MYDIR/src (e.g. with a.datas + Tree("src")

规范文件更改:一开始

import os
MYDIR = os.path.realpath(SPECPATH)
sys.path.append(MYDIR)
import mypyinsthelpers  # allows to reuse the code in multiple projects

然后在(未修改的)a = Analysis(... 部分之后我添加.

Then after the (unmodified) a = Analysis(... section I add.

to_rmv_from_pyc = mypyinsthelpers.mk_copy_n_compile(a.pure, MYDIR)

# modified creation of pyz`
pyz = PYZ(a.pure - to_rmv_from_pyc, a.zipped_data,
             cipher=block_cipher)

我会进一步详细说明mypyinsthelpers.mk_copy_n_compile函数

I will detail the function mypyinsthelpers.mk_copy_n_compile further down

更改收集阶段:

代替

coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
...

我写道:

coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas + Tree("src"),
...

这里是mypyinsthelpers.mk_copy_n_compile()

import compileall
import os
import shutil
from pathlib import Path


def mk_copy_n_compile(toc, src_tree):
    """
    - copy source files to a destination directory
    - compile them as pyc
    - delete source
    """
    dst_base_path = os.path.join(src_tree, "src")
    to_rm = []
    # copy files to destination tree
    for entry in toc:
        modname, src, typ = entry
        assert typ == "PYMODULE"
        assert src.endswith(".py") or src.endswith(".pyw")
        # TODO: might add logic to skip some files (keep them in PYC)
        to_rm.append(entry)

        if src.endswith("__init__.py"):
            modname += ".__init__"

        m_split = modname.split(".")
        m_split[-1] += ".py"
        dst_dir = os.path.join(dst_base_path, *m_split[:-1])
        dst_path = os.path.join(dst_dir, m_split[-1])
        if not os.path.isdir(dst_dir):
            os.makedirs(dst_dir)
        print(entry[:2], dst_path)
        shutil.copy(src, dst_path)

    # now compile all files and rmv src
    top_tree = src_tree
    src_tree = os.path.join(src_tree, "src")
    curdir = os.getcwd()
    os.chdir(dst_base_path)
    for path in Path(dst_base_path).glob("**/*.py"):
        # TODO: might add code to keep some files as source
        compileall.compile_file(
            str(path.relative_to(dst_base_path)), quiet=1, legacy=True)
        path.unlink()
    os.chdir(curdir)
    return to_rm

这篇关于pyinstaller 可执行文件的差异更新(修改嵌入式 PYZ-00.pyz)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆