Python运行时：重新编译和重用C库 [英] Python runtime: recompiling and reusing C library

问题描述

我正在开发一种工具，用于对用户定义函数进行一些数值分析。
的想法是在Python中创建一个方便的UI，用户可以在其中输入C函数，然后按一个按钮-并接收一些输出数据。
计算可能要花费几分钟或几小时，因此仅Numpy性能是不可接受的。

I am developing a tool for some numerical analysis of user-defined functions. The idea is to make a convenient UI in Python, where user can enter C function, then press a button - and receive some output data. Computations can take minutes or hours, so Numpy-only performance is not acceptable.

我尝试了以下方法：基于Python的UI调用gcc，从用户函数编译dll，该dll比我在Cython包装中基于C的核心算法所使用的函数还要多。它可以工作，但是由于无法完全卸载python模块，因此在关闭整个UI程序并再次运行之前，我无法重新编译用户定义的函数。

I have tried the following approach: the Python-based UI calls gcc, compiles dll from user functions that is than used by my core C-based algorithms in Cython wrappings. It works, but since there is no way to fully unload the python module, I can not recompile user-defined function until the whole UI program is closed and run again.

我现在看到的唯一方法是将计算核心和UI流程分开，然后使它们通过共享内存/消息进行交互。当用户想要更新其功能时，该程序将终止内核，重新编译dll并再次启动内核。

The only way I see now is to separate the computational core and UI processes and then make them interact via shared memory/messaging. As user wants to update his function, the program terminates the core, recompiles dll and starts the core again.

在这种情况下，您可以建议任何常规做法吗？

Can you suggest any common practice in such cases?

谢谢！

推荐答案

Python确实非常好多重处理支持（实际上不是很好的线程支持），因此您可以为每个表达式生成一个新的python进程评估，编译并在新进程中加载​​dll，然后让其将结果传达回父进程。当产生的进程退出时，应卸载所有内容。

Python has really good multiprocessing support (and really not very good threading support), so you could spawn a new python process for each expression to be evaluated, compile and load the dll in the new process, then have it communicate the results back to the parent process. When the spawned process exits, everything should be unloaded.

这篇关于Python运行时：重新编译和重用C库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！