如何在多个C扩展名之间共享C-Singleton [英] How to share a C-singleton between multiple C-extensions

问题描述

我有一个静态库(或一堆c/cpp文件)，其中包含一个单例，并由两个不同的C扩展名使用/链接.但是，C库中的单例不再具有单例的作用:

I have a static library (or a bunch of c/cpp-files for that matter) which contains a singleton and is used by/linked to two different C-extensions. However, the singleton from the C-library doesn't behave like a singleton anymore:

import getter
import setter

# set singleton:
setter.set(21)
# get singleton:
print("singleton: ", getter.get()) 
#prints the old value:42

这里是一个最小的示例，为简单起见，使用Cython演示了此问题(所有文件都在同一文件夹中):

Here is a minimal example illustrating this problem using Cython for the sake of simplicity (all files are in the same folder):

C库:

//lib.h:
int get_singleton(void);
void set_singleton(int new_val);

//lib.c:
#include "lib.h"

static int singleton=42;
int get_singleton(void){
    return singleton;
}
void set_singleton(int new_val){
    singleton=new_val;
}

两个Cython扩展名:

The two Cython extensions:

# getter.pyx:
#cython: language_level=3

cdef extern from "lib.h":
    int get_singleton()

def get():
    return get_singleton()

# setter.pyx:
#cython: language_level=3

cdef extern from "lib.h":
    void set_singleton(int new_val);

def set(new_val):
    set_singleton(new_val)

此 SO-post 之后的设置文件:

#setup.py
from setuptools import setup, find_packages, Extension

setup(
      name='singleton_example',
      ext_modules=[Extension('getter', sources=['getter.pyx']), 
                   Extension('setter', sources=['setter.pyx']),],
      # will be build as static libraries and automatically passed to linker for all extensions:
      libraries = [('static_lib', {'sources': ["lib.c"]}) ] 
     )

通过 python setup.py build_clib build_ext --inplace 构建后，即可运行上述python脚本.

After building via python setup.py build_clib build_ext --inplace, the above python script can be run.

在多个(Cython)-C扩展名之间共享C-singleton的正确方法是什么?

What is the correct way to share a C-singleton between multiple (Cython)-C-extensions?

推荐答案

当前的问题是变量 singleton 存在两次:一次在扩展名 setter 中，一次在扩展名 getter 中(还有函数 get_singleton 和 set_singleton 存在两次，即每个都有两个不同的地址)，这或多或少地违反了一个定义规则( ODR )，即使该规则仅存在于C ++中.违反ODR并不是世界末日，但是在大多数情况下，此行为变得不可移植，因为不同的链接器/编译器/OS对这种情况的处理方式不同.

The problem at hand is that the variable singleton exists twice: once in the extension setter and once in the extension getter (also functions get_singleton and set_singleton exist twice, i.e. having two different addresses each), this is more or less a violation of one definition rule (ODR) even if this rule only exists in C++. Violation of ODR is not the end of the world, but in most cases the behavior becomes not portable because different linkers/compilers/OSes handle this situation differently.

例如，对于Linux上的共享库，我们使用符号插入.但是，Python使用 ldopen 而没有 RTLD_GLOBAL (用 RTLD_LOCAL 隐式表示)来加载C扩展名，从而防止符号插入.我们可以在Python中强制使用 RTLD_GLOBAL :

For example, for shared libraries on Linux we have the symbol-interposition. However, Python uses ldopen without RTLD_GLOBAL (means implicitly with RTLD_LOCAL) for loading of C-extensions, thus preventing symbol-interposition. We could enforce the usage of RTLD_GLOBAL in Python:

import sys; import ctypes;
sys.setdlopenflags(sys.getdlopenflags() | ctypes.RTLD_GLOBAL)

在导入 getter 和 setter 之前，再次恢复单例属性.但是，这在Windows上不起作用，因为dll不支持符号插入.

prior to importing getter and setter and restore the singleton-property again. This however would not work on Windows, because dlls don't support symbol-interposition.

确保单一属性"的可移植方式.这是为了避免违反ODR，并且为了实现这一点，应该使静态库/文件动态化.该动态库仅由该过程加载一次，因此确保了我们只有一个 singleton .

The portable way to ensure the "singleton-property" is to avoid a violation of ODR and in order to achieve that one should make the static library/bunch of files dynamic. This dynamic library will be loaded only once by the process, thus ensuring that we have only one singleton.

根据情况，有一些如何使用此dll的选项:

Depending on the scenario, there are some options how this dll can be used:

1. 扩展只能在本地使用，而不能通过共享对象(请参阅 SO-post )进行分发，或者是dll(参见此 SO-post ).
2. 扩展仅在某些平台上分发，然后可以预先构建共享的对象/dll并像第三方库一样分发它们，例如，请参见此符号的结果，直到运行时为止，而不是基础操作系统的链接器/加载器，这使得例如稍后更容易根据动态库创建进一步的扩展.

1. the extensions are used only locally and not distributed, using shared objects (see this SO-post) or dll (see this SO-post).
2. extensions are only distributed on some platforms, then it is possible to prebuild shared objects/dlls and distribute them like a third-party library, see for example this SO-post
3. it is possible to override setuptools' build_clib command, so it will build a shared object/dll instead of a static library, which will be used when extensions are linked and copied to the installation. However, adding more extension, which would use this very dll is quite cumbersome (even if not impossible).
4. it is possible to write a cython-wrapper of the dll, which has the advantage of using the Python's machinery for loading and posponing the resultion of the symbols until the run-time, rather than linkers/loaders of the underlying OSes, which makes it for example easier to create futher extensions depending on the dynamic library later on.

我认为默认情况下应使用最后一种方法.这是一个可能的实现:

I think the last approach should be used by default. Here is a possible implementation:

1. 创建静态库的包装，并通过 pxd 文件公开其功能:

1. Create a wrapper of the static library and expose its functionality via a pxd-file:

# lib_wrapper.pxd
cdef int get_singleton()
cdef void set_singleton(int new_value)

#lib_wrapper.pyx
cdef extern from "lib.h":
    int c_get_singleton "get_singleton" ()
    void c_set_singleton "set_singleton" (int new_val)

cdef int get_singleton():
    return c_get_singleton()

cdef void set_singleton(int new_val):
    c_set_singleton(new_val)

一个重要部分:包装器引入了一个间接级别(因此引发了许多应该自动执行的样板代码编写)，因此，在其他模块中使用它时，既不需要头文件也不使用c文件/库

An important part: the wrapper introduces a level of indirection (thus inducing a lot of boiler-plate code writing which should be automatized), thus when using it in further modules neither header-files nor c-files/libraries are needed.

1. 调整其他模块，它们仅需 cimport 包装器:

# getter.pyx:
#cython: language_level=3
cimport lib_wrapper
def get():
    return lib_wrapper.get_singleton()

# setter.pyx:
#cython: language_level=3
cimport lib_wrapper
def set(new_val):
    lib_wrapper.set_singleton(new_val)

1. 设置不再需要 build_clib -步骤:

from setuptools import setup, find_packages, Extension

setup(
      name='singleton_example',
      ext_modules=[Extension('lib_wrapper', sources=['lib_wrapper.pyx', 'lib.c']),
                   Extension('getter', sources=['getter.pyx']), 
                   Extension('setter', sources=['setter.pyx']),],
     )

通过 python setup.py build_ext --inplace 构建后(在源代码发行版中，即 python setup.py build sdist )，h文件将丢失，但是对于这个问题，有很多不同的解决方案)该示例将 set / get 设置为相同的单例(因为只有一个).

After building via python setup.py build_ext --inplace (in the source distribution, i.e. python setup.py build sdist the h-file will be missing, but there are many different solutions for this problem possible) the example will set/get the same singleton (because there is only one).

这篇关于如何在多个C扩展名之间共享C-Singleton的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！