使用Cython和Setuptools来编译多个扩展时,请设置默认编译器



我目前有一个项目,我们大量使用Cython来加快我们的所有子模型/类的速度,我希望在Linux系统上使用clang而不是gcc编译它们。为此,我目前具有以下方式收集所有扩展的函数:

def extensions():
    try:
        from Cython.Build import cythonize
    except ImportError:
        def cythonize(*args, **kwargs):
            print("Hint: Wrapping import of cythonize in extensions()")
            from Cython.Build import cythonize
            return cythonize(*args, **kwargs)
    try:
        import numpy
        lstIncludes = [numpy.get_include()]
    except ImportError:
        lstIncludes = []
    extensionArguments = {
        'include_dirs':
        lstIncludes + ['package/core', 'package/inspect', 'util'],
        'extra_compile_args': compilerArguments,
        'extra_link_args': linkerArguments,
        'define_macros': defineMacros
    }
    return cythonize(
        [Extension("*", ["package/*.pyx"], **extensionArguments),
         Extension("*", ["package/algs/*.pyx"], **extensionArguments),
         Extension("*", ["package/core/*.pyx"], **extensionArguments)],
        compiler_directives=cythonDirectives)

此外,我设置

os.environ["CC"] = "clang"
os.environ["CXX"] = "clang"

setup.py中,当我调用python setup.py build_ext --inplace时,第一个Cython扩展是使用clang构建的,但第二个是用gcc构建的。如果我调用

,这种行为不会改变
export CC=clang; python setup.py build_ext --inplace

然后,再次使用clang构建了Cythonized Extensions的第一个未编译,但第二个以及以下所有内容都是使用gcc构建的。

如何在编译过程中设置和修复所有Cython模块的编译器?

编辑:经过几周的考虑,我仍然没有上述问题的解决方案,并且也发生在多个机器上,因此这不是我特定设置的结果。

我终于发现了,必须强行覆盖disutils从我们的系统配置中提取的东西。为此,必须通过以下方式修改上述代码

来分别为每个扩展程序设置某些选项。
from distutils import sysconfig
def extensions():
    '''
    Handle generation of extensions (a.k.a "managing cython compilery").
    '''
    try:
        from Cython.Build import cythonize
    except ImportError:
        def cythonize(*args, **kwargs):
            print("Hint: Wrapping import of cythonize in extensions()")
            from Cython.Build import cythonize
            return cythonize(*args, **kwargs)
    try:
        import numpy
        lstIncludes = [numpy.get_include()]
    except ImportError:
        lstIncludes = []
    extensionArguments = {
        'include_dirs':
        lstIncludes + ['package/core', 'package/inspect', 'util'],
        'extra_compile_args': compilerArguments,
        'extra_link_args': linkerArguments,
        'define_macros': defineMacros
    }
    # me make damn sure, that disutils does not mess with our
    # build process
    sysconfig.get_config_vars()['CFLAGS'] = ''
    sysconfig.get_config_vars()['OPT'] = ''
    sysconfig.get_config_vars()['PY_CFLAGS'] = ''
    sysconfig.get_config_vars()['PY_CORE_CFLAGS'] = ''
    sysconfig.get_config_vars()['CC'] = 'gcc'
    sysconfig.get_config_vars()['CXX'] = 'g++'
    sysconfig.get_config_vars()['BASECFLAGS'] = ''
    sysconfig.get_config_vars()['CCSHARED'] = '-fPIC'
    sysconfig.get_config_vars()['LDSHARED'] = 'gcc -shared'
    sysconfig.get_config_vars()['CPP'] = ''
    sysconfig.get_config_vars()['CPPFLAGS'] = ''
    sysconfig.get_config_vars()['BLDSHARED'] = ''
    sysconfig.get_config_vars()['CONFIGURE_LDFLAGS'] = ''
    sysconfig.get_config_vars()['LDFLAGS'] = ''
    sysconfig.get_config_vars()['PY_LDFLAGS'] = ''
    return cythonize(
        [Extension("*", ["package/*.pyx"], **extensionArguments),
         Extension("*", ["package/algs/*.pyx"], **extensionArguments),
         Extension("*", ["package/core/*.pyx"], **extensionArguments)],
        compiler_directives=cythonDirectives,
        nthreads=4
    )

以这种方式,人们可以完全控制构建扩展时发生的任何事情。

最新更新