我目前有一个项目,我们大量使用Cython来加快我们的所有子模型/类的速度,我希望在Linux系统上使用clang
而不是gcc
编译它们。为此,我目前具有以下方式收集所有扩展的函数:
def extensions():
try:
from Cython.Build import cythonize
except ImportError:
def cythonize(*args, **kwargs):
print("Hint: Wrapping import of cythonize in extensions()")
from Cython.Build import cythonize
return cythonize(*args, **kwargs)
try:
import numpy
lstIncludes = [numpy.get_include()]
except ImportError:
lstIncludes = []
extensionArguments = {
'include_dirs':
lstIncludes + ['package/core', 'package/inspect', 'util'],
'extra_compile_args': compilerArguments,
'extra_link_args': linkerArguments,
'define_macros': defineMacros
}
return cythonize(
[Extension("*", ["package/*.pyx"], **extensionArguments),
Extension("*", ["package/algs/*.pyx"], **extensionArguments),
Extension("*", ["package/core/*.pyx"], **extensionArguments)],
compiler_directives=cythonDirectives)
此外,我设置
os.environ["CC"] = "clang"
os.environ["CXX"] = "clang"
在setup.py
中,当我调用python setup.py build_ext --inplace
时,第一个Cython扩展是使用clang
构建的,但第二个是用gcc
构建的。如果我调用
export CC=clang; python setup.py build_ext --inplace
然后,再次使用clang
构建了Cythonized Extensions的第一个未编译,但第二个以及以下所有内容都是使用gcc
构建的。
如何在编译过程中设置和修复所有Cython模块的编译器?
编辑:经过几周的考虑,我仍然没有上述问题的解决方案,并且也发生在多个机器上,因此这不是我特定设置的结果。
我终于发现了,必须强行覆盖disutils
从我们的系统配置中提取的东西。为此,必须通过以下方式修改上述代码
from distutils import sysconfig
def extensions():
'''
Handle generation of extensions (a.k.a "managing cython compilery").
'''
try:
from Cython.Build import cythonize
except ImportError:
def cythonize(*args, **kwargs):
print("Hint: Wrapping import of cythonize in extensions()")
from Cython.Build import cythonize
return cythonize(*args, **kwargs)
try:
import numpy
lstIncludes = [numpy.get_include()]
except ImportError:
lstIncludes = []
extensionArguments = {
'include_dirs':
lstIncludes + ['package/core', 'package/inspect', 'util'],
'extra_compile_args': compilerArguments,
'extra_link_args': linkerArguments,
'define_macros': defineMacros
}
# me make damn sure, that disutils does not mess with our
# build process
sysconfig.get_config_vars()['CFLAGS'] = ''
sysconfig.get_config_vars()['OPT'] = ''
sysconfig.get_config_vars()['PY_CFLAGS'] = ''
sysconfig.get_config_vars()['PY_CORE_CFLAGS'] = ''
sysconfig.get_config_vars()['CC'] = 'gcc'
sysconfig.get_config_vars()['CXX'] = 'g++'
sysconfig.get_config_vars()['BASECFLAGS'] = ''
sysconfig.get_config_vars()['CCSHARED'] = '-fPIC'
sysconfig.get_config_vars()['LDSHARED'] = 'gcc -shared'
sysconfig.get_config_vars()['CPP'] = ''
sysconfig.get_config_vars()['CPPFLAGS'] = ''
sysconfig.get_config_vars()['BLDSHARED'] = ''
sysconfig.get_config_vars()['CONFIGURE_LDFLAGS'] = ''
sysconfig.get_config_vars()['LDFLAGS'] = ''
sysconfig.get_config_vars()['PY_LDFLAGS'] = ''
return cythonize(
[Extension("*", ["package/*.pyx"], **extensionArguments),
Extension("*", ["package/algs/*.pyx"], **extensionArguments),
Extension("*", ["package/core/*.pyx"], **extensionArguments)],
compiler_directives=cythonDirectives,
nthreads=4
)
以这种方式,人们可以完全控制构建扩展时发生的任何事情。