Conda包bug?二进制兼容



我在一个没有root访问权限的系统上使用远程Jupyter笔记本,甚至没有可以进行许多调整的shell。我可以从Conda的存档中检索软件包,并在笔记本单元中运行安装如下软件包的函数

!conda install /path/to/package-vvv.tar.bz2

我遇到的情况,我猜错了版本号,安装的东西是不兼容的。错误消息如下所示,在numpy或mkl中二进制不兼容。

现在我在Ubuntu 20.10笔记本上重新跟踪问题,我有管理权限。我有一个可重复的问题要展示和分享。

  1. 创建一个python, numpy和pandas版本相同的环境,就像我们在远程机器上一样:
$ conda create -n cenv-py368 python=3.6.8 pandas=1.1.2 numpy=1.15.4
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.5.12
latest version: 4.9.2
Please update conda by running
$ conda update -n base -c defaults conda
## Package Plan ##
environment location: /home/pauljohn/LinuxDownloads/miniconda3/envs/cenv-py368
added / updated specs:
- numpy=1.15.4
- pandas=1.1.2
- python=3.6.8

The following packages will be downloaded:
package                    |            build
---------------------------|-----------------
libffi-3.2.1               |    hf484d3e_1007          52 KB
python-3.6.8               |       h0371630_0        34.4 MB
libgcc-ng-9.1.0            |       hdf63c60_0         8.1 MB
libstdcxx-ng-9.1.0         |       hdf63c60_0         4.0 MB
blas-1.0                   |              mkl           6 KB
_libgcc_mutex-0.1          |             main           3 KB
------------------------------------------------------------
Total:        46.6 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex:   0.1-main
blas:            1.0-mkl
ca-certificates: 2021.1.19-h06a4308_0
certifi:         2020.12.5-py36h06a4308_0
intel-openmp:    2020.2-254
libedit:         3.1.20191231-h14c3975_1
libffi:          3.2.1-hf484d3e_1007
libgcc-ng:       9.1.0-hdf63c60_0
libgfortran-ng:  7.3.0-hdf63c60_0
libstdcxx-ng:    9.1.0-hdf63c60_0
mkl:             2020.2-256
mkl-service:     2.3.0-py36he8ac12f_0
mkl_fft:         1.2.0-py36h23d657b_0
mkl_random:      1.1.1-py36h0573a6f_0
ncurses:         6.2-he6710b0_1
numpy:           1.15.4-py36h7e9f1db_0
numpy-base:      1.15.4-py36hde5b4d6_0
openssl:         1.1.1i-h27cfd23_0
pandas:          1.1.2-py36he6710b0_0
pip:             20.3.3-py36h06a4308_0
python:          3.6.8-h0371630_0
python-dateutil: 2.8.1-pyhd3eb1b0_0
pytz:            2021.1-pyhd3eb1b0_0
readline:        7.0-h7b6447c_5
setuptools:      52.0.0-py36h06a4308_0
six:             1.15.0-pyhd3eb1b0_0
sqlite:          3.33.0-h62c20be_0
tk:              8.6.10-hbc83047_0
wheel:           0.36.2-pyhd3eb1b0_0
xz:              5.2.5-h7b6447c_0
zlib:            1.2.11-h7b6447c_3
Proceed ([y]/n)? y

Downloading and Extracting Packages
libffi-3.2.1         | 52 KB     | ##################################### | 100%
python-3.6.8         | 34.4 MB   | ##################################### | 100%
libgcc-ng-9.1.0      | 8.1 MB    | ##################################### | 100%
libstdcxx-ng-9.1.0   | 4.0 MB    | ##################################### | 100%
blas-1.0             | 6 KB      | ##################################### | 100%
_libgcc_mutex-0.1    | 3 KB      | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate cenv-py368
#
# To deactivate an active environment, use
#
#     $ conda deactivate
  1. 激活环境

  2. 安装,例如,名为" fastparquette ":

(cenv-py368) $ conda install fastparquet
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.12
latest version: 4.9.2
Please update conda by running
$ conda update -n base -c defaults conda

## Package Plan ##
environment location: /home/pauljohn/LinuxDownloads/miniconda3/envs/cenv-py368
added / updated specs:
- fastparquet
The following packages will be downloaded:
package                    |            build
---------------------------|-----------------
pyparsing-2.4.7            |     pyhd3eb1b0_0          59 KB
packaging-20.9             |     pyhd3eb1b0_0          35 KB
------------------------------------------------------------
Total:          95 KB
The following NEW packages will be INSTALLED:
fastparquet: 0.5.0-py36h6323ea4_1
libllvm10:   10.0.1-hbcb73fb_5
llvmlite:    0.34.0-py36h269e1b5_4
numba:       0.51.2-py36h0573a6f_1
packaging:   20.9-pyhd3eb1b0_0
pyparsing:   2.4.7-pyhd3eb1b0_0
thrift:      0.11.0-py36hf484d3e_0
Proceed ([y]/n)? y

Downloading and Extracting Packages
pyparsing-2.4.7      | 59 KB     | ##################################### | 100%
packaging-20.9       | 35 KB     | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
  1. 观察导入失败
(cenv-py368) $ python
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fastparquet
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/pauljohn/LinuxDownloads/miniconda3/envs/cenv-py368/lib/python3.6/site-packages/fastparquet/__init__.py", line 5, in <module>
from .core import read_thrift
File "/home/pauljohn/LinuxDownloads/miniconda3/envs/cenv-py368/lib/python3.6/site-packages/fastparquet/core.py", line 9, in <module>
from . import encoding
File "/home/pauljohn/LinuxDownloads/miniconda3/envs/cenv-py368/lib/python3.6/site-packages/fastparquet/encoding.py", line 13, in <module>
from .speedups import unpack_byte_array
File "fastparquet/speedups.pyx", line 1, in init fastparquet.speedups
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject
>>> AA

你同意我发现一个bug吗?

似乎Conda应该工作,或者它应该说没有兼容的fastparquet版本。

该错误通常表明NumPy比使用它的库(在本例中为fastparquet)的兼容性更老。尝试将Python版本更新到3.7或3.8;Python 3.6和NumPy 1.15目前不在推荐版本范围内。(将Python更新到3.7+也会更新NumPy;当你做conda update ...时,通常不会这样做。一些食谱pin到>=一些最小的版本,这一个似乎没有。

https://numpy.org/neps/nep - 0029 deprecation_policy.html #支持表格

这是您正在导入的某些Python库的准备过程中的一个缺陷。当像fastparquet这样的包的作者没有正确地为他们的包设置numpy或python的最小兼容版本时,Conda环境协调无法知道包是不正确的。Conda提供这个包作为一个解决方案,尽管事实上它不是。

从更广泛的意义上说,这是Conda查找兼容包的方式中的一个缺陷。也许它正在按预期工作,所以它不是一个bug。但这是一个缺陷,因为当用户设置numpy=1.15时,Conda给出的正确答案应该是"没有兼容的包"。但是,因为Conda依赖于所贡献包的版本依赖关系,所以它不能这样做。

我在为RedHat或Debian Linux系统打包时没有遇到过同样的问题,它们往往报告"什么都没有";而不是提供不准确的匹配。

最新更新