CuDNN 在二进制安装的 cupy 中不可用

Question

我使用二进制包cupy-cuda92安装了Cupy，但是CuDNN没有加载。

$ pip install cupy-cuda92==5.4.0 chainer==5.4.0
$ python -c 'import chainer; chainer.print_runtime_info()'
/home/xxxxx/.pyenv/versions/3.5.2/lib/python3.5/site-packages/chainer/backends/cuda.py:98: UserWarning: cuDNN is not enabled.
Please reinstall CuPy after you install cudnn
(see https://docs-cupy.chainer.org/en/stable/install.html#install-cudnn).
  'cuDNN is not enabled.n'
Platform: Linux-4.4.0-103-generic-x86_64-with-debian-stretch-sid
Chainer: 5.3.0
NumPy: 1.16.2
CuPy:
  CuPy Version          : 5.3.0
  CUDA Root             : /usr/local/cuda
  CUDA Build Version    : 9020
  CUDA Driver Version   : 9020
  CUDA Runtime Version  : 9020
  cuDNN Build Version   : None
  cuDNN Version         : None
  NCCL Build Version    : 2307
  NCCL Runtime Version  : 2402
iDeep: Not Available

我尝试卸载并重新安装chainer和cupy --no-cache-dir但没有发现任何区别。我还确认 CUDA 是 9.2。

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Wed_Apr_11_23:16:29_CDT_2018
Cuda compilation tools, release 9.2, V9.2.88

Answer 1

我自己找到了问题和解决方案。

首先，我尝试直接导入 cudnn。

$ python
Python 3.5.2 (default, Mar 25 2019, 10:54:56)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cupy.cuda.cudnn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/xxxxx/.pyenv/versions/3.5.2/lib/python3.5/site-packages/cupy/cuda/cudnn.cpython-35m-x86_64-linux-gnu.so: symbol cudnnGetBatchNormalizationTrainingExReserveSpaceSize, version libcudnn.so.7 not defined in file libcudnn.so.7 with link time reference

因此存在一种库版本不匹配。我检查了我的LD_LIBRARY_PATH，发现通过cudnnenv安装了不同版本的 CuDNN。我将它们从LD_LIBRARY_PATH中删除，问题就消失了。

$ python -c 'import chainer; chainer.print_runtime_info()'
Platform: Linux-4.4.0-103-generic-x86_64-with-debian-stretch-sid
Chainer: 5.3.0
NumPy: 1.16.2
CuPy:
  CuPy Version          : 5.3.0
  CUDA Root             : /usr/local/cuda
  CUDA Build Version    : 9020
  CUDA Driver Version   : 9020
  CUDA Runtime Version  : 9020
  cuDNN Build Version   : 7402
  cuDNN Version         : 7500
  NCCL Build Version    : 2307
  NCCL Runtime Version  : 2402
iDeep: Not Available