Anaconda中的Python包不同于命令行到jupyterhub(linux)



这里有个奇怪的问题。我正在Ubuntu 20.04上使用Anaconda来设置一些python环境。具体来说,我试图设置tensorflow来使用我的GPU,但这只是一个细节。问题是,当我打开一个终端并执行以下命令(对于名为tf的环境(时,一切都能正常工作:

conda activate tf
python

然后在python提示符下运行

import tensorflow as tf
tf.config.list_physical_devices()

输出很好,这表明我有一个GPU:

[PhysicalDevice(name='/physical_device:CPU:0',device_type='CPU'(,PhysicalDevice(name='/physical_device:GPU:0',device_type='GPU'(]

然后我启动Jupyterlab,我为它添加了一个名为tf的环境,然后我运行相同的命令,它说它找不到一些包,只将CPU显示为物理设备。这就是问题所在。这是输出:

2022-02-16 09:20:03.656417: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:936] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-16 09:20:03.656737: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-02-16 09:20:03.656836: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory
2022-02-16 09:20:03.656927: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublasLt.so.11'; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory
2022-02-16 09:20:03.658936: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory
2022-02-16 09:20:03.659034: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusparse.so.11'; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory
2022-02-16 09:20:03.659198: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1850] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]

如果我使用这种环境打开控制台,也会发生同样的情况。但在jupyter中,如果我打开一个常规的终端和activate tf,那么它可以正常工作。

为什么这里的行为不同?这是我第一次看到这种情况发生。有人知道如何让jupyter的行为与命令行的行为相同吗?

谢谢。

UPDATE我认为问题可能是$PATH变量,但我强制jupyter中的路径与终端会话中显示的路径相同,但仍然会出现相同的错误。

Jupyter笔记本电脑运行的环境具有错误的LD_LIBRARY_PATH,因此它找不到使用GPU所需的CUDA或cudnn库。

最新更新