训练我的空间模型可以"Could not load dynamic library 'libcudart.so.11.0'",但无论如何都会继续



我的问题其实很快。我在一个venv中使用Spacy,遵循在线教程。我刚刚训练了我的模型,虽然训练本身结束了,但它也返回了以下错误:

(venv) [annemarie@annemarie-linux TrainingData]$ python3.8 -m spacy train config/config.cfg --paths.train corpus/spacy-docbins/train.spacy --paths.dev corpus/spacy-docbins/test.spacy --output output/models --training.eval_frequency 10 --training.max_steps 300
2022-07-28 08:35:25.488466: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-07-28 08:35:25.488481: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-07-28 08:35:26.409531: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:939] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-07-28 08:35:26.409662: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409694: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409721: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublasLt.so.11'; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409747: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcufft.so.10'; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409773: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcurand.so.10'; dlerror: libcurand.so.10: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409799: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409824: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusparse.so.11'; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409850: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2022-07-28 08:35:26.409857: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1850] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
ℹ Saving to output directory: output/models
ℹ Using CPU
=========================== Initializing pipeline ===========================
[2022-07-28 08:35:26,788] [INFO] Set up nlp object from config
[2022-07-28 08:35:26,794] [INFO] Pipeline: ['ner']
[2022-07-28 08:35:26,797] [INFO] Created vocabulary
[2022-07-28 08:35:26,797] [INFO] Finished initializing nlp object
[2022-07-28 08:35:26,845] [INFO] Initialized pipeline components: ['ner']
✔ Initialized pipeline
============================= Training pipeline =============================
ℹ Pipeline: ['ner']
ℹ Initial learn rate: 0.001
E    #       LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE 
---  ------  --------  ------  ------  ------  ------
0       0     68.44    0.00    0.00    0.00    0.00
3      10    615.49    0.00    0.00    0.00    0.00
6      20    295.06   28.57   57.14   19.05    0.29
10      30    210.32   46.67   77.78   33.33    0.47
13      40    154.60   55.00   57.89   52.38    0.55
16      50     56.73   80.00   84.21   76.19    0.80
20      60     20.64   92.68   95.00   90.48    0.93
23      70      2.85   92.68   95.00   90.48    0.93
26      80      0.01   92.68   95.00   90.48    0.93
30      90      0.00   92.68   95.00   90.48    0.93
33     100      0.00   87.80   90.00   85.71    0.88
37     110      0.00   92.68   95.00   90.48    0.93
41     120      0.00   92.68   95.00   90.48    0.93
45     130      0.00   92.68   95.00   90.48    0.93
50     140      0.00   92.68   95.00   90.48    0.93
55     150      0.00   92.68   95.00   90.48    0.93
60     160      0.00   92.68   95.00   90.48    0.93
65     170      0.00   92.68   95.00   90.48    0.93
70     180      0.00   92.68   95.00   90.48    0.93
75     190      0.00   92.68   95.00   90.48    0.93
80     200      0.00   92.68   95.00   90.48    0.93
85     210      0.00   92.68   95.00   90.48    0.93
90     220      0.00   92.68   95.00   90.48    0.93
95     230      0.00   87.80   90.00   85.71    0.88
100     240      0.00   87.80   90.00   85.71    0.88
105     250      0.00   87.80   90.00   85.71    0.88
110     260      0.00   87.80   90.00   85.71    0.88
115     270      0.00   92.68   95.00   90.48    0.93
120     280      0.00   92.68   95.00   90.48    0.93
125     290      0.00   92.68   95.00   90.48    0.93
130     300      0.00   92.68   95.00   90.48    0.93
✔ Saved pipeline to output directory
output/models/model-last

现在,我知道我可能只需要安装"libcudart.so.11.0"。然而,无论是教程还是我调查的任何其他关于Spacy的文档都没有提到我需要单独安装它,所以我很困惑,也许我做错了什么,或者安装了有错的东西。

我用pip安装了spacy,我用的是Python3.8,我在Manjaro(Arch(上。我应该安装libcudart吗?我从来没有使用过python,这是我第一次使用spacy,所以如果这个问题很奇怪,我很抱歉。


EDIT:这让它更令人困惑,测试模型显示它的功能与我训练的一样。然而,它会对spacy.load("pathtomodel")发出与上面相同的警告。

这些警告与tensorflow有关,而这些管道没有使用spacyspacy train

如果spacy可用,它会自动导入tensorflow,这就是您看到警告的原因,但它们不应该影响典型的spacy使用。

如果您在未安装tensorflow的venv中工作,则警告应该会消失。

相关内容

最新更新