无法在spaCy中加载德国BERT模型



这是我的问题:我正在从事德语文本分类项目。我为此使用空间,并决定微调其预训练的BERT模型以获得更好的结果。但是,当我尝试将其加载到代码中时,它向我显示错误。

这是我所做的:

  1. 已安装的空间变压器:pip install spacy-transformers
  2. 下载德国BERT模型:python -m spacy download de_trf_bertbasecased_lg.它已成功下载并向我显示:✔ Download and installation successful You can now load the model via spacy.load('de_trf_bertbasecased_lg')
  3. 编写了以下代码:

import spacy nlp = spacy.load('de_trf_bertbasecased_lg')

输出是:

Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
nlp = spacy.load('de_trf_bertbasecased_lg')
File "C:PythonPython37libsite-packagesspacy__init__.py", line 30, in load
return util.load_model(name, **overrides)
File "C:PythonPython37libsite-packagesspacyutil.py", line 164, in load_model
return load_model_from_package(name, **overrides)
File "C:PythonPython37libsite-packagesspacyutil.py", line 185, in load_model_from_package
return cls.load(**overrides)
File "C:PythonPython37libsite-packagesde_trf_bertbasecased_lg__init__.py", line 12, in load
return load_model_from_init_py(__file__, **overrides)
File "C:PythonPython37libsite-packagesspacyutil.py", line 228, in load_model_from_init_py
return load_model_from_path(data_path, meta, **overrides)
File "C:PythonPython37libsite-packagesspacyutil.py", line 196, in load_model_from_path
cls = get_lang_class(lang)
File "C:PythonPython37libsite-packagesspacyutil.py", line 70, in get_lang_class
if lang in registry.languages:
File "C:PythonPython37libsite-packagescatalogue.py", line 56, in __contains__
has_entry_point = self.entry_points and self.get_entry_point(name)
File "C:PythonPython37libsite-packagescatalogue.py", line 140, in get_entry_point
return entry_point.load()
File "C:PythonPython37libsite-packagesimportlib_metadata__init__.py", line 94, in load
module = import_module(match.group('module'))
File "C:PythonPython37libimportlib__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:PythonPython37libsite-packagesspacy_transformers__init__.py", line 1, in <module>
from .language import TransformersLanguage
File "C:PythonPython37libsite-packagesspacy_transformerslanguage.py", line 5, in <module>
from .util import is_special_token, pkg_meta, ATTRS, PIPES, LANG_FACTORY
File "C:PythonPython37libsite-packagesspacy_transformersutil.py", line 2, in <module>
import transformers
File "C:PythonPython37libsite-packagestransformers__init__.py", line 20, in <module>
from .file_utils import (TRANSFORMERS_CACHE, PYTORCH_TRANSFORMERS_CACHE, PYTORCH_PRETRAINED_BERT_CACHE,
File "C:PythonPython37libsite-packagestransformersfile_utils.py", line 37, in <module>
import torch
File "C:PythonPython37libsite-packagestorch__init__.py", line 81, in <module>
ctypes.CDLL(dll)
File "C:PythonPython37libctypes__init__.py", line 356, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

如果我在 PyCharm 中运行相同的代码,它还会在上述所有代码之前显示这两行:

2020-05-19 18:00:55.132721: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not 
load dynamic library 'cudart64_100.dll'; dlerror: cudart64_100.dll not found
2020-05-19 18:00:55.132990: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart 
dlerror if you do not have a GPU set up on your machine.

如果我做对了,这两行抱怨我没有GPU。但是,根据文档,即使没有GPU,我也应该能够使用BERT。

所以我现在真的很想寻求你的帮助。

我还应该提到,我以前使用过de_core_news_sm模型,它工作得很好。

我也已经尝试了几种解决方案,但没有一种奏效。我试过了: 这个和这个。我还尝试卸载所有与spacy相关的库并重新安装它们。也无济于事。

我正在与:

视窗 10 家庭

版蟒蛇:3.7.2

空间:2.2.4

空间变压器:0.5.1

将不胜感激任何帮助或建议!

这可能是您安装torch的问题。从干净的虚拟环境中开始,然后按照此处的说明安装torch,CUDA 为无:https://pytorch.org/get-started/locally/。然后使用pip安装spacy-transformers

尝试:

python -m spacy download de_trf_bertbasecased_lg
python -m spacy download de_trf_bertbasecased_lg-2.2.0
python -m spacy link de_trf_bertbasecased_lg
python -m spacy link de_trf_bertbasecased_lg-2.2.0

或:

python -m spacy download de
python -m spacy.de.download all

或直接从以下位置下载:

https://deepset.ai/german-bert

然后加载:

nlp = spacy.load(r'Path_To_Filede_trf_bertbasecased_lg-2.2.0')

最新更新