无法保留内存块,在python中导入json错误


import pandas as pd
with open(r'data.json') as f:
df = pd.read_json(f, encoding='utf-8')

我得到了一个";无法保留内存块";错误JSON文件的大小为300MB。为Python中运行的程序保留内存有限制吗?我在电脑上有8GB的RAM,使用Windows 10。

loading of json file into df
Traceback (most recent call last):
File "C:Program FilesJetBrainsPyCharm 2018.1.4helperspydevpydev_run_in_console.py", line 52, in run_file
pydev_imports.execfile(file, globals, locals)  # execute the script
File "C:Program FilesJetBrainsPyCharm 2018.1.4helperspydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"n", file, 'exec'), glob, loc)
File "C:/Users/Beorn/PycharmProjects/project_0/projekt/test.py", line 7, in <module>
df = pd.read_json(f, encoding='utf-8')
File "C:UsersBeornAppDataLocalProgramsPythonPython36-32libsite-packagespandasiojsonjson.py", line 422, in read_json
result = json_reader.read()
File "C:UsersBeornAppDataLocalProgramsPythonPython36-32libsite-packagespandasiojsonjson.py", line 529, in read
obj = self._get_object_parser(self.data)
File "C:UsersBeornAppDataLocalProgramsPythonPython36-32libsite-packagespandasiojsonjson.py", line 546, in _get_object_parser
obj = FrameParser(json, **kwargs).parse()
File "C:UsersBeornAppDataLocalProgramsPythonPython36-32libsite-packagespandasiojsonjson.py", line 638, in parse
self._parse_no_numpy()
File "C:UsersBeornAppDataLocalProgramsPythonPython36-32libsite-packagespandasiojsonjson.py", line 853, in _parse_no_numpy
loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Could not reserve memory block
PyDev console: starting.
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 02:47:15) [MSC v.1900 32 bit (Intel)] on win32

因此,在阅读了大量的文章和解决方案后,我决定通过清除无用的数据来减少文件大小。也许你会发现这很有用。顺便说一句,我在某个地方读到,你需要的内存至少是JSON文件的25倍,所以在我的情况下,我需要超过8GB。

with open('data.json', 'r') as data_file:
data = json.load(data_file)
print(data.keys())
del data['author']
with open('datav2.json', 'w') as data_file:
data = json.dump(data, data_file)

最新更新