获取类型错误：无法腌制_thread.lock 对象

Question

我正在查询MongoDB以获取字典列表，对于列表中的每个字典，我正在对值进行一些比较。根据比较结果，我将字典、比较结果和其他值存储在 mongoDB 集合中计算的值。我正在尝试通过调用多处理来执行此操作，但收到此错误。

def save_for_doc(doc_id):
#function to get the fields of doc
fields = get_fields(doc_id)
no_of_process = 5
doc_col_size = 30000
chunk_size = round(doc_col_size/no_of_process)
chunk_ranges = range(0, no_of_process*chunk_size, chunk_size)
processes = [ multiprocessing.Process(target=save_similar_docs, args= 
(doc_id,client,fields,chunks,chunk_size)) for chunks in chunk_ranges]
for prc in processes:
prc.start()
def save_similar_docs(arguments):
#This function process the args and saves the results to MongoDB. Does not 
#return anything as the end result is directly stored.

下面是错误：

File "H:/Desktop/Performance Improvement/With_Process_Pool.py", line 144, 
in save_for_doc
prc.start()
File "C:ProgramDataAnaconda3libmultiprocessingprocess.py", line 105, 
in start
self._popen = self._Popen(self)
File "C:ProgramDataAnaconda3libmultiprocessingcontext.py", line 223, 
in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:ProgramDataAnaconda3libmultiprocessingcontext.py", line 322, 
in _Popen
return Popen(process_obj)
File "C:ProgramDataAnaconda3libmultiprocessingpopen_spawn_win32.py", 
line 65, in __init__
reduction.dump(process_obj, to_child)
File "C:ProgramDataAnaconda3libmultiprocessingreduction.py", line 60, 
in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects

此错误是什么意思？请解释一下，我该如何克服。

Answer 1

文档说你不能将客户端从主进程复制到子进程，你必须在分叉后创建连接。客户端对象无法复制，请在分叉进程后创建连接。

在 Unix 系统上，多处理模块使用 fork(( 生成进程。将 MongoClient 实例与 fork(( 一起使用时必须小心。具体来说，MongoClient的实例不得从父进程复制到子进程。相反，父进程和每个子进程必须创建自己的MongoClient实例。

http://api.mongodb.com/python/current/faq.html#id21