python多处理和mmap

Question

我的目标是生成在后台运行的新进程，并将mmap对象作为参数传递给它。玩具示例：

from multiprocessing import Process
import mmap
import time 

def modify(buf):
time.sleep(10)
buf[0:100] = b"xy" * 50

def main_job(t):
time.sleep(t)


if __name__ == "__main__":
BUF = mmap.mmap(-1, length=100, access=mmap.ACCESS_WRITE)
BUF[0:100] = b"a" * 100
p = Process(target=modify, args=(BUF,))
p.daemon=True
p.start() #background job started
main_job(100) #man job started in parallel
print(BUF[0:100])

TypeError：无法pickle'mmap.mmap'对象

这里我有主作业，它只是睡眠，我想用后台作业生成进程，并将mmap-obj作为参数传递给子进程。问题是mmap对象是不可拾取的，并且无法将mmap和多处理一起使用。但我不知道如何创建后台作业，例如通过os.fork()传递函数和参数。如何解决这个问题？

Answer 1

mmap.mmap在Windows或Posix版本中，需要一个有效的文件描述符编号作为其第一个参数。-1不是有效的文件描述符编号
EDIT：-1是一个正确的fd值，用于指示匿名mmap

我想你不明白mmap到底是干什么的。您不将mmap对象提供给另一个进程，而是主进程和子进程都必须创建自己的mmap，但都由同一个文件支持。

以下是您的代码和两个修复(和减少睡眠时间(：

from multiprocessing import Process
import mmap
import time

def modify(filename):
time.sleep(5)
with open(filename, "r+b") as file, 
mmap.mmap(file.fileno(), length=100, access=mmap.ACCESS_WRITE) as buf:
buf[0:100] = b"xy" * 50

def main_job(t):
time.sleep(t)

if __name__ == "__main__":
filename = "shared_mem_file"
with open(filename, "r+b") as file, 
mmap.mmap(file.fileno(), length=100, access=mmap.ACCESS_WRITE) as BUF:
BUF[0:100] = b"a" * 100
BUF.flush()
p = Process(target=modify, args=(filename,))
p.daemon = True
p.start()  # background job started
main_job(10)  # man job started in parallel
print(BUF[0:100])
BUF.close()

输出为：

b'xyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxy'