如何使用多处理将CSV文件转换为GZIP文件.Python中的池

Question

我正在使用gzip将csv文件转换为gunzip文件。我正在尝试实现多处理，以加快处理速度。但当我运行代码时，它甚至不会给我输出，它会继续运行，也不会出错。下面是我的代码

from multiprocessing import Pool
import gzip
path= r'D:All Files'
def convertToGzip(file):
file_ = os.path.join(path, file)
print('File {} is being Converted'.format(file_))
with open(os.path.join(path, file_), 'rb') as f_in:
with gzip.open(file+'.gz', 'wb') as f_out:
shutil.copyfileobj(f_in, f_out)

a_pool = Pool(processes=4)
result = a_pool.map(convertToGzip, os.listdir(path))

我不明白这里的问题在哪里，因为我对多处理器了解不多

Answer 1

你可以试试这个：

import gzip
import os
import shutil
from multiprocessing import Pool
PATH = "D:All Files"

def convert_to_gzip(fname):
fpath = os.path.join(PATH, fname)
print("File {} is being Converted".format(fpath))
with open(fpath, "rb") as f_in:
with gzip.open(fpath + ".gz", "wb") as f_out:
shutil.copyfileobj(f_in, f_out)

def main():
with Pool() as pool:
result = pool.map(convert_to_gzip, os.listdir(PATH))

if __name__ == "__main__":
main()