当我使用星图时，我无法将多个参数传递给函数

Question

我正在做多次实验，但每次都要花很多时间，所以我尝试使用包multiprocessing，但我有一个错误。

函数定义如下：

def run_exp_anxia_sim(num_exp,num1,num2, 
eps, isfuzzy, remove, using_train, iscompress):
if using_train == False:
results,x , y,dic1,dic2 = classificator_pos_neg(all_pos, all_neg, test_anxia,
num_feat1=num1,num_feat2=num2,
tau=eps,fuzzy=isfuzzy,remove_stop=remove,
train_data =using_train,compress = iscompress)
result_name = 'result_anxia_key' + str(num_exp) + '.txt'
path_name =  '/content/sample_data'+ '/' + result_name

with open(path_name, "w") as f:
f.write("Experimento de anorexia número: " + str(num_exp) + 'n')
f.write("Confusion matrix: n")
f.write(str(confusion_matrix(test_labels_anxia, results)) + 'n')
f.write('Metrics classification n')
f.write(str(metrics.classification_report(test_labels_anxia, results)) + 'n')
f.close()
return f1_score(test_labels_anxia, results)

if using_train == True:
seed_val = 42
np.random.seed(seed_val)
parameters = {'C': [.05, .12, .25, .5, 1, 2, 4]}

results, x, y,z,dic1,dic2 = classificator_pos_neg(all_pos, all_neg, test_anxia,num1,num2, 
tau=eps,fuzzy = isfuzzy,remove_stop=remove, train_data =using_train,compress = iscompress)
svr = svm.LinearSVC(class_weight='balanced', dual=False)
grid_anorexia = GridSearchCV(estimator=svr, param_grid=parameters, n_jobs=8, scoring='f1_macro', cv=5)
grid_anorexia.fit(z, tr_labels)
y_pred = grid_anorexia.predict(results)
a1 = grid_anorexia.best_params_
p, r, f, _ = precision_recall_fscore_support(test_labels_anxia, y_pred, average='macro', pos_label=1)
result_name = 'result_anxia_key' + str(num_exp) + '.txt'
path_name = '/content/sample_data' + '/' + result_name


with open(path_name, "w") as f:
f.write("Experimento de anorexia número: " + str(num_exp) + 'n')
f.write("Confusion matrix: n")
f.write(str(confusion_matrix(test_labels_anxia, y_pred)) + 'n')
f.write('Metrics classification n')
f.write(str(metrics.classification_report(test_labels_anxia, y_pred)) + 'n')
f.write('Best parameter:n')
f.write(str(a1))
f.write('n')
f.close()

return f1_score(test_labels_anxia, y_pred), a1

论点是：

arg1 = [1,2]
arg6 = [1000,1500]
arg7 = [1000,1500]
arg8 = [0.99]*2
arg10 = [True]*2
arg11 = [False]*2
arg12 = [False]*2
arg13 = [False]*2

但是当我运行时

with multiprocessing.Pool(processes=multiprocessing.cpu_count()) as pool:
x,y,z,w,t = pool.apply(run_exp_anxia_sim, zip(arg1,arg6,arg7,arg8,arg10,arg11,arg12,arg13))
print(x,y,z,w,t)

上面写着

TypeError: run_exp_anxia_sim() missing 6 required positional arguments: 'num2', 'eps', 'isfuzzy', 'remove', 'using_train', and 'iscompress'

当我对CCD_ 2进行同样的尝试时，我会得到一个关于未打包值的错误。

我能做什么？

Answer 1

您正在混合两种不同的东西。

首先，pool.apply的有用性是有限的，因为它会阻塞直到结果准备好，这在使用多处理时可能不是您想要的。

其次，当使用pool.apply时，您提供的参数会直接解压缩到目标函数。因此，因为zipping创建了一个元组的可迭代项，所以您将把整个可迭代项作为第一个参数传递，这就是为什么会出现错误，因为函数只有一个参数。

以下是如何在这种情况下使用pool.apply：

from multiprocessing import Pool

def run_exp_anxia_sim(num_exp,num1,num2,
eps, isfuzzy, remove, using_train, iscompress):
print(locals())
if __name__ == "__main__":
arg1 = [1, 2]
arg6 = [1000, 1500]
arg7 = [1000, 1500]
arg8 = [0.99] * 2
arg10 = [True] * 2
arg11 = [False] * 2
arg12 = [False] * 2
arg13 = [False] * 2
all_args = zip(arg1, arg6, arg7, arg8, arg10, arg11, arg12, arg13)
with Pool(4) as pool:
results = [pool.apply(run_exp_anxia_sim, args=args) for args in all_args]
print(results)

以下是如何使用pool.starmap(在这里更有意义(：

all_args = zip(arg1, arg6, arg7, arg8, arg10, arg11, arg12, arg13)
with Pool(4) as pool:
results = pool.starmap(run_exp_anxia_sim, all_args)
print(results)