如何打印scikit learn调度的作业数?

Question

我计划在AWS上运行以下代码

from sklearn.metrics import pairwise_distances
pw_distance = pairwise_distances(model_in, model_out, n_jobs=-1)

如何打印使用n_jobs=-1调度的作业数?我知道我可以使用AWS检查监控CPU使用情况，但我也想要直接读出。

Answer 1

像这样:

from joblib import effective_n_jobs
print(effective_n_jobs(-1))

pairwise_distances在引擎盖下使用Parallel;如果您愿意，可以对其进行测试:

from time import sleep
from joblib import Parallel, delayed
Parallel(n_jobs=-1, verbose=10)(delayed(sleep)(.2) for _ in range(10))

输出:

[Parallel(n_jobs=-1)]: Using backend LokyBackend with 12 concurrent workers.
[Parallel(n_jobs=-1)]: Done   3 out of  10 | elapsed:    0.6s remaining:    1.3s
[Parallel(n_jobs=-1)]: Done   5 out of  10 | elapsed:    0.6s remaining:    0.6s
[Parallel(n_jobs=-1)]: Done   7 out of  10 | elapsed:    0.6s remaining:    0.3s
[Parallel(n_jobs=-1)]: Done  10 out of  10 | elapsed:    0.6s finished