Numpy种子有时不适用于dask功能

Question

在跨 dask 分布式函数调用运行时，种子设定有时会失败。希望将种子值传递给一组大多数时间都有效的 MC 模拟试验;但是，并非总是如此。该问题归结为以下示例：

from dask.distributed import Client
import numpy as np
def get_rand4seed(seedx):
np.random.seed(seedx)
rand1 = np.random.rand(1)[0]
return seedx, rand1
seedrange = 100
seed_ids = np.arange(0,seedrange).tolist()
client = Client()
a = client.map(get_rand4seed, seed_ids)
results = client.gather(a)
client.close()
for result in results:
# take seed packed in result and calculate correct 1st random number
np.random.seed(result[0])
correct_result = np.random.rand(1)[0]
# comparing with 1st random number calculated in parallelized func
comparison = 'seed=%s, dask=%s, correct=%s' % (result[0], result[1], correct_result)
if result[1] != correct_result:
print('DIFF: %s' % comparison)
else:
pass
#print(comparison)

通常，5% 到 10% 的案例不正确，在第 10 项左右之后，错误的可能性似乎更大。而且，有时所有 100 个项目都是正确的。示例结果：

DIFF: seed=10, dask=0.6503742417395917, correct=0.771320643266746
DIFF: seed=18, dask=0.5054533737348429, correct=0.6503742417395917
DIFF: seed=26, dask=0.038561680881409655, correct=0.30793495262497084
DIFF: seed=34, dask=0.780100460524675, correct=0.038561680881409655
DIFF: seed=69, dask=0.6063543377764754, correct=0.29624916167243354
DIFF: seed=77, dask=0.29624916167243354, correct=0.9191090317991818
DIFF: seed=85, dask=0.6575115686178157, correct=0.620373814553256
DIFF: seed=93, dask=0.3072410093435699, correct=0.6063543377764754

Answer 1

Python 3.6.9， dask 2.9.0

我无法运行您的代码...抱怨不使用if __name__ == '__main__':然后它给了我这个

NameError: name 'results' is not defined
distributed.nanny - WARNING - Restarting worker

于是，我看了看爸爸。像这样打包并重写您的代码

import dask.bag as db
import numpy as np

def get_rand4seed(seedx):
np.random.seed(seedx)
rand1 = np.random.rand(1)[0]
return seedx, rand1

seedrange = 100
b = db.from_sequence(np.arange(seedrange), npartitions=4)
results = b.map(get_rand4seed).compute()
for result in results:
np.random.seed(result[0])
correct_result = np.random.rand(1)[0]
comparison = 'seed=%s, dask=%s, correct=%s' % (
result[0], result[1], correct_result)
if result[1] != correct_result:
print('DIFF: %s' % comparison)
else:
pass

代码执行完美，不打印任何内容，我想这意味着一切都很顺利。