conturrent.futures.threadpoolexecutor比列表理解要慢

Question

我正在使用列表理解vs consturrent.futures来测试一个琐碎的函数：

class Test:
    @staticmethod
    def something(times = 1):
        return sum([1 for i in range(times)])
    @staticmethod
    def simulate1(function, N):
        l = []
        for i in range(N):
            outcome = function()
            l.append(outcome)
        return sum(l) / N
    @staticmethod
    def simulate2(function, N):
        import concurrent.futures
        with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
            l = [outcome for outcome in executor.map(lambda x: function(), range(N))]
        return sum(l) / N
    @staticmethod
    def simulate3(function, N):
        import concurrent.futures
        l = 0
        with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
            futures = [executor.submit(function) for i in range(N)]
            for future in concurrent.futures.as_completed(futures):
                l += future.result()
        return l / N
def simulation():
    simulationRate = 100000
    import datetime
    s = datetime.datetime.now()
    o = Test.simulate1(lambda : Test.something(10), simulationRate)
    print((datetime.datetime.now() - s))
    s = datetime.datetime.now()
    o = Test.simulate2(lambda : Test.something(10), simulationRate)
    print((datetime.datetime.now() - s))
    s = datetime.datetime.now()
    o = Test.simulate3(lambda : Test.something(10), simulationRate)
    print((datetime.datetime.now() - s))
simulation()

测量时间，我得到了：

0:00:00.258000
0:00:10.348000
0:00:10.556000

我正在开始并发，所以我不明白什么是阻止线程更快地运行的瓶颈。

Answer 1

如果将任务功能更改为此，则会看到差异：

def something(n):
    """ simulate doing some io based task.
    """
    time.sleep(0.001)
    return sum(1 for i in range(n))

在我的Mac Pro上，这给出了：

0:00:13.774700
0:00:01.591226
0:00:01.489159

conturrent.future这次更加明显。

原因是：您正在模拟基于CPU的任务，因为Python的GIL，同意。

consturrent.future为异步执行可呼叫提供了一个高级接口，您将其用于错误的场景。