Python Multiprocessing - ApplyResult.get() NameError： 未定义全局名称'self'

Question

我目前正在尝试使用 Python 多处理包来使 CPU 密集型进程运行得更快。我有一个非常大的 numpy 矩阵，并希望使用 Pool 和 apply_async 来拆分工作来计算矩阵中的值。但是，当我对函数运行单元测试以测试它是否有效时，出现错误"NameError：未定义全局名称'self'"。我在Google或StackOverflow上也找不到任何有帮助的东西。知道为什么会发生这种情况吗？

Pytest 输出：

_____________________ TestBuildEMMatrix.test_build_em_matrix_simple _____________________
self = <mixemt_master.mixemt2.preprocess_test.TestBuildEMMatrix testMethod=test_build_em_matrix_simple>
    def test_build_em_matrix_simple(self):
            reads = ["1:A,2:C", "1:T,2:C", "3:T,4:T", "2:A,4:T"]
            in_mat = preprocess.build_em_matrix(self.ref, self.phy,
>                                                                                   reads, self.haps, self.args)
preprocess_test.py:272:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
preprocess.py:239: in build_em_matrix
    results[i] = results[i].get()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <multiprocessing.pool.ApplyResult object at 0x7f4218ea07d0>, timeout = None
    def get(self, timeout=None):
        self.wait(timeout)
        if not self._ready:
            raise TimeoutError
        if self._success:
            return self._value
        else:
>           raise self._value
E           NameError: global name 'self' is not defined
/vol/hpc/apps/python-anaconda2-4.3.1-abat/install/lib/python2.7/multiprocessing/pool.py:567: NameError
--------------------------------- Captured stdout call ----------------------------------
False

以及相关的 Python 函数：

def build_em_matrix_process(markers, haplogroups, pos_obs, mut_prob, column_length, start_index, end_index):
    columns = [[prob_for_vars(markers, haplogroups[j], pos_obs, mut_prob) for j in xrange(column_length)]
        for i in xrange(start_index, end_index)]
    return columns
def build_em_matrix(refseq, phylo, reads, haplogroups, args):   
    """
    Returns the matrix that describes the probabiliy of each read
    originating in each haplotype.
    """
    hvb_mat = HapVarBaseMatrix(refseq, phylo)
    read_hap_mat = numpy.empty((len(reads), len(haplogroups)))
    if args.verbose:
        sys.stderr.write('Building EM input matrix...n')
    num_processors = args.p
    pool = Pool(processes = num_processors);
    results = []
    partition_size = int(math.ceil(len(reads) / float(num_processors)))
    for i in xrange(num_processors):
        start_index = i * partition_size
        end_index = (i + 1) * partition_size
        pos_obs = pos_obs_from_sig(reads[i])
        results.append(pool.apply_async(build_em_matrix_process, (hvb_mat.markers, haplogroups, pos_obs, hvb_mat.mut_prob, len(haplogroups), start_index, end_index)))
    column = 0
    for i in xrange(num_processors):
        results[i].wait()
        print results[i].successful()
        results[i] = results[i].get()
        for j in xrange[len(results)]:
            read_hap_mat[column] = results[i][j]
            column += 1
    if args.verbose:
        sys.stderr.write('Done.nn')
    return read_hap_mat

在调用 'results[i].wait((] 之后，添加了一条语句 'print results[I].success(('，它将 False 打印到 stdout。我不确定为什么这没有返回 true，因为我在build_em_matrix_process中找不到任何错误。

Answer 1

我又仔细研究了一下代码，找到了答案！

为了完成此操作，我将build_em_matrix_process调用的类的实例方法重构为顶级方法。事实证明，我不小心在方法的主体中留下了对自我的引用。当我运行测试时，错误似乎来自 ApplyResult.get(( 本身的代码，而不是正在调用的顶级方法中的代码。