Scattercv位移大于int最大值

Question

我正在尝试将2D矩阵分布在具有一些重叠区域的多个处理器上。我注意到mpi4py的函数scartv不适用于2d数组，当时我被迫使用1D数组。这里的问题是，一些处理器中的位移可能大于int的极限。请问我该如何解决这个问题？

import numpy as np
from mpi4py import MPI
from math import ceil
# Dimensions of the matrix
N = 50000
M = 50000
comm = MPI.COMM_WORLD
size = comm.Get_size()
rank = comm.Get_rank()
n = ceil(N/size)
offset = N-(size-1)*n  # Offset
start = np.zeros(size, dtype=np.float64)  # Start rows of the chunks
end = np.zeros(size, dtype=np.float64)  # End rows of the chunks
sz_loc = np.zeros(size, dtype=np.float64)  # Size of he local chunk
for i in range(size):
pp = n 
if i == size - 1:
pp = offset
start[i] = max(0, i * n - mf)
end[i] = min(N, i * n + pp + mf)
sz_loc[i] = (end[i]-start[i])*M
if rank == 0:
Im = np.array(np.round(10*np.random.rand(N, M)),
dtype=np.float32).ravel()  
else: 
Im = None 
Im_loc = np.zeros((int(sz_loc[rank]),), dtype=np.float32)
comm.Barrier()
comm.Scatterv([Im, sz_loc, start*M, MPI.FLOAT], Im_loc, root=0)

这是我得到的错误，例如使用10个处理器

comm.Scatterv([Im, sz_loc, start*M, MPI.FLOAT], Im_loc, root=0)
File "mpi4py/MPI/Comm.pyx", line 626, in mpi4py.MPI.Comm.Scatterv
File "mpi4py/MPI/msgbuffer.pxi", line 538, in mpi4py.MPI._p_msg_cco.for_scatter
File "mpi4py/MPI/msgbuffer.pxi", line 440, in mpi4py.MPI._p_msg_cco.for_cco_send
File "mpi4py/MPI/msgbuffer.pxi", line 313, in mpi4py.MPI.message_vector
File "mpi4py/MPI/asarray.pxi", line 22, in mpi4py.MPI.chkarray
File "mpi4py/MPI/asarray.pxi", line 15, in mpi4py.MPI.getarray
OverflowError: value too large to convert to int

Answer 1

最后，我使用Create_contiguous((函数定义了一个新的数据类型。