在1D NumPy数组上创建NaN填充元素的滑动窗口

Question

我有一个时间序列x[0], x[1], ... x[n-1]，存储为1维numpy数组。我想把它转换成下面的矩阵:

NaN,        ... , NaN ,   x[0]
NaN,        ... , x[0],   x[1]
.
.
NaN,  x[0], ... , x[n-3],x[n-2]
x[0], x[1], ... , x[n-2],x[n-1]

我想用这个矩阵来加速时间序列的计算。在numpy或scipy中有这样的功能吗?(我不想在python中使用for循环来完成)

Answer 1

np.lib.stride_tricks.as_strided -

def nanpad_sliding2D(a):
    L = a.size
    a_ext = np.concatenate(( np.full(a.size-1,np.nan) ,a))
    n = a_ext.strides[0]
    strided = np.lib.stride_tricks.as_strided     
    return strided(a_ext, shape=(L,L), strides=(n,n))

示例运行-

In [41]: a
Out[41]: array([48, 82, 96, 34, 93, 25, 51, 26])
In [42]: nanpad_sliding2D(a)
Out[42]: 
array([[ nan,  nan,  nan,  nan,  nan,  nan,  nan,  48.],
       [ nan,  nan,  nan,  nan,  nan,  nan,  48.,  82.],
       [ nan,  nan,  nan,  nan,  nan,  48.,  82.,  96.],
       [ nan,  nan,  nan,  nan,  48.,  82.,  96.,  34.],
       [ nan,  nan,  nan,  48.,  82.,  96.,  34.,  93.],
       [ nan,  nan,  48.,  82.,  96.,  34.,  93.,  25.],
       [ nan,  48.,  82.,  96.,  34.,  93.,  25.,  51.],
       [ 48.,  82.,  96.,  34.,  93.,  25.,  51.,  26.]])

strides的内存效率

正如@Eric在评论中提到的，这种基于跨步的方法将是一种内存效率高的方法，因为输出将只是NaNs-padded 1D版本的视图。让我们测试一下-

In [158]: a   # Sample 1D input
Out[158]: array([37, 95, 87, 10, 35])
In [159]: L = a.size  # Run the posted approach
     ...: a_ext = np.concatenate(( np.full(a.size-1,np.nan) ,a))
     ...: n = a_ext.strides[0]
     ...: strided = np.lib.stride_tricks.as_strided     
     ...: out = strided(a_ext, shape=(L,L), strides=(n,n))
     ...: 
In [160]: np.may_share_memory(a_ext,out) O/p might be a view into extended version
Out[160]: True

让我们通过给a_ext赋值，然后检查out，来确认输出实际上是一个视图。

a_ext和out的初始值:

In [161]: a_ext
Out[161]: array([ nan,  nan,  nan,  nan,  37.,  95.,  87.,  10.,  35.])
In [162]: out
Out[162]: 
array([[ nan,  nan,  nan,  nan,  37.],
       [ nan,  nan,  nan,  37.,  95.],
       [ nan,  nan,  37.,  95.,  87.],
       [ nan,  37.,  95.,  87.,  10.],
       [ 37.,  95.,  87.,  10.,  35.]])

修改a_ext:

In [163]: a_ext[:] = 100

查看新的out:

In [164]: out
Out[164]: 
array([[ 100.,  100.,  100.,  100.,  100.],
       [ 100.,  100.,  100.,  100.,  100.],
       [ 100.,  100.,  100.,  100.,  100.],
       [ 100.,  100.,  100.,  100.,  100.],
       [ 100.,  100.,  100.,  100.,  100.]])

确认为视图。

最后，让我们测试一下内存需求:

In [131]: a_ext.nbytes
Out[131]: 72
In [132]: out.nbytes
Out[132]: 200

因此，尽管输出显示为200字节，但实际上只是72字节，因为它是扩展数组的视图，其大小为72字节。

---

Scipy's toeplitz -

的另一个方法

from scipy.linalg import toeplitz
out = toeplitz(a, np.full(a.size,np.nan) )[:,::-1]