将Python列表重塑为numpy数组

Question

我有一个用例，我需要从一个训练好的LSTM模型(TensorFlow = 2.8和Python = 3.9)中进行预测，其中数据集很大，导致GPU卡上的OOM错误。我的周转是预测每个批处理并将结果张量存储在Python3列表中:

# Create TF dataset to iterate over-
train_dataset = tf.data.Dataset.from_tensor_slices(X_train).batch(batch_size)
# batch_size = 256
# Python3 list to contain all model predictions-
preditcions = []
for x in train_dataset:
y_pred = model_e2d2.predict(x)
preditcions.append(y_pred)
# Sanity check-
len(preditcions)
# (2048)
# Each batch of prediction has the size: (256, 5, 11).
# Sanity check-
preditcions[0].shape, preditcions[-1].shape
# ((256, 5, 11), (222, 5, 11))

目的是将精度Python3列表转换为所需形状的numpy数组:(524254,5,11)

?

Answer 1

一般情况下可以这样做:

predictions = np.concatenate(predictions, axis=0)

这使您可以控制要连接的轴。

看这里给出的例子https://numpy.org/doc/stable/reference/generated/numpy.concatenate.html#numpy-concatenate

>>a = np.array([[1, 2], [3, 4]])
>>b = np.array([[5, 6]])
>>np.concatenate([a, b], axis=0) #slight difference from example.
# instead of tuple I have passed a list as is OP's requirement
array([[1, 2],
[3, 4],
[5, 6]])

感谢@hpaulj纠正我的错误。