如何遍历数据集并训练模型

Question

我想构建一个模型，其中我想迭代第一个80000个值并训练模型，然后训练下一个80000点，依此类推。这样做是可能的，也是允许的吗?

length=80000
train_data=[]
train_tar=[]
for i in range (0, len(X_train), length):
train_data[i]=X_train.iloc[i:i+length, :]
train_tar[i]=Y_train.iloc[i:i+length, :]


X_training, X_val, Y_training, Y_val = train_test_split(train_data[i], train_tar[i], test_size=0.40,   shuffle=False )

scaler1= StandardScaler()
X_training =scaler1.fit_transform(X_training[i])
X_val[i]=scaler1.transform(X_val[i])
X_test[i]=scaler1.transform(X_test[i])
scaler2= StandardScaler()

Y_training[i] =scaler2.fit_transform(Y_train[i])
Y_val[i]=scaler2.transform(Y_val[i])
Y_test[i]=scaler2.transform(Y_test[i])

train_gen[i] = tf.keras.utils.timeseries_dataset_from_array(  X_training[i], Y_training[i],  sequence_length=160, sequence_stride=1, batch_size=256,sampling_rate=1,shuffle=False)

val_gen[i] = tf.keras.utils.timeseries_dataset_from_array( X_val[i], Y_val[i], sequence_length=160, sequence_stride=1, batch_size=256,sampling_rate=1,shuffle=False)

batch =train_gen
inputs, target=batch
input= inputs.shape[1], inputs.shape[2]
print(input)


def ann():


model = Sequential()
model.add(Dense(1000, input_shape=input))
model.add(Dense(100))
model.add(Flatten())
model.add(Dense(2, activation='linear'))
model.compile(optimizer=Adam(learning_rate = 1e-6), loss= 'mse', metrics=(['accuracy']))
model.summary()
return model
model = ann()

history=model.fit(train_gen[i], validation_data=val_gen[i], shuffle=True,epochs=10,verbose=1)

但是我在下一行得到错误，我该如何解决这个问题

Indexerror:

train_data[i]=X_train.iloc[i:i+length, :]
IndexError: list assignment index out of range

Answer 1

i不是索引train_data和train_tar的正确变量。使用枚举来解决问题:

# Now:
# i -> the index of the current loop
# j -> the slicing index
for i, j in enumerate(range(0, len(X_train), length)):
train_data.append(X_train.iloc[j:j+length, :])
train_tar.append(Y_train.iloc[j:j+length, :])
# Rest of your code