如何从TFRecordDataset中计算steps_per_epoch

Question

训练tensorflow model时:

model.fit(..., steps_per_epoch=10000, ....)

我想从提供的数据集计算steps_per_epoch:

dataset = tf.data.TFRecordDataset([filenames])
dataset = dataset.repeat(1)
dataset = dataset.batch(512)
total = 0
for i in dataset:
total += 1
print("Total is {}".format(total))

输出为:

Total is 393

steps_per_epoch是否等于393?

或

steps_per_epoch=393 / 512?

Answer 1

参数steps_per_epoch只有在我们使用大型数据集时才是模型训练的一部分。steps_per_epoch决定在单个数据集中训练的批次，以提高模型的准确性。

steps_per_epoch取决于batch_size和training_set的大小

batch_size=50
trainingsize = 30000 
def calculate_spe(y, batch_size):
return int(math.ceil((1. * y) / batch_size)) 
stepsPerEpoch = calculate_spe(trainingsize, batch_size)

在计算每个epoch的步数后，您可以将其用作:

model.fit(..., steps_per_epoch = stepsPerEpoch , ....)