如何使用tfa.metrics.F1Score与image_dataset_from_directory正确吗?<

Question

Colab代码在这里:

我遵循这里的文档来获得多类预测的结果

当我使用

训练时

#last layer
tf.keras.layers.Dense(2, activation='softmax')
model.compile(optimizer="adam",
loss=tf.keras.losses.CategoricalCrossentropy(),
metrics=[tf.keras.metrics.CategoricalAccuracy(),
tfa.metrics.F1Score(num_classes=2, average='macro')])

我

144/144 [==] - 8s 54ms/step - loss: 0.0613 - categorical_accuracy: 0.9789 - f1_score: 0.9788 - val_loss: 0.0826 - val_categorical_accuracy: 0.9725 - val_f1_score: 0.9722

当我这样做的时候:

model.evaluate(val_ds)

我

16/16 [==] - 0s 15ms/step - loss: 0.0826 - categorical_accuracy: 0.9725 - f1_score: 0.9722
[0.08255868405103683, 0.9725490212440491, 0.9722140431404114]

我想使用官网的metric.result。当我加载下面的代码，我得到0.4875028这是错误的。如何得到正确的predicted_categories和true_categories?

metric = tfa.metrics.F1Score(num_classes=2, average='macro')
predicted_categories = model.predict(val_ds)
true_categories = tf.concat([y for x, y in val_ds], axis=0).numpy() 
metric.update_state(true_categories, predicted_categories)
result = metric.result()
print(result.numpy())
#0.4875028

下面是我加载数据的方法

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
main_folder,
validation_split=0.1,
subset="training",
label_mode='categorical',
seed=123,
image_size=(dim, dim))
val_ds = tf.keras.preprocessing.image_dataset_from_directory(
main_folder,
validation_split=0.1,
subset="validation",
label_mode='categorical',
seed=123,
image_size=(dim, dim))

Answer 1

From: https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image_dataset_from_directory

tf.keras.preprocessing.image_dataset_from_directory(
directory, labels='inferred', label_mode='int',
class_names=None, color_mode='rgb', batch_size=32, image_size=(256,
256), shuffle=True, seed=None, validation_split=None, subset=None,
interpolation='bilinear', follow_links=False
)

默认的shuffle是True，这对你的val_ds是一个问题，我们不想洗牌。

正确的指标是在培训期间报告的指标;此外，我建议您也可以手动检索验证数据集，并在对其进行预测后检查指标(不一定通过flow_from_directory())。