具有sparse_categorical_crossentropy的Tensoflow Keras - Nan损失.&l

Question

我对tensorflow和深度学习很陌生。我试图训练一个简单的网络，我在第一个历元上得到NaN损失。我检查了重量，它们也变成了纳米。我试图将学习率降低到1e-8。即使这样也无济于事。请让我知道我做错了什么。

import tensorflow as tf
import numpy as np
a = tf.constant(
np.array([
[ 8, 51,  1, 30,  3, 30],
[ 1,  5,  2,  1,  1,  1],
[11, 29,  1,  1,  1,  1],
[ 1, 43,  1, 44, 27, 45],
[ 1,  1,  1,  1,  1, 19],
])
)
l = tf.constant(np.array([[2], [1], [1], [2], [3]]))
model = tf.keras.Sequential([
tf.keras.layers.Dense(3, activation='softmax', input_shape=[6])
])
optimizer = tf.keras.optimizers.Adam(lr=1e-8)
model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer)
print(model.summary())
history = model.fit(a,l, epochs=1, verbose=2)

Answer 1

你的l形状不是分类交叉熵的正确形状。此外，你的标签必须从0到2，而不是从1到3。

import tensorflow as tf
import numpy as np
a = tf.constant(
np.array([
[ 8, 51,  1, 30,  3, 30],
[ 1,  5,  2,  1,  1,  1],
[11, 29,  1,  1,  1,  1],
[ 1, 43,  1, 44, 27, 45],
[ 1,  1,  1,  1,  1, 19],
])
)
l = tf.constant(np.array([1, 0, 0, 1, 2]))
model = tf.keras.Sequential([
tf.keras.layers.Dense(3, activation='softmax', input_shape=[6])
])
optimizer = tf.keras.optimizers.Adam(lr=1e-8)
model.compile(loss='sparse_categorical_crossentropy', optimizer=optimizer)
print(model.summary())
history = model.fit(a,l, epochs=3, verbose=2)

Model: "sequential_11"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_39 (Dense)             (None, 3)                 21        
=================================================================
Total params: 21
Trainable params: 21
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/3
1/1 - 0s - loss: 0.2769
Epoch 2/3
1/1 - 0s - loss: 0.2769
Epoch 3/3
1/1 - 0s - loss: 0.2769

Answer 2

错误在这里:

l = tf.constant(np.array([[2], [1], [1], [2], [0]])) # change the last index