VGG19权重在单次迭代后在所有层中都变成了NaN

Question

我正在使用预先训练的VGG19模型练习神经风格迁移。当我执行训练时，经过 1 次迭代，VGG19 所有层中的所有权重都变为 NaN。当我跳过对成本函数执行 sess.run(( 的部分时，权重保持不变，正如预期的那样。但这是否意味着我的成本函数导致了问题？

def model_nst(sess, input_image, num_iterations = 2):
# Initialize global variables (you need to run the session on the initializer)
sess.run(tf.global_variables_initializer())
# Run the noisy input image (initial generated image) through the model.
sess.run(model['input'].assign(input_image))
for i in range(num_iterations):
# Run the session on the train_step to minimize the total cost
sess.run(train_step)
# Compute the generated image by running the session on the current model['input']
generated_image = sess.run(model['input'])
print("generated_image:")
print(generated_image) #becomes NaN too after 1 iteration
if i%1 == 0:
#Jc = sess.run(J_content)
#Js = sess.run(J_style)
#Jt = sess.run(J)
#print("Iteration " + str(i) + " :")
#print("total cost = " + str(Jt))
#print("content cost = " + str(Jc))
#print("style cost = " + str(Js))
# save current generated image in the "/output" directory
util.save_image(util.CONFIG.OUTPUT_DIR + str(i) + ".png", generated_image)
# save last generated image
util.save_image('out/generated_image.jpg', generated_image)
return generated_image

Answer 1

您使用哪些数据来执行迁移学习？数据可能具有 NAN。如果您的数据形状或类型错误等，也可能发生这种情况。

如果上述内容已签出，则提高学习率或减小批量大小。初始权重导致梯度消失或爆炸。更改批大小将绕过此问题。