TF2.0:转换模型:还原保存的模型时出错:检查点(根)中未解析的对象



我正在尝试恢复检查点并预测不同句子的NMT注意力模型。在恢复检查点和预测时,我得到胡言乱语的结果,并在下面发出警告:

Unresolved object in checkpoint (root).optimizer.iter: attributes {
name: "VARIABLE_VALUE"
full_name: "Adam/iter"
checkpoint_key: "optimizer/iter/.ATTRIBUTES/VARIABLE_VALUE"
}

以下是我收到的其他警告和结果:

WARNING: Logging before flag parsing goes to stderr.
W1008 09:57:52.766877 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer.iter
W1008 09:57:52.767037 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer.beta_1
W1008 09:57:52.767082 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer.beta_2
W1008 09:57:52.767120 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer.decay
W1008 09:57:52.767155 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer.learning_rate
W1008 09:57:52.767194 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.embedding.embeddings
W1008 09:57:52.767228 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.gru.state_spec
W1008 09:57:52.767262 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.fc.kernel
W1008 09:57:52.767296 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.fc.bias
W1008 09:57:52.767329 4594230720 util.py:244] Unresolved object in checkpoint: (root).encoder.embedding.embeddings
W1008 09:57:52.767364 4594230720 util.py:244] Unresolved object in checkpoint: (root).encoder.gru.state_spec
W1008 09:57:52.767396 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.gru.cell.kernel
W1008 09:57:52.767429 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.gru.cell.recurrent_kernel
W1008 09:57:52.767461 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.gru.cell.bias
W1008 09:57:52.767493 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.attention.W1.kernel
W1008 09:57:52.767526 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.attention.W1.bias
W1008 09:57:52.767558 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.attention.W2.kernel
W1008 09:57:52.767590 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.attention.W2.bias
W1008 09:57:52.767623 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.attention.V.kernel
W1008 09:57:52.767657 4594230720 util.py:244] Unresolved object in checkpoint: (root).decoder.attention.V.bias
W1008 09:57:52.767688 4594230720 util.py:244] Unresolved object in checkpoint: (root).encoder.gru.cell.kernel
W1008 09:57:52.767721 4594230720 util.py:244] Unresolved object in checkpoint: (root).encoder.gru.cell.recurrent_kernel
W1008 09:57:52.767755 4594230720 util.py:244] Unresolved object in checkpoint: (root).encoder.gru.cell.bias
W1008 09:57:52.767786 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.embedding.embeddings
W1008 09:57:52.767818 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.fc.kernel
W1008 09:57:52.767851 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.fc.bias
W1008 09:57:52.767884 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.embedding.embeddings
W1008 09:57:52.767915 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.gru.cell.kernel
W1008 09:57:52.767949 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.gru.cell.recurrent_kernel
W1008 09:57:52.767981 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.gru.cell.bias
W1008 09:57:52.768013 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.attention.W1.kernel
W1008 09:57:52.768044 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.attention.W1.bias
W1008 09:57:52.768077 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.attention.W2.kernel
W1008 09:57:52.768109 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.attention.W2.bias
W1008 09:57:52.768143 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.attention.V.kernel
W1008 09:57:52.768175 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.attention.V.bias
W1008 09:57:52.768207 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.gru.cell.kernel
W1008 09:57:52.768239 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.gru.cell.recurrent_kernel
W1008 09:57:52.768271 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.gru.cell.bias
W1008 09:57:52.768303 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.embedding.embeddings
W1008 09:57:52.768335 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.fc.kernel
W1008 09:57:52.768367 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.fc.bias
W1008 09:57:52.768399 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.embedding.embeddings
W1008 09:57:52.768431 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.gru.cell.kernel
W1008 09:57:52.768463 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.gru.cell.recurrent_kernel
W1008 09:57:52.768495 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.gru.cell.bias
W1008 09:57:52.768527 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.attention.W1.kernel
W1008 09:57:52.768559 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.attention.W1.bias
W1008 09:57:52.768591 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.attention.W2.kernel
W1008 09:57:52.768623 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.attention.W2.bias
W1008 09:57:52.768654 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.attention.V.kernel
W1008 09:57:52.768686 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.attention.V.bias
W1008 09:57:52.768718 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.gru.cell.kernel
W1008 09:57:52.768750 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.gru.cell.recurrent_kernel
W1008 09:57:52.768782 4594230720 util.py:244] Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.gru.cell.bias
W1008 09:57:52.768816 4594230720 util.py:252] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/alpha/guide/checkpoints#loading_mechanics for details.
Input: <start> hola <end>
Predicted translation: ? attack now relax hello 

最后的警告说:

"检查点已恢复(例如 tf.train.Checkpoint.restore 或 tf.keras.Model.load_weights(,但并非所有检查点值都已使用......" 什么意思?

这意味着您没有使用已还原的所有检查点值。

发生这种情况是因为您正在还原具有训练信息(如优化器变量(的模型,但您仅将其用于预测,而不是训练。预测时,您不需要保存的优化器值,这就是为什么程序告诉您它们未被使用的原因。

如果使用此还原的模型对新数据进行训练,则此警告将消失。

您可以使用model.load_weights(...).expect_partial()tf.train.Checkpoint.restore(...).expect_partial()来使这些警告静音。

更好的解决方案是在训练时仅保存推理所需的变量:

saver = tf.train.Saver(tf.model_variables())

tf.model_variables()是模型中用于推理的变量对象的子集(请参阅张量流文档(。

任何在 Tensorflow 对象检测(一种继续训练的有效解决方案(中遇到此错误的人,都会将pipeline.config中的num_steps值更新为比上次运行更高的值:

源语言:

num_steps: 25000
optimizer {
momentum_optimizer: {
learning_rate: {
cosine_decay_learning_rate {
learning_rate_base: .04
total_steps: 25000

更新:

num_steps: 50000
optimizer {
momentum_optimizer: {
learning_rate: {
cosine_decay_learning_rate {
learning_rate_base: .04
total_steps: 50000

我用了:tf.train.Checkpoint.restore(...(。expect_partial(( 恢复我的检查点并将其用于推理。它对我有用

最新更新