Tensorflow Dataset/Iterator，用于在CNN中使用训练和测试数据进行评估

Question

设置：我想通过在回归设置中的输入批次上训练 CNN 来预测一个值。我还想评估和计算每个纪元后的损失，所以我需要在运行时在数据集之间切换。

Input: [num_examples, height, width, channels] -> [num_examples, y]

我想使用新的数据集 API，因为我想避免在训练期间自己喂食批次。

我也不想将我的数据集存储在计算图中，因为数据集大于 2GB，但足够小以存储在内存中。

这是我当前的设置：

def initialize_datasets(x, y,...):
dataset_train = tf.data.Dataset.from_tensor_slices((x, y))
dataset_train = dataset_train.apply(tf.contrib.data.shuffle_and_repeat(buffer_size=examples_train, count=epochs))
dataset_train = dataset_train.batch(batch_size)
dataset_test = tf.data.Dataset.from_tensor_slices((x, y))
dataset_test = dataset_test.apply(tf.contrib.data.shuffle_and_repeat(buffer_size=examples_test, count=-1))
dataset_test = dataset_test.batch(batch_size)
# Iterator 
iterator_train = dataset_train.make_initializable_iterator()
iterator_test = dataset_test.make_initializable_iterator()
return iterator_train, iterator_test

def get_input_batch_data(testing, iterator_train, iterator_test):
features, labels = tf.cond(testing, lambda: iterator_test.get_next(), lambda: iterator_train.get_next())
return features, labels

然后在my model()函数中：

#1
iterator_train, iterator_test = initialize_datasets(x, y, ...)
#2
features, labels = get_input_batch_data(testing, iterator_train, 
iterator_test)
# forward pass, loss, etc
...
with tf.Session as sess:
#initialize with train data, trainX[num_examples, height, width, channels]
sess.run(iterator_train.initializer, feed_dict={x: trainX, y: trainY, 
batch_size: batchsize})
#initialize with test data
sess.run(iterator_test.initializer, feed_dict={x: testX, y: testY, 
batch_size: NUM_EXAMPLES_TEST})
for i in range(EPOCHS)
for j in range(NUM_BATCHES)
_, batch_loss = sess.run([train_step, loss], feed_dict={testing: 
False,  i: iters_total, pkeep: p_keep})   
# after 1 epoch, calculate loss and whole test data set
epoch_test_loss = sess.run(loss, feed_dict={testing: True, i: 
iters_total, pkeep: 1}) 

这是输出：

Iter: 44, Epoch: 0 (8.46s), Train-Loss: 103011.18, Test-Loss: 100162.34
Iter: 89, Epoch: 1 (4.17s), Train-Loss: 93699.51, Test-Loss: 92130.21
Iter: 134, Epoch: 2 (4.13s), Train-Loss: 90217.82, Test-Loss: 88978.74
Iter: 179, Epoch: 3 (4.14s), Train-Loss: 88503.13, Test-Loss: 87515.81
Iter: 224, Epoch: 4 (4.18s), Train-Loss: 87336.62, Test-Loss: 86486.40
Iter: 269, Epoch: 5 (4.10s), Train-Loss: 86388.38, Test-Loss: 85637.64
Iter: 314, Epoch: 6 (4.14s), Train-Loss: 85534.52, Test-Loss: 84858.43
Iter: 359, Epoch: 7 (4.29s), Train-Loss: 84693.19, Test-Loss: 84074.78
Iter: 404, Epoch: 8 (4.20s), Train-Loss: 83973.64, Test-Loss: 83314.47
Iter: 449, Epoch: 9 (4.40s), Train-Loss: 83149.73, Test-Loss: 82541.73

问题：

• 此输出向我表明我的数据集管道不起作用，因为测试损失是根据训练数据计算的，反之亦然，因为这些损失彼此太接近
• 我将使用哪种类型的迭代器和数据集来执行此任务？

我也在这里上传了整个模型：https://github.com/toemm/TF-CNN-regression/blob/master/BA-CNN_so.ipynb

Answer 1

显而易见的答案是：您不希望在同一图形中执行此操作，因为评估图形与训练图形不同。

• 辍学具有固定乘数(无采样)
• BatchNorm 使用累积统计信息，不更新 EMA

所以解决方案实际上是构建两个不同的东西，例如

import numpy as np
import tensorflow as tf

X_train = tf.constant(np.ones((100, 2)), 'float32')
X_val = tf.constant(np.zeros((10, 2)), 'float32')
iter_train = tf.data.Dataset.from_tensor_slices(
X_train).make_initializable_iterator()
iter_val = tf.data.Dataset.from_tensor_slices(
X_val).make_initializable_iterator()

def graph(x, is_train=True):
return x

output_train = graph(iter_train.get_next(), is_train=True)
output_val = graph(iter_val.get_next(), is_train=False)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(iter_train.initializer)
sess.run(iter_val.initializer)
for train_iter in range(100):
print(sess.run(output_train))
for train_iter in range(10):
print(sess.run(output_val))