我正在使用tensorflow中输入管道的数据集API(版本:R1.2(。我构建了数据集并以128的批量大小进行了批处理。
不幸的是, dataset.output_shape
在第一个维度中返回维度(无(,因此RNN引起了错误:
Traceback (most recent call last):
File "untitled1.py", line 188, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "untitled1.py", line 121, in main
run_training()
File "untitled1.py", line 57, in run_training
is_training=True)
File "/home/harold/huawei/ConvLSTM/ConvLSTM.py", line 216, in inference
initial_state=initial_state)
File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 566, in dynamic_rnn
dtype=dtype)
File "/home/harold/anaconda2/envs/tensorflow_py2.7/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 636, in _dynamic_rnn_loop
"Input size (depth of inputs) must be accessible via shape inference,"
ValueError: Input size (depth of inputs) must be accessible via shape inference, but saw value None.
我认为此错误是由输入形状引起的,第一个维度应为批处理大小,但不要进行。
这是代码:
origin_dataset = Dataset.BetweenS_Dataset(FLAGS.data_path)
train_dataset = origin_dataset.train_dataset
test_dataset = origin_dataset.test_dataset
shuffle_train_dataset = train_dataset.shuffle(buffer_size=10000)
shuffle_batch_train_dataset = shuffle_train_dataset.batch(128)
batch_test_dataset = test_dataset.batch(FLAGS.batch_size)
iterator = tf.contrib.data.Iterator.from_structure(
shuffle_batch_train_dataset.output_types,
shuffle_batch_train_dataset.output_shapes)
(images, labels) = iterator.get_next()
training_init_op = iterator.make_initializer(shuffle_batch_train_dataset)
test_init_op = iterator.make_initializer(batch_test_dataset)
print(shuffle_batch_train_dataset.output_shapes)
i打印output_shapes
,并给出:
(TensorShape([Dimension(None), Dimension(36), Dimension(100)]), TensorShape([Dimension(None)]))
我想应该是128,因为我已经批处理数据集:
(TensorShape([Dimension(128), Dimension(36), Dimension(100)]), TensorShape([Dimension(128)]))
此功能已与drop_remainder
参数一样添加如下:
batch_test_dataset = test_dataset.batch(FLAGS.batch_size, drop_remainder=True)
来自文档:
drop_remainder :(可选。(tf. -bool标量tf.tensor,表示是否应在情况下删除最后一批的元素,其较小的元素;默认行为不是掉落较小的批次。
它们在实现中的硬编码批量大小,它总是会返回(tf 1.3(。
def _padded_shape_to_batch_shape(s):
return tensor_shape.vector(None).concatenate(
tensor_util.constant_value_as_shape(s))
这样,它们可以批量所有元素(例如dataset_size=14
,batch_size=5
,last_batch_size=4
(。
您可以使用dataset.filter和dataset.map来修复此问题
d = contrib.data.Dataset.from_tensor_slices([[5] for x in range(14)])
batch_size = 5
d = d.batch(batch_size)
d = d.filter(lambda e: tf.equal(tf.shape(e)[0], batch_size))
def batch_reshape(e):
return tf.reshape(e, [args.batch_size] + [s if s is not None else -1 for s in e.shape[1:].as_list()])
d = d.map(batch_reshape)
r = d.make_one_shot_iterator().get_next()
print('dataset_output_shape = %s' % r.shape)
with tf.Session() as sess:
while True:
print(sess.run(r))
输出
dataset_output_shape =(5,1(
[[5] [5] [5] [5] [5]]
[[5] [5] [5] [5] [5]]
OutofrangeError