如何优化 Tensorflow CNN?

Question

我对Tensorflow很陌生，所以如果我的问题被问到无知，我深表歉意。

我有一个非常简单的CNN Tensorflow，它可以拍摄图像并输出另一个图像。批处理大小仅为 5，在 epoch 之间运行需要几分钟，并且经常在 5 个 epoch 后崩溃。(我在具有 3.6 GB RAM 的 Mac 上使用 python 3.6.5

(这是我程序的一个片段

learning_rate = 0.01
inputs_ = tf.placeholder(tf.float32, (None, 224, 224, 3), name='inputs')
targets_ = tf.placeholder(tf.float32, (None, 224, 224, 1), name='targets')
### Encoder
conv1 = tf.layers.conv2d(inputs=inputs_, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 224x224x32
maxpool1 = tf.layers.max_pooling2d(conv1, pool_size=(2,2), strides=(2,2), padding='same')
# Now 112x112x32
conv2 = tf.layers.conv2d(inputs=maxpool1, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 112x112x32
maxpool2 = tf.layers.max_pooling2d(conv2, pool_size=(2,2), strides=(2,2), padding='same')
# Now 56x56x32
conv3 = tf.layers.conv2d(inputs=maxpool2, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 56x56x32
maxpool3 = tf.layers.max_pooling2d(conv3, pool_size=(2,2), strides=(2,2), padding='same')
# Now 28x28x32
conv4 = tf.layers.conv2d(inputs=maxpool3, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 28x28x32
maxpool4 = tf.layers.max_pooling2d(conv4, pool_size=(2,2), strides=(2,2), padding='same')
# Now 14x14x32
conv5 = tf.layers.conv2d(inputs=maxpool4, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 14x14x32
maxpool5 = tf.layers.max_pooling2d(conv5, pool_size=(2,2), strides=(2,2), padding='same')
# Now 7x7x32
conv6 = tf.layers.conv2d(inputs=maxpool5, filters=16, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 7x7x16
encoded = tf.layers.max_pooling2d(conv6, pool_size=(2,2), strides=(2,2), padding='same')
# Now 4x4x16
### Decoder
upsample1 = tf.image.resize_images(encoded, size=(7,7), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
# Now 7x7x16
conv7 = tf.layers.conv2d(inputs=upsample1, filters=16, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 7x7x16
upsample2 = tf.image.resize_images(conv7, size=(14,14), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
# Now 14x14x16
conv8 = tf.layers.conv2d(inputs=upsample2, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 14x14x32
upsample3 = tf.image.resize_images(conv8, size=(28,28), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
# Now 28x28x32
conv9 = tf.layers.conv2d(inputs=upsample3, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 28x28x32
upsample4 = tf.image.resize_images(conv9, size=(56,56), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
# Now 56x56x32
conv10 = tf.layers.conv2d(inputs=upsample3, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 56x56x32
upsample5 = tf.image.resize_images(conv10, size=(112,112), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
# Now 112x112x32
conv11 = tf.layers.conv2d(inputs=upsample5, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 112x112x32
upsample6 = tf.image.resize_images(conv11, size=(224,224), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
# Now 224x224x32
conv12 = tf.layers.conv2d(inputs=upsample6, filters=32, kernel_size=(3,3), padding='same', activation=tf.nn.relu)
# Now 224x224x32
logits = tf.layers.conv2d(inputs=conv12, filters=1, kernel_size=(3,3), padding='same', activation=None)
#Now 224x224x1
# Pass logits through sigmoid and calculate the cross-entropy loss
loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=targets_, logits=logits)
# Get cost and define the optimizer
cost = tf.reduce_mean(loss)
opt = tf.train.AdamOptimizer(learning_rate).minimize(cost)
imagelist = ... #array of all images with 3 channels
imagelabellist = ... #array of all images with 1 channel
epochs = 15
for e in range(epochs):
imgs_large = imagelist
imgs_target_large = imagelabellist
shaped_imgs = tf.image.resize_images(imgs_large, [224, 224])
shaped_imgs_target = tf.image.resize_images(imgs_target_large, [224, 224])
# Get images from the batch
imgs = sess.run(shaped_imgs)
imgs_target = sess.run(shaped_imgs_target)
batch_cost, _ = sess.run([cost, opt], feed_dict={inputs_: imgs, targets_: imgs_target})

这是CNN的输出

epoch: #1
0 minutes between epoch
epoch: #2
3 minutes between epoch
epoch: #3
3 minutes between epoch
epoch: #4
12 minutes between epoch
epoch: #5

。

我愿意接受有关如何解决此问题的任何建议。谢谢。

Answer 1

tf.image.resize_images

是一个图形操作，因此您将更多节点附加到图形中(这解释了运行时间的增加(。在训练循环添加之前sess.graph.finalize()如果添加节点，它将抛出错误来检查这一点。

如果您将resize_images移到循环之外，这应该可以解决问题。