如何对LSTM网络进行逐像素分类？具体来说，在Tensorflow中。
我的直觉告诉我，输出张量(来自代码的pred和y)应该是与输入图像具有相同分辨率的二维张量。换句话说，输入图像将是200x200，而输出分类将是200x200。
Udacity课程包括一个示例LSTM网络，其中输入图像为28x28。然而，它是一个图像(作为一个整体——手写MNIST数据集)分类网络。
我的想法是，我可以用[n_input][n_steps]替换所有维度为[n_classes]的张量(下面的代码)。然而，它在矩阵乘法中抛出了一个错误。
Udacity示例代码部分如下：
n_input = 28 # MNIST data input (img shape: 28*28)
n_steps = 28 # timesteps
n_hidden = 128 # hidden layer num of features
n_classes = 10 # MNIST total classes (0-9 digits)
# tf Graph input
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])
# Define weights
weights = {
'hidden': tf.Variable(tf.random_normal([n_input, n_hidden])),
'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))
}
biases = {
'hidden': tf.Variable(tf.random_normal([n_hidden])),
'out': tf.Variable(tf.random_normal([n_classes]))
}
def RNN(x, weights, biases):
# Prepare data shape to match `rnn` function requirements
# Current data input shape: (batch_size, n_steps, n_input)
# Permuting batch_size and n_steps
x = tf.transpose(x, [1, 0, 2])
# Reshaping to (n_steps*batch_size, n_input)
x = tf.reshape(x, [-1, n_input])
# Split to get a list of 'n_steps' tensors of shape (batch_size, n_hidden)
# This input shape is required by `rnn` function
x = tf.split(0, n_steps, x)
# Define a lstm cell with tensorflow
lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
pdb.set_trace()
# Get lstm cell output
outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)
# Linear activation, using rnn inner loop last output
return tf.matmul(outputs[-1], weights['out']) + biases['out']
pred = RNN(x, weights, biases)
-----------------------------------------------------------------------------
然后我的代码看起来是这样的：
n_input = 200 # data data input (img shape: 28*28)
n_steps = 200 # timesteps
n_hidden = 128 # hidden layer num of features
n_classes = 2 # data total classes (0-9 digits)
# tf Graph input
x = tf.placeholder("float", [None, n_input, n_steps])
y = tf.placeholder("float", [None, n_input, n_steps])
# Define weights
weights = {
'hidden': tf.Variable(tf.random_normal([n_input, n_hidden]), dtype="float32"),
'out': tf.Variable(tf.random_normal([n_hidden, n_input, n_steps]), dtype="float32")
}
biases = {
'hidden': tf.Variable(tf.random_normal([n_hidden]), dtype="float32"),
'out': tf.Variable(tf.random_normal([n_input, n_steps]), dtype="float32")
}
def RNN(x, weights, biases):
# Prepare data shape to match `rnn` function requirements
# Current data input shape: (batch_size, n_steps, n_input)
# Permuting batch_size and n_steps
pdb.set_trace()
x = tf.transpose(x, [1, 0, 2])
# Reshaping to (n_steps*batch_size, n_input)
x = tf.reshape(x, [-1, n_input])
# Split to get a list of 'n_steps' tensors of shape (batch_size, n_hidden)
# This input shape is required by `rnn` function
x = tf.split(0, n_steps, x)
# Define a lstm cell with tensorflow
lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
pdb.set_trace()
# Get lstm cell output
outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)
# Linear activation, using rnn inner loop last output
# return tf.matmul(outputs[-1], weights['out']) + biases['out']
return tf.batch_matmul(outputs[-1], weights['out']) + biases['out']
pred = RNN(x, weights, biases)
线return tf.batch_matmul(outputs[-1], weights['out']) + biases['out']就是问题所在。因为outputs是2D张量的向量，而weights['out']是3D张量的向量。
我想也许我可以更改outputs的维度，但这需要深入RNN对象(在API中)。
我有什么选择？我可以做一些整形吗？如果是，我应该重塑什么，以什么方式重塑？
您不能与维度为3的形状为[n_hidden, n_input, n_step]的矩阵进行矩阵乘法运算。
您可以输出维度为[batch_size, n_input * n_step]的向量，然后将其重塑为[batch_size, n_input, n_step]。
weights = {
'hidden': ... ,
'out': tf.Variable(tf.random_normal([n_hidden, n_input * n_steps]), dtype="float32")
}
biases = {
'hidden': ... ,
'out': tf.Variable(tf.random_normal([n_input * n_steps]), dtype="float32")
}
# ...
pred = RNN(x, weights, biases)
pred = tf.reshape(pred, [-1, n_input, n_steps])
在您的模型上
然而，您在这里所做的是在图像的每一列上使用RNN。您试图获取图像的每一个切片(总共200个)并对其进行迭代，这根本不会产生好的结果。
如果你想处理图像，我建议你看看TensorFlow的这篇教程，在那里你可以学习使用卷积，它比RNN在图像上更有效。