在Theano/Lasagne中创建前馈自动编码器时出现尺寸错误

Question

我想创建一个简单的自编码器，有3000个输入，2个隐藏和3000个输出神经元:

def build_autoencoder(input_var=None):
    l_in = InputLayer(shape=(None,3000), input_var=input_var)
    l_hid = DenseLayer(
            l_in, num_units=2,
            nonlinearity=rectify,
            W=lasagne.init.GlorotUniform())
    l_out = DenseLayer(
            l_hid, num_units=3000,
            nonlinearity=softmax)
    return l_out 

训练数据的形状如下:

train.shape = (3000,3)

输入、目标和损失函数的定义:

import sys
import os
import time
import numpy as np
import theano
import theano.tensor as T
import lasagne
from lasagne.updates import rmsprop
from lasagne.layers import DenseLayer, DropoutLayer, InputLayer
from lasagne.nonlinearities import rectify, softmax
from lasagne.objectives import categorical_crossentropy
# Creating the Theano variables
input_var = T.dmatrix('inputs')
target_var = T.dmatrix('targets')
# Building the Theano expressions on these variables
network = build_autoencoder(input_var)
prediction = lasagne.layers.get_output(network)
loss = categorical_crossentropy(prediction, target_var)
loss = loss.mean()
test_prediction = lasagne.layers.get_output(network,
                                                    deterministic=True)
test_loss = categorical_crossentropy(test_prediction, target_var)
test_loss = test_loss.mean()
test_acc = T.mean(T.eq(T.argmax(test_prediction, axis=1), target_var),
                          dtype=theano.config.floatX)

我只是运行一个epoch，但得到一个错误:

params = lasagne.layers.get_all_params(network, trainable=True)
updates = rmsprop(loss, params, learning_rate=0.001)
# Compiling the graph by declaring the Theano functions
train_fn = theano.function([input_var, target_var],
                                   loss, updates=updates)
val_fn = theano.function([input_var, target_var],
                                 [test_loss, test_acc])
# For loop that goes each time through the hole training
# and validation data
print("Starting training...")
for epoch in range(1):
    # Going over the training data
    train_err = 0
    train_batches = 0
    start_time = time.time()
    print 'test1'
    train_err += train_fn(train, train)
    train_batches += 1
    # Going over the validation data
    val_err = 0
    val_acc = 0
    val_batches = 0
    err, acc = val_fn(train, train)
    val_err += err
    val_acc += acc
    val_batches += 1
    # Then we print the results for this epoch:
    print("Epoch {} of {} took {:.3f}s".format(epoch + 1, num_epochs, time.time() - start_time))
    print("training loss:tt{:.6f}".format(train_err / train_batches))
    print("validation loss:tt{:.6f}".format(val_err / val_batches))
    print("validation accuracy:tt{:.2f} %".format(val_acc / val_batches * 100))

错误:

ValueError: ('shapes (3000,3) and (3000,2) not aligned: 3 (dim 1)3000 (dim 0)'，(3000, 3)，(3000, 2))应用导致错误的节点:Dot22(inputs, W) topossort索引:3输入类型:[TensorType(float64，输入形状:[(3000,3)，(3000, 2)]输入步长:[(24,8)，(16,8)]输入值:['not]显示'，'不显示']输出客户端:[[Elemwise{添加、no_inplace} (Dot22.0, InplaceDimShuffle {x, 0} 0),Elemwise{复合{(i0 * (Abs (i1) + i2和i3))}} [(0,(TensorConstant{(1,1) of 0.5}， Elemwise{add,no_inplace}.0，Dot22.0 InplaceDimShuffle {x, 0} 0)]]

对我来说，似乎自动编码器的瓶颈是问题所在。什么好主意吗?

Answer 1

刚刚从我的IBM学院(Erwan)得到了一些帮助，我已经发布了这个GIST的工作解决方案，相关部分是这些:

首先，得到正确的训练数据的形状:

train.shape = (3, 3000)

然后在InputLayer上使用相同的形状:

def build_autoencoder(input_var=None):
    l_in = InputLayer(shape=(3, 3000), input_var=input_var)
    l_hid = DenseLayer(
            l_in, num_units=2,
            nonlinearity=rectify,
            W=lasagne.init.GlorotUniform())
    l_out = DenseLayer(
            l_hid, num_units=3000,
            nonlinearity=softmax)
return l_out

所以这解决了，下一个问题是在训练期间获得下降成本，但这是另一个主题:)