测试集的准确性不会增加

Question

我正在使用图像数据集，作为学习的机会，从头开始编码多项式。

我尝试了多种不同的批量大小(50、100、200(和学习率(.001、.05、.1、.5(。

我似乎仍然不能超过1%，所以我想知道我的代码中是否缺少某些东西，或者这是否是我可以通过浅层学习方法获得的最好的方法。我尝试了使用sklearn进行逻辑回归，可以得到大约7%(我知道，非常糟糕！(，并试图在tensorflow中为它重新创建代码。

有没有办法判断它是否真的在改善？任何帮助都非常感谢！谢谢

import numpy as np
import tensorflow as tf 
from keras.datasets import cifar100
(x_train, y_train), (x_test, y_test) = cifar100.load_data()

x_train_data= np.zeros((50000, 32, 32))
for i, x in enumerate(x_train):
x_train_data[i] = rgb2gray(x)

x_test_data= np.zeros((10000, 32, 32))
for i, x in enumerate(x_test):
x_test_data[i] = rgb2gray(x)
#convert data to a vector
x_train_data = x_train_data.reshape((50000, -1))
x_test_data = x_test_data.reshape((10000, -1))

NUM_CLASSES = 100
X_DIM = 32
Y_DIM = 32
PIXELS_PER_SAMPLE = X_DIM*Y_DIM
#create placeholders
X =  tf.placeholder(tf.float32, [None, PIXELS_PER_SAMPLE])
Y = tf.placeholder(tf.float32, [None, NUM_CLASSES])

#create variables
with tf.variable_scope("multi_class_logistic_model", reuse=tf.AUTO_REUSE):
W = tf.get_variable('Weight_matrix', initializer = tf.random_normal(shape = (X_DIM*Y_DIM, NUM_CLASSES)))    
W_o= tf.get_variable('bias', initializer = tf.random_normal(shape = [NUM_CLASSES]))
Y_pred = tf.matmul(X, W)  + W_o

#convert values to probability vector using softmax
Y_pred_prob = tf.nn.softmax(logits=Y_pred)
#create loss function (cross entropy)
loss = -tf.reduce_mean(Y * tf.log(Y_pred_prob))
#create accuracy measurement
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(Y_pred,1),tf.argmax(Y,1)),tf.float32))
#create optimizer
opt = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(loss)

BATCH_SIZE = 100
NUM_EPOCHS = 10000
#function to batch data. One hot encodes the labels
def next_batch(num, data, labels):
'''
Return a total of `num` random samples and labels. 
'''
idx = np.arange(0 , len(data))
np.random.shuffle(idx)
idx = idx[:num]
data_shuffle = [data[ i] for i in idx]
labels_shuffle = [labels[ i] for i in idx]
onehot_encoded = list()
for value in labels_shuffle:
letter = [0 for _ in range(100)]
letter[value] = 1
onehot_encoded.append(letter)
return np.asarray(data_shuffle), np.asarray(onehot_encoded)

#one hot encode the labels test set
y_test_onehot_encoded = list()
for value in y_test.ravel():
letter = [0 for _ in range(100)]
letter[value] = 1
y_test_onehot_encoded.append(letter)
y_test_onehot_encoded_array = np.array(y_test_onehot_encoded)

#run tf session
train_losses, val_losses = [], []
train_accuracies, val_accuracies = [], []
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for eidx in range(NUM_EPOCHS):
epoch_acc, epoch_loss = [], []
for bidx in range(x_train_data.shape[0]// BATCH_SIZE):
xs, ys = next_batch(BATCH_SIZE, x_train_data, y_train.ravel())
xs = xs.astype(np.float32)
_, train_loss, train_acc= sess.run([opt,loss,accuracy], feed_dict={X: xs,Y: ys})
if (bidx+1)%100 == 0: # print result every 100 batch
print('epoch {} training batch {} loss {} accu {}'.format(eidx +1 , bidx +1, train_loss, train_acc))
epoch_acc.append(train_acc)
epoch_loss.append(train_loss)
print('##################################')
val_acc, val_loss = sess.run([accuracy, loss],
feed_dict= {X:x_test_data, Y: y_test_onehot_encoded_array})
print('epoch {} # test accuracy {} $ test loss {}'.format(eidx +1, val_acc, val_loss ))
print('##################################') 
# Let keep epoch level values for plotting
train_losses.append(np.mean(epoch_loss))
train_accuracies.append(np.mean(epoch_acc))
val_losses.append(val_loss)
val_accuracies.append(val_acc)

我每个纪元的输出：

epoch 1679 training batch 100 loss nan accu 0.009999999776482582
epoch 1679 training batch 200 loss nan accu 0.0
epoch 1679 training batch 300 loss nan accu 0.019999999552965164
epoch 1679 training batch 400 loss nan accu 0.0
epoch 1679 training batch 500 loss nan accu 0.009999999776482582
##################################
epoch 1679 # test accuracy 0.009999999776482582 $ test loss nan

Answer 1

你的损失会'nan'，这是因为你的损失函数不鲁棒，即当Y_pred_prob为零时，它会-inf。 您可以像这样更改它：

#create loss function (cross entropy)
epsilon = 1e-16
loss = -tf.reduce_mean(Y * tf.log(Y_pred_prob + epsilon))

那应该可以！