如果相同数据的测试准确性波动太大，这意味着什么?

Question

我正在为我自己的数据集运行张量流教程中提供的基本分类代码，有时我的测试精度为 1.0，有时为 0。所以我不明白当我真的没有改变任何东西时，结果怎么会如此极端。我只是多次编译相同的python代码

import scipy.io
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
mat1 = scipy.io.loadmat('matlab7.mat');
mat2 =  scipy.io.loadmat('matlab8.mat');
Training = mat1['Training7'];
Testing = mat2['Testing8'];
Class_Training = mat2['Class_Training8'];
Class_Testing = mat2['Class_Testing8'];

Training = np.ascontiguousarray(Training.T) ;
Testing = np.ascontiguousarray(Testing.T) ;
model = keras.Sequential([
keras.layers.Flatten(input_shape=(14, 98)),
keras.layers.Dense(128, activation=tf.nn.relu),
keras.layers.Dense(2, activation=tf.nn.softmax)
])

model.compile(optimizer=tf.train.AdamOptimizer(),
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])

model.fit(Training, Class_Training, epochs=100)

test_loss, test_acc = model.evaluate(Testing, Class_Testing)
print('Test accuracy:', test_acc)

来自训练集的一些数据

Answer 1

您应该检查两件事：您的数据是否正确加载？您的网络是否正常工作？

要检查数据是否已正确加载，您应该打印一些示例示例并将其与电子表格进行比较。您还应该确保数据中没有 NaN。您可以使用 numpy 执行此操作：如果数组具有 NaN 值，np.any(np.is_nan(array))将返回 true。

要检查您的网络是否正常工作，您应该使用玩具数据集对其进行测试。下面的代码示例训练网络区分从均匀分布和正态分布中采样的向量。该模型应该能够在 100 个 epoch 后完美地过度拟合训练数据，并且它应该具有非常高的测试准确率(我得到了 0.96(。请注意，代码在开始时设置 TensorFlow 随机种子，以确保权重始终以相同的方式初始化。

import numpy as np
import tensorflow as tf
from tensorflow import keras as k
# Make sure weight initialization is always the same
tf.set_random_seed(0)
n_train = 100
n_test = 50
n_dim = 10
train_size = (n_train, n_dim)
test_size = (n_test, n_dim)
rand = np.random.RandomState(2)
# The training data consists of vectors sampled from a uniform distribution 
# (label 0) and vectors sampled from a normal distribution (label 1).
X_train = np.concatenate([rand.uniform(size=train_size), rand.normal(size=train_size)])
Y_train = np.concatenate([np.zeros((n_train, 1)), np.ones((n_train, 1))])
X_test = np.concatenate([rand.uniform(size=test_size), rand.normal(size=test_size)])
Y_test = np.concatenate([np.zeros((n_test, 1)), np.ones((n_test, 1))])                    
model = k.Sequential([
k.layers.Flatten(input_shape=(n_dim,)),
k.layers.Dense(128, activation=tf.nn.relu),
k.layers.Dense(2, activation=tf.nn.softmax)
])
model.compile(optimizer=tf.train.AdamOptimizer(), 
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(X_train, Y_train, epochs=100)
test_loss, test_acc = model.evaluate(X_test, Y_test)
print('Test accuracy:', test_acc)

希望这有帮助！