使用 LSTM 有状态传递上下文黑白批处理;上下文传递中可能存在一些错误，没有得到好的结果?

Question

我在将数据提供给网络之前已经检查了数据。数据是正确的。

使用 LSTM 并传递上下文黑白批处理。 per_class_accuracy正在发生变化，但损失并没有下降。卡了很久，不确定代码中是否有错误？

我有基于不平衡数据集的多类分类问题

Dataset_type：CSV

Dataset_size： 20000

基于传感器的CSV数据

X =

0.69861111111111111,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0Y = 离开屋

每类精度： {'leaveHouse'： 0.34932855， 'getDressed'： 1.0， 'idle'： 0.8074534， 'prepareBreakfast'： 0.8， 'goToBed'： 0.35583413， 'getDrink'： 0.0， 'takeShower'： 1.0， 'useToilet'： 0.0， 'eatBreakfast'： 0.8857143}

训练：

# Using loss weights, the inverse of class frequency
criterion = nn.CrossEntropyLoss(weight = class_weights)
hn, cn = model.init_hidden(batch_size)
for i, (input, label) in enumerate(trainLoader):
hn.detach_()
cn.detach_()
input = input.view(-1, seq_dim, input_dim)
if torch.cuda.is_available():
input = input.float().cuda()
label = label.cuda()
else:
input = input.float()
label = label
# Forward pass to get output/logits
output, (hn, cn) = model((input, (hn, cn)))
# Calculate Loss: softmax --> cross entropy loss
loss = criterion(output, label)#weig pram
running_loss += loss
loss.backward()  # Backward pass
optimizer.step()  # Now we can do an optimizer step
optimizer.zero_grad()  # Reset gradients tensors

网络

class LSTMModel(nn.Module):
def init_hidden(self, batch_size):
self.batch_size = batch_size
if torch.cuda.is_available():
hn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim).cuda()
# Initialize cell state
cn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim).cuda()
else:
hn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim)
# Initialize cell state
cn = torch.zeros(self.layer_dim, self.batch_size, self.hidden_dim)
return hn, cn
def __init__(self, input_dim, hidden_dim, layer_dim, output_dim, seq_dim):
super(LSTMModel, self).__init__()
# Hidden dimensions
self.hidden_dim = hidden_dim
# Number of hidden layers
self.layer_dim = layer_dim
self.input_dim = input_dim
# Building your LSTM
# batch_first=True causes input/output tensors to be of shape
# (batch_dim, seq_dim, feature_dim)
self.lstm = nn.LSTM(self.input_dim, hidden_dim, layer_dim, batch_first=True)
# Readout layer
self.fc = nn.Linear(hidden_dim, output_dim)
self.relu = nn.ReLU()
self.softmax = nn.Softmax(dim=1)
self.seq_dim = seq_dim
def forward(self, inputs):
# Initialize hidden state with zeros
input, (hn, cn) = inputs
input = input.view(-1, self.seq_dim, self.input_dim)
# time steps
out, (hn, cn) = self.lstm(input, (hn, cn))
# Index hidden state of last time step
out = self.fc(out[:, -1, :])
out = self.softmax(out)
return out, (hn,cn)

Answer 1

您可能遇到的一个问题是CrossEntropyLoss将对数 softmax 操作与负对数似然损失相结合，但您在模型中应用 softmax。您应该将原始日志从最后一层传递到CrossEntropyLoss。

我也不能说没有看到模型向前传递，但看起来您正在将维度 1 上的 softmax 应用于(我推断(形状为batch_size, sequence_length, output_dim的张量，而您应该沿输出暗淡应用它。