这似乎是一个常见的错误，人们得到，但我不能真正理解真正的原因。我有麻烦弄清楚在哪里内联更换正在发生。我的转发功能:
def forward(self, input, hidden=None):
if hidden is None :
hidden = self.init_hidden(input.size(0))
out, hidden = self.lstm(input, hidden)
out = self.linear(out)
return out, hidden
训练循环
def training(dataloader, iterations, device):
torch.autograd.set_detect_anomaly(True)
model = NModel(662, 322, 2, 1)
hidden = None
model.train()
loss_fn = F.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
running_loss = []
last_loss = 0
for i, (feature, label) in tqdm(enumerate(dataloader)):
optimizer.zero_grad()
outputs, hidden = model(feature, hidden)
loss = loss_fn(outputs, label)
print("loss item" , loss.item())
running_loss.append(loss.item())
loss.backward(retain_graph=True)
optimizer.step()
if i%1000 == 0:
last_loss = len(running_loss) /1000
return last_loss
错误的堆栈跟踪
Traceback (most recent call last):
File "main.py", line 18, in <module>
main()
File "main.py", line 14, in main
training(dataloader=training_loader, iterations=3, device=0)
File "/home//gitclones/feature-extraction/training.py", line 30, in training
loss.backward(retain_graph=True)
File "/home/miniconda3/envs/pytorch-openpose/lib/python3.7/site-packages/torch/_tensor.py", line 307, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home//miniconda3/envs/pytorch-openpose/lib/python3.7/site-packages/torch/autograd/__init__.py", line 156, in backward
allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [322, 1288]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
当我删除optimizer.step()代码运行，但我认为没有反向传播发生。
[编辑]奇怪的是，现在它是如何工作的，当我不传递隐藏状态作为输入在forward pass
def forward(self, input, hidden=None):
if hidden is None :
hidden = self.init_hidden(input.size(0))
out, hidden = self.lstm(input)
out = self.linear(out)
return out, hidden
在optimizer.step()修复错误后添加hidden = tuple([each.data for each in hidden])，但在隐藏值上的梯度为零。您可以使用hidden = tuple([each.detach() for each in hidden])