在执行时是否使用dropout层权重?

Question

我正试图对神经网络的执行机制有一个很好的理解。为此，我正在研究一些张量流示例，并试图实现我自己的执行实现。

在使用MNSET示例时，我发现Dropout(0.2)(这里)层是实际上是训练网结构的一部分。您可以使用model.get_weights()[2]检查它的权重。文档表明Dropout在执行时是一个noop。如果这是真的，权重会被忽略吗?或者训练的权重在执行时是否像常规的Dense层一样使用?

Answer 1

来自这里的文档:

The Dropout layer randomly sets input units to 0 with a frequency of rate at each step during training time, which helps prevent overfitting. Inputs not set to 0 are scaled up by 1/(1 - rate) such that the sum over all inputs is unchanged.
Note that the Dropout layer only applies when training is set to True 
such that no values are dropped during inference. 
When using model.fit, training will be appropriately set to True 
automatically, and in other contexts, you can set the 
kwarg explicitly to True when calling the layer.

Answer 2

我相信我知道我在看什么了。

model.get_weights()[2]为非dropout层权值。

每个层实际上在get_weights列表中有两个条目，边权重和层偏差向量。

get_weights返回的结构体的维度为:

784 x128128128年x1010

与"edge_weights + biases"结论