如何在 TensorFlow 中的堆叠 LSTM 单元之间添加 dropout 层

Question

我可以创建一个具有两个分层 LSTM 的 RNN 网络，如下所示：

lstm_cell1 = tf.nn.rnn_cell.BasicLSTMCell(50)
lstm_cell2 = tf.nn.rnn_cell.BasicLSTMCell(100)
lstm_net = tf.nn.rnn_cell.MultiRNNCell([lstm_cell1, lstm_cell2])

但是现在我还想在每个 lstm 单元格之后包含 dropout 层。像这样，

tf.nn.rnn_cell.MultiRNNCell([tf.nn.dropout(lstm_cell1, 0.8), tf.nn.dropout(lstm_cell2, 0.8)])

我如何实现这一点？

Answer 1

lstm_cell = tf.contrib.rnn.BasicLSTMCell(n_hidden_units)    
lstm_dropout = tf.contrib.rnn.DropoutWrapper(lstm_cell,input_keep_prob=keep_prob, output_keep_prob=keep_prob)
lstm_layers = tf.contrib.rnn.MultiRNNCell([lstm_dropout]* 2)

Answer 2

这是我的堆叠 LSTM 辍学代码。Mashood Tanveer的答案已经足够好了，但我想补充一点，对于MultiRNNCell，你最好不要使用[cell]*num_layer。这是因为[cell]*num_layer会将一个 LSTM 实例堆叠到一个列表中，这可能会导致维度不匹配。除非你知道它将输出什么维度，否则我建议你使用这样的代码。

[tf.contrib.rnn.BasicLSTMCell(hidden_size) for _ in range(num_layers)]
fw_lstms = []
for _ in range(num_layers):
    cell = tf.contrib.rnn.BasicLSTMCell(hidden_size) 
    cell = tf.nn.rnn_cell.DropoutWrapper(cell, output_keep_prob=1-drop_prob)
    fw_lstms.append(cell)
bw_lstms = []
for _ in range(num_layers):
    cell = tf.contrib.rnn.BasicLSTMCell(hidden_size) 
    cell = tf.nn.rnn_cell.DropoutWrapper(cell, output_keep_prob=1-drop_prob, 
                                         input_keep_prob=1-drop_prob)
    bw_lstms.append(cell)    
#bw_lstms = [tf.contrib.rnn.BasicLSTMCell(hidden_size) for _ in range(num_layers)]
fw_init_state_ls = [lstm.zero_state(batch_size, tf.float32) for lstm in fw_lstms]
bw_init_state_ls = [lstm.zero_state(batch_size, tf.float32) for lstm in bw_lstms]
outputs, final_states_fw, final_states_bw = tf.contrib.rnn.stack_bidirectional_dynamic_rnn(cells_fw = fw_lstms, cells_bw = bw_lstms,
   inputs = inputs,
   initial_states_fw = fw_init_state_ls, 
   initial_states_bw = bw_init_state_ls)
bi_final_state = tf.concat([final_states_fw[-1][1], final_states_bw[-1][1]], 1)