如何在seq2seq_model的注意力解码器中获取注意力值来绘制bleu分数

Question

我正在研究一个语言翻译模型。
1. 我想使用bleu score将http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/中提到的数据可视化。

2.  
for a in xrange(num_heads):
    with variable_scope.variable_scope("Attention_%d" % a):
      y = linear(query, attention_vec_size, True)
      y = array_ops.reshape(y, [-1, 1, 1, attention_vec_size])
      # Attention mask is a softmax of v^T * tanh(...).
      s = math_ops.reduce_sum(
          v[a] * math_ops.tanh(hidden_features[a] + y), [2, 3])
      a = nn_ops.softmax(s)
      # Now calculate the attention-weighted vector d.
      d = math_ops.reduce_sum(
          array_ops.reshape(a, [-1, attn_length, 1, 1]) * hidden,
          [1, 2])
      ds.append(array_ops.reshape(d, [-1, attn_size]))
  return ds

如何修改代码来检索可视化的"a"值?

Answer 1

首先需要将对这些张量的引用保存在python列表中。然后将python列表传递给session.run函数。结果将是一个包含这些张量numpy值的列表。