如何将计算出的损失应用于图形?

Question

我是Tensorflow的新手，并试图编写一个玩具鉴别器问题。我的设置方式，损失是根据expert_actions和novice_actions计算的。但是，当我尝试使用计算的损失进行优化时，我遇到了错误。错误是ValueError: No variables to optimize.我确实明白我收到错误是因为没有feed_dict.但是，我不知道解决方案。

class discriminator:
def __init__(self,n_actions, learning_rate):
self.n_actions = n_actions
self.learning_rate_dist = learning_rate
self.graph = tf.Graph()
with self.graph.as_default():
self.dis_input = tf.placeholder(tf.float32, [None, self.n_actions])
self.discriminator_function()
init = tf.global_variables_initializer()
self.sess = tf.Session(graph=self.graph)
self.sess.run(init)
def discriminator_function(self, hidden = None):
if hidden == None:
hidden = 16
x = tf.layers.dense(self.dis_input,hidden,tf.nn.relu)
x = tf.layers.dense(x,hidden,tf.nn.relu)
self.dis_output = tf.layers.dense(x,1)

def discriminator(self,expert_actions,novice_actions):
expert_out = self.sess.run(self.dis_output,feed_dict={self.dis_input : expert_actions})
novice_out = self.sess.run(self.dis_output,feed_dict={self.dis_input : novice_actions})
loss = tf.reduce_mean(tf.log(expert_out) + tf.log(1.-novice_out))
# update discriminator loss
optimize = tf.train.AdamOptimizer(self.learning_rate_dis).minimize(-loss)
self.sess.run(optimize) #error over here
return loss
if __name__ == '__main__':
d = discriminator(2,0.001)
expert_actions = np.random.randint(2, size=10)
novice_actions = np.random.randint(2, size=10)
d.discriminator(expert_actions,novice_actions)

Answer 1

您正在尝试使用expert_out优化loss = tf.reduce_mean(tf.log(expert_out) + tf.log(1.-novice_out))，novice_out是numpy数组。输入和损耗之间没有变量，用于计算梯度。

您的鉴别器函数应该是这样的：

def discriminator(self,expert_actions,novice_actions):
#Make sure you add the new ops and variables to the graph defined.  
with self.graph.as_default(): 
loss = tf.reduce_mean(tf.log('Should be a tensor that is part of the graph and not a numpy array))
optimize = tf.train.AdamOptimizer(0.01).minimize(-loss)
self.sess.run(tf.global_variables_initializer())
#pass the inputs here
loss = self.sess.run([loss, optimize], feed_dict={self.dis_input : expert_actions}) 
return loss