tf_agents-dqn无法初始化

Question

即使tf.agents initialize((不需要输入变量，这一行

agent.initialize()

产生此错误

TypeError: initialize() missing 1 required positional argument: 'self'

我尝试了agent。initialize(agent(，因为它显然想要自我传递…显然这不起作用XD

我怀疑问题可能是这条线路

print(type(agent))   

生成

<class 'abc.ABCMeta'>

但这可能很正常。。。

##################################

我下面的整个脚本是可复制的

###  for 9 by 9 connect 4 board 
#
import tensorflow as tf
from tf_agents.networks import q_network
from tf_agents.agents.dqn import dqn_agent
import tf_agents
import numpy as np
print(tf.__version__)
print(tf_agents.__version__)
import tensorflow.keras

observation_spec  = tf.TensorSpec(   #   observation tensor = the whole board , ideally 0's, 1's , 2's for empty, occupied by player 1 , occupied by player 2
[9,9],
dtype=tf.dtypes.float32,
name=None
)
action_spec  = tf_agents.specs.BoundedArraySpec(    
[1],                         ### tf_agents.networks.q_network only seems to take an action of size 1  
dtype= type(1) ,     #tf.dtypes.float64,
name=None, 
minimum=0,
maximum=2
)
#######################################
def make_tut_layer(size):
return tf.keras.layers.Dense(
units= size,
activation= tf.keras.activations.relu,
kernel_initializer=tf.keras.initializers.RandomNormal(mean=0., stddev=1.)
)
def make_q_layer(num_actions):
q_values_layer = tf.keras.layers.Dense (        # last layer gives probability distribution over all actions so we can pick best action 
num_actions ,
activation = tf.keras.activations.relu , 
kernel_initializer = tf.keras.initializers.RandomUniform( minval = 0.03 , maxval = 0.03),
bias_initializer = tf.keras.initializers.Constant(-0.2)
) 
return q_values_layer;

############################## stick together layers below
normal_layers = []
for i in range(3):
normal_layers.append(make_tut_layer(81))
q_layer = make_q_layer(9)
q_net = keras.Sequential(normal_layers + [q_layer])
######################################
agent = dqn_agent.DqnAgent
(
observation_spec,    ### bonus question, why do i get syntax errors when i try to label variables like ---> time_step_spec = observation_spec, gives me SyntaxError: invalid syntax   on the = symbol       
action_spec,
q_net,
tf.keras.optimizers.Adam(learning_rate= 0.001 )
)
eval1 = agent.policy
print(eval1)
eval2= agent.collect_policy
print(eval2)
print(type(agent))  
agent.initialize()
print(" done ")

并产生输出。

2.9.2
0.13.0
<property object at 0x000001A13268DA90>
<property object at 0x000001A13268DAE0>
<class 'abc.ABCMeta'>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [53], in <cell line: 73>()
71 print(eval2)
72 print(type(agent))
---> 73 agent.initialize()
74 print(" done ")
TypeError: initialize() missing 1 required positional argument: 'self'

我的代理人打字可以吗？应该是<类"abc.ABCMeta">

为什么我的代理无法初始化？

Answer 1

我想，答案很简单：不能只将(移动到函数调用的下一行。

你正在有效地做什么：

使agent成为dqn_agent.DqnAgent(类(的别名

agent = dqn_agent.DqnAgent

计算表达式并丢弃其结果

(
observation_spec,
action_spec,
q_net,
tf.keras.optimizers.Adam(learning_rate= 0.001 )
)

这也回答了额外的问题&因为它不是一个函数调用，所以没有命名参数，表达式中也不允许赋值(python就是这么说的(。

将开口支架放在dqn_agent.DqnAgent的正后方，它应该工作：

agent = dqn_agent.DqnAgent(
observation_spec,
action_spec,
q_net,
tf.keras.optimizers.Adam(learning_rate= 0.001 )
)