我想用pyomo从行为数据集中估计RL模型的参数。
#dummy data
dis_data = pd.DataFrame([0,1,0,0,0,1], columns=['reward'])
dis_data['Expt']=str(1)
dis_data = dis_data.set_index('Expt')
def max_likelihood_four(x,data):
lr=x.lr
sigma=x.sigma
time = data.shape[0]
v = np.zeros(([2, time]))
pr = np.zeros(([2, time]))+0.5
pr_log = np.zeros(([time]))
for t in range(time-1):
pr[1, t] = 1 / (1 + np.exp(-(v[1, t] - v[0, t])/ sigma))
pr[0, t] = 1-pr[1, t]
outcome=int(data.ix[t,'reward'])
v[choice, t + 1] = v[choice, t] + lr * (outcome - v[choice, t])
v[1 - choice, t + 1] = v[1 - choice, t]
pr_log[t] = np.log(pr[choice, t])
# print(i)
return -np.sum(pr_log)
def pyomo_fit1(df):
mdl = ConcreteModel()
mdl.lr = Var(initialize=np.random.random(1), bounds=(0, 1))
mdl.sigma = Var(initialize=np.random.random(1), bounds=(0, 10))
residuals = max_likelihood_four(mdl,df)
mdl.obj = Objective(expr=residulas, sense=minimize)
SolverFactory('ipopt').solve(mdl)
return [mdl.lr,mdl.sigma]
parameter_values_1, r1 = pyomo_fit1(dis_data)
我得到这个错误:
"不能将标量分量"sigma"视为索引分量";
np.random.random(1)
返回一个numpy.ndarray
,而不是整数或浮点值。当您尝试将mdl.sigma
初始化为标量pyomo变量时,需要使用标量值,或者在本例中使用np.random.random(1)[0]
(数组中唯一的元素(。
如果您将mdl.sigma
(以及mdl.lr
(的代码替换为:
mdl.sigma = Var(initialize=np.random.random(1)[0], bounds=(0, 10))
则错误("不能将标量分量‘sigma’视为索引分量"(错误应该消失。
mdl = ConcreteModel()
mdl.lr = Var(initialize=np.random.random(1)[0], bounds=(0, 1))
mdl.sigma = Var(initialize=np.random.random(1)[0], bounds=(0, 10))