在scikit优化中使用KerasProgsor的示例

Question

我正在使用很棒的scikit优化工具箱进行超参数优化。我的目标是比较keras和scikit学习模型。

根据示例https://scikit-optimize.github.io/stable/auto_examples/sklearn-gridsearchcv-replacement.html#sphx-glr auto示例sklearn-gridsearchcv替换了仅py的scikit学习模型。尝试以下代码不允许在BayesSearchCV中集成keras模式。

# Function to create model, required for KerasRegressor
def create_model(optimizer='rmsprop', init='glorot_uniform'):
# create model
model = Sequential()
model.add(Dense(12, input_dim=8, kernel_initializer=init, activation='relu'))
model.add(Dense(8, kernel_initializer=init, activation='relu'))
model.add(Dense(1, kernel_initializer=init, activation='linear'))
# Compile model
model.compile(loss='mse', optimizer=optimizer, metrics=['r2'])
return model
model = KerasRegressor(build_fn=create_model, verbose=0)
NN_search = {
'model': [model()],
'model__optimizers': optimizers,
'model__epochs' : epochs, 
'model__batch_size' : batches, 
'model__init' : init
}

有人成功地将KerasClassifier/Regressor合并到BayesSearch简历中吗？

Answer 1

好吧，我找到了一个选项来定义基于全局参数构建的模型。因此，在scikit opt最小化函数内部，调用目标函数，在这里设置全局参数，并在create_model_NN函数中使用，该函数建立在keras scikit learn KerasProgsor Wrapper上。

def create_model_NN():
#start the model making process and create our first layer
model = Sequential()
model.add(Dense(num_input_nodes, input_shape=(40,), activation=activation
))
#create a loop making a new dense layer for the amount passed to this model.
#naming the layers helps avoid tensorflow error deep in the stack trace.
for i in range(num_dense_layers):
name = 'layer_dense_{0}'.format(i+1)
model.add(Dense(num_dense_nodes,
activation=activation,
name=name
))
#add our classification layer.
model.add(Dense(1,activation='linear'))

#setup our optimizer and compile
adam = Adam(lr=learning_rate)
model.compile(optimizer=adam, loss='mean_squared_error',
metrics=['mse'])
return model
def objective_NN(**params):
print(params)
global learning_rate
learning_rate=params["learning_rate"]
global num_dense_layers
num_dense_layers=params["num_dense_layers"]
global num_input_nodes
num_input_nodes=params["num_input_nodes"]
global num_dense_nodes
num_dense_nodes=params["num_dense_nodes"]
global activation
activation=params["activation"]

model = KerasRegressor(build_fn=create_model, epochs=100, batch_size=1000, verbose=0)
X_train, X_test, y_train, y_test = train_test_split(X_time, y_time, test_size=0.33, random_state=42)

model.fit(X_train, y_train)

y_pr = model.predict(X_test)

res = metrics.r2_score(y_test, y_pr)
return -res

称之为

res_gp = gp_minimize(objective_NN, space_NN, n_calls=10, random_state=0)