如何使用不同的优化器层在tensorflow?

Question

我有三个密集的图层。我想看一个例子，其中不同的优化器用于这三个层(例如，RMSProp, Adadelta, Adam)。
像这样:

tf.keras.layers.Dense(250, ..., optimizer='rmsprop')
tf.keras.layers.Dense(250, ..., optimizer=tf.keras.optimizers.Adadelta(learning_rate=1))
tf.keras.layers.Dense(10,  ..., optimizer='adam')

Answer 1

解决方案:

import tensorflow_addons as tfa
# ...
optimizers = [
tf.keras.optimizers.RMSprop(),
tf.keras.optimizers.Adadelta(learning_rate=5),
tf.keras.optimizers.Adam()
]
optimizers_and_layers = [(optimizers[0], model.layers[0]),
(optimizers[1], model.layers[1]),
(optimizers[2], model.layers[2])]
optimizer = tfa.optimizers.MultiOptimizer(optimizers_and_layers)
model.compile(optimizer=optimizer, ...)

For saving -save_weights_only=True:

checkpoint_path_best = "best.h5"
modelcheckpoint_best = tf.keras.callbacks.ModelCheckpoint(checkpoint_path_best, monitor='val_accuracy', save_best_only=True, mode='max', save_weights_only=True)
model.fit(train_data, train_labels, ..., callbacks=[modelcheckpoint_best])

和加载:

model.load_weights('best.h5')