Keras具有共享权值的密集模型

Question

我想在keras中创建一个具有共享权重的Dense模型。我的方法描述如下。我有一个叫做dense的密集网络。dense_x、dense_y和dense_z应共享权值(即使用dense)。然后将这三个网络的输出连接起来并馈送到另一个密集的网络中。

然而，不知何故，这种方法不起作用。如何以正确的方式使用共享权重?

num_nodes = 310
input_tensor_x = Input(shape=(310,))
input_tensor_y = Input(shape=(310,))
input_tensor_z = Input(shape=(310,))
dense = Dense(num_nodes)
dense = BatchNormalization()(dense)
dense = Activation('relu')(dense)
dense = Dropout(0.4)(dense)
dense = Dense(num_nodes // 2)
dense = BatchNormalization()(dense)
dense = Activation('relu')(dense)
dense = Dropout(0.4)(dense)
dense = Dense(num_nodes // 4)
dense = BatchNormalization()(dense)
dense = Activation('relu')(dense)
dense = Dropout(0.4)(dense)
dense_x = dense(input_tensor_x) #<=== shared above dense network
dense_y = dense(input_tensor_y) #<=== shared above dense network
dense_z = dense(input_tensor_z) #<=== shared above dense network
merge_layer = tf.keras.layers.Concatenate()([dense_x, dense_y, dense_z])
merged_nodes = 3*(num_nodes // 4) // 2
dense2 = Dense(merged_nodes)(merge_layer)
dense2 = BatchNormalization()(dense2)
dense2 = Activation('relu')(dense2)
dense2 = Dropout(0.4)(dense2)
dense2 = Dense(merged_nodes // 2)(dense2)
dense2 = BatchNormalization()(dense2)
dense2 = Activation('relu')(dense2)
dense2 = Dropout(0.4)(dense2)
output_tensor = Dense(3, activation='softmax')(dense2)
fcnn_model = Model(inputs=[input_tensor_x, input_tensor_y, input_tensor_z], outputs=output_tensor)
opt = Adam(lr=learning_rate)
fcnn_model.compile(loss='categorical_crossentropy',
optimizer=opt, metrics=['accuracy', tf.keras.metrics.AUC()])

Answer 1

最简单的方法是在顶部初始化共享层，然后将以下层传递给它们。注意不要覆盖它们

我初始化了一个Dense层和一个BatchNormalization层，因为它们是唯一具有可训练权值的层

num_nodes = 310
input_tensor_x = Input(shape=(310,))
input_tensor_y = Input(shape=(310,))
input_tensor_z = Input(shape=(310,))
def shared_dense(inp):

dense = Dense(num_nodes)(inp)
dense = BatchNormalization()(dense)
dense = Activation('relu')(dense)
dense = Dropout(0.4)(dense)
dense = Dense(num_nodes // 2)(dense)
dense = BatchNormalization()(dense)
dense = Activation('relu')(dense)
dense = Dropout(0.4)(dense)
dense = Dense(num_nodes // 4)(dense)
dense = BatchNormalization()(dense)
dense = Activation('relu')(dense)
dense = Dropout(0.4)(dense)
return Model(inp, dense, name='shared_dense')
shared_dense = shared_dense(Input(shape=(310,)))
dense_x = shared_dense(input_tensor_x) #<=== shared 
dense_y = shared_dense(input_tensor_y) #<=== shared 
dense_z = shared_dense(input_tensor_z) #<=== shared 
merge_layer = Concatenate()([dense_x, dense_y, dense_z])
merged_nodes = 3*(num_nodes // 4) // 2
dense2 = Dense(merged_nodes)(merge_layer)
dense2 = BatchNormalization()(dense2)
dense2 = Activation('relu')(dense2)
dense2 = Dropout(0.4)(dense2)
dense2 = Dense(merged_nodes // 2)(dense2)
dense2 = BatchNormalization()(dense2)
dense2 = Activation('relu')(dense2)
dense2 = Dropout(0.4)(dense2)
output_tensor = Dense(3, activation='softmax')(dense2)
fcnn_model = Model(inputs=[input_tensor_x, input_tensor_y, input_tensor_z], 
outputs=output_tensor)
fcnn_model.compile(loss='categorical_crossentropy',
optimizer=Adam(lr=1e-3), 
metrics=['accuracy', tf.keras.metrics.AUC()])