Keras：模仿 PyTorch 的 conv2d 和线性/密集权重初始化？

Question

我正在将一个模型从PyTorch移植到Keras/Tensorflow，我想确保我使用相同的算法进行权重初始化。我如何模仿PyTorch的权重初始化在Keras?

Answer 1

如果重构PyTorch初始化代码，您会发现权重初始化算法非常简单。代码中的注释是正确的;读一下这条评论，然后模仿一下。

下面是Keras/Tensorflow的模拟代码:

import tensorflow as tf
from tensorflow.keras import layers
class PytorchInitialization(tf.keras.initializers.VarianceScaling):
def __init__(self, seed=None):
super().__init__(
scale=1 / 3, mode='fan_in', distribution='uniform', seed=seed)
# Conv layer
conv = layers.Conv2D(32, 3, activation="relu", padding="SAME",
input_shape=(28, 28, 1),
kernel_initializer=PytorchInitialization(),
bias_initializer=PytorchInitialization())
# Dense / linear layer
classifier = layers.Dense(10,
kernel_initializer=PytorchInitialization(),
bias_initializer=PytorchInitialization(),