我想在Tensorflow中编写一个自定义层，在其中我需要保存该层以前的权重(因此对于最后n个epoch中的每一个(。在自定义图层中，我将在哪里执行此操作？
例如，这就是tensorflow示例中自定义层的样子：
class MyDenseLayer(tf.keras.layers.Layer):
def __init__(self, num_outputs):
super(MyDenseLayer, self).__init__()
self.num_outputs = num_outputs
def build(self, input_shape):
self.kernel = self.add_weight("kernel",
shape=[int(input_shape[-1]),
self.num_outputs])
def call(self, input):
return tf.matmul(input, self.kernel)
以下是我的方法：
1-使用ModelCheckpoint回调
在这里我创建了一个前馈神经网络。然后，我使用tf.keras.callbacks.ModelCheckpoint在每个历元上保存模型。最后，我加载一个保存的模型并访问其权重。
-首先，让我们创建一个简单的前馈神经网络(当然，你可以使用任何其他层或模型(：
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras import Sequential
from tensorflow.keras.callbacks import ModelCheckpoint
from sklearn.datasets import make_blobs
from tensorflow.keras.utils import to_categorical
model = Sequential()
model.add(Dense(units=3 , input_dim=5 , activation='relu', name='Dense_1'))
model.add(Dense(units=2 , activation='softmax', name='Dense_2'))
model.summary()
-以下是摘要输出：
Model: "sequential_6"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
Dense_1 (Dense) (None, 3) 18
_________________________________________________________________
Dense_2 (Dense) (None, 2) 8
=================================================================
Total params: 26
Trainable params: 26
Non-trainable params: 0
_________________________________________________________________
-创建用于训练模型的伪数据：
train_x , train_y = make_blobs(n_samples=1000, centers=2, n_features=5)
train_y = to_categorical(train_y,2)
-编译和训练模型：
LOG_DIRECTORY = './stackoverflow/'
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model_checkpoint = ModelCheckpoint(LOG_DIRECTORY+'weights{epoch:03d}.h5',
save_freq='epoch',
verbose=1)
model.fit(train_x, train_y,
batch_size=32,
epochs=100,
verbose=1,
callbacks=[model_checkpoint]
)
-让我们看看fit的一些输出：
Epoch 1/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 00001: saving model to ./stackoverflow/weights001.h5
Epoch 2/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 00002: saving model to ./stackoverflow/weights002.h5
Epoch 3/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 00003: saving model to ./stackoverflow/weights003.h5
Epoch 4/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 00004: saving model to ./stackoverflow/weights004.h5
正如您在上面的输出中看到的，模型在每个epoch之后都会使用model_checkpoint = ModelCheckpoint ...进行保存。
-加载保存的模型：
from tensorflow.keras.models import load_model
saved_model =load_model(LOG_DIRECTORY+'weights097.h5')
-打印层的重量：
print(saved_model.layers[0].weights)
在您的情况下，请使用所需图层的索引。
-输出：
输出是权重和偏差的tf.Variable列表
[<tf.Variable 'Dense_1/kernel:0' shape=(5, 3) dtype=float32, numpy=
array([[ 0.44274166, -0.46638554, -0.40543374],
[-0.81307524, -0.43660507, -0.51048666],
[-0.69864446, 0.37800577, -0.06189097],
[-0.12871675, 0.36555207, 0.6326951 ],
[ 0.13829602, 0.56905323, 0.09383805]], dtype=float32)>,
<tf.Variable 'Dense_1/bias:0' shape=(3,) dtype=float32, numpy=array([-0.02371155, -0.06548308, 0.17505823], dtype=float32)>]
-如果你想把它们放在numpy.array表格中：
print(saved_model.layers[0].kernel.numpy())
print(saved_model.layers[0].bias.numpy())
-输出：
array([[ 0.44274166, -0.46638554, -0.40543374],
[-0.81307524, -0.43660507, -0.51048666],
[-0.69864446, 0.37800577, -0.06189097],
[-0.12871675, 0.36555207, 0.6326951 ],
[ 0.13829602, 0.56905323, 0.09383805]], dtype=float32)
array([-0.02371155, -0.06548308, 0.17505823], dtype=float32)
在这种情况下，我们保存整个模型。但是，如果只想保存一个层的权重，可以为此创建一个自定义的callback。
2-使用自定义回调
我通过继承tensorflow.keras.callbacks.Callback创建了一个回调。目前它所做的唯一一件事是打印一个层的权重，您还可以添加用pickle或numpy保存该层权重的代码。
from tensorflow.keras.callbacks import Callback
import pickle
import numpy as np
class CustomCallback(Callback):
def __init__(self, save_path='./logDir', layer_index = 0):
self.save_path = save_path
self.layer_index = layer_index
def on_epoch_end(self, epoch, logs=None):
# access the model weihts
weights_of_first_layer = self.model.layers[self.layer_index].weights
# Do some printing
print(f'nnIn the custom callback, Epoch {epoch}: ')
print(f'First layer weights: n{weights_of_first_layer}')
print('nn')
# get weights in the numpy array format
weights = self.model.layers[self.layer_index].kernel.numpy()
biases = self.model.layers[self.layer_index].bias.numpy()
# Now here you can use numpy or pickle to save the weights
#using pickle
#pickle.dump()
# using numpy
# np.save()
-让我们看看这个回调的作用：
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
custom_callback = CustomCallback()
model.fit(train_x, train_y,
batch_size=32,
epochs=100,
verbose=1,
callbacks=[custom_callback]
)
-一些fit输出：
Epoch 1/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
In the custom callback, Epoch 0:
First layer weights:
[<tf.Variable 'Dense_1/kernel:0' shape=(5, 3) dtype=float32, numpy=
array([[ 0.270913 , -0.52936906, -0.703977 ],
[-0.9254448 , -0.4501195 , -0.6954986 ],
[-0.84005284, 0.34274203, -0.3004068 ],
[-0.24770388, 0.34638566, 0.43633664],
[ 0.2538888 , 0.5864706 , 0.28424913]], dtype=float32)>, <tf.Variable 'Dense_1/bias:0' shape=(3,) dtype=float32, numpy=array([ 0.02709604, -0.07876156, 0.2504825 ], dtype=float32)>]
Epoch 2/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
In the custom callback, Epoch 1:
First layer weights:
[<tf.Variable 'Dense_1/kernel:0' shape=(5, 3) dtype=float32, numpy=
array([[ 0.27090982, -0.52937293, -0.703984 ],
[-0.92544633, -0.4501213 , -0.6955019 ],
[-0.84005505, 0.3427393 , -0.3004116 ],
[-0.24770552, 0.3463836 , 0.43633294],
[ 0.25389025, 0.5864727 , 0.28425238]], dtype=float32)>, <tf.Variable 'Dense_1/bias:0' shape=(3,) dtype=float32, numpy=array([ 0.02709637, -0.07876115, 0.2504833 ], dtype=float32)>]
Epoch 3/100
32/32 [==============================] - 0s 1ms/step - loss: 0.0000e+00 - accuracy: 1.0000
一些有用的文档：
编写自己的回调
将数组保存到NumPy.npy格式的二进制文件中
如何使用pickle在Python 中保存和加载变量
tf.keras.callbacks.ModelCheckpoint
call()方法是为每个批次调用的。
在__init__()中添加计数器并在call():中保存
class MyDenseLayer(tf.keras.layers.Layer):
def __init__(self, num_outputs):
super(MyDenseLayer, self).__init__()
self.num_outputs = num_outputs
self.counter = 0
def call(self, input):
self.counter += 1
if self.counter % batches_per_epoch == 0:
# add saving here
return tf.matmul(input, self.kernel)