TimeDistributed层应用了几个卷积层的错误

Question

我的tf.keras.layers.TimeDistributed层有问题(https://www.tensorflow.org/api_docs/python/tf/keras/layers/TimeDistributed)。

我知道TimeDistributed可以用于将单层(密集、卷积…(应用于一组输入，从而获得一组输出。

不仅如此，直到最近我还能够使用它来应用整个"；子模型"；所有输入。也就是说，一系列的层，而不仅仅是一层。Patrice Ferlet在这里解释了一个例子(https://medium.com/smileinnovation/training-neural-network-with-image-sequence-an-example-with-video-as-input-c3407f7a0b0f)。

使用该源作为例子；子模型"；像这样：

import keras
from keras.layers import Conv2D, BatchNormalization, 
MaxPool2D, GlobalMaxPool2D
def build_convnet(shape=(112, 112, 3)):
momentum = .9
model = keras.Sequential()
model.add(Conv2D(64, (3,3), input_shape=shape,
padding='same', activation='relu'))
model.add(Conv2D(64, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
model.add(MaxPool2D())
model.add(Conv2D(128, (3,3), padding='same', activation='relu'))
model.add(Conv2D(128, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
model.add(MaxPool2D())
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
model.add(MaxPool2D())
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
# flatten...
model.add(GlobalMaxPool2D())
return model

然后将这个子模型包含在一个高级模型中，该高级模型调用TimeDistributed和整个初始子模型(convnet(。

from keras.layers import TimeDistributed, GRU, Dense, Dropout
def action_model(shape=(5, 112, 112, 3), nbout=3):
# Create our convnet with (112, 112, 3) input shape
convnet = build_convnet(shape[1:])
# then create our final model
model = keras.Sequential()
# add the convnet with (5, 112, 112, 3) shape
model.add(TimeDistributed(convnet, input_shape=shape))
# here, you can also use GRU or LSTM
model.add(GRU(64))
# and finally, we make a decision network
model.add(Dense(1024, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(512, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(64, activation='relu'))
model.add(Dense(nbout, activation='softmax'))
return model

现在这工作得很好，我可以得到调用的模型结构

mod=action_model()
mod.summary()

但是，如果我使用keras的预定义架构(如VGG16(作为主干来定义convnet模型，则似乎存在错误。(我还需要更改keras.Sequential by tf.keras.models.Sequential(

import tensorflow as tf
from keras.layers import Flatten
def build_convnet():
prevModel = tf.keras.applications.vgg16.VGG16(
include_top=False,
input_shape=(112, 112, 3),
weights='imagenet'  # ImageNet weights
)
model = tf.keras.models.Sequential()
model.add(prevModel)
model.add(Flatten())
return model
def action_model(shape=(5, 112, 112, 3), nbout=3):
# Create our convnet with (112, 112, 3) input shape
convnet = build_convnet()
# then create our final model
model = tf.keras.models.Sequential()
# add the convnet with (5, 112, 112, 3) shape
model.add(TimeDistributed(convnet, input_shape=shape))
# here, you can also use GRU or LSTM
model.add(GRU(64))
# and finally, we make a decision network
model.add(Dense(1024, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(512, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(64, activation='relu'))
model.add(Dense(nbout, activation='softmax'))
return model

当我在定义了基于VGG16的体系结构后运行此程序时

mod=action_model()
mod.summary()

我得到以下错误：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-106-c8c6108a1d66> in <module>()
----> 1 mod=action_model()
2 mod.summary()
1 frames
/usr/local/lib/python3.7/dist-packages/keras/layers/wrappers.py in __init__(self, layer, **kwargs)
121           'Please initialize `TimeDistributed` layer with a '
122           '`tf.keras.layers.Layer` instance. You passed: {input}'.format(
--> 123               input=layer))
124     super(TimeDistributed, self).__init__(layer, **kwargs)
125     self.supports_masking = True
ValueError: Please initialize `TimeDistributed` layer with a `tf.keras.layers.Layer` instance. You passed: <tensorflow.python.keras.engine.sequential.Sequential object at 0x7fbb36266a50>

因此，现在python似乎在抱怨我使用的TimeDistributed输入不是单层的。这没有任何意义，因为最初的示例运行良好，并且还涉及使用TimeDistributed的多个层。除此之外，VGG16型号几周前也运行良好。

我正在Google CoLab中运行所有这些。

有人能帮我弄清楚这里发生了什么事吗？这是由新的tensorflow 2.5.0版本引起的吗？无论我走到哪里，我都会看到人们使用TimeDistributed来应用单层，但到目前为止，应用整个顺序模型的效果还不错(尽管文档中没有明显提及(。

谢谢！

Answer 1

您得到以上ValueError是由于混合了tf.keras和keras导入，而TF2.5不支持这一点。

工作代码如下所示

import tensorflow as tf
#from keras.layers import TimeDistributed, GRU, Dense, Dropout
from tensorflow.keras.layers import Flatten, GRU, Dropout, TimeDistributed, Dense
def build_convnet():
prevModel = tf.keras.applications.vgg16.VGG16(
include_top=False,
input_shape=(112, 112, 3),
weights='imagenet'  # ImageNet weights
)
model = tf.keras.models.Sequential()
model.add(prevModel)
model.add(Flatten())
return model
def action_model(shape=(5, 112, 112, 3), nbout=3):
# Create our convnet with (112, 112, 3) input shape
convnet = build_convnet()
# then create our final model
model = tf.keras.models.Sequential()
# add the convnet with (5, 112, 112, 3) shape
model.add(TimeDistributed(convnet, input_shape=shape))
# here, you can also use GRU or LSTM
model.add(GRU(64))
# and finally, we make a decision network
model.add(Dense(1024, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(512, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(64, activation='relu'))
model.add(Dense(nbout, activation='softmax'))
return model
mod=action_model()
mod.summary()

输出：

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
time_distributed_1 (TimeDist (None, 5, 4608)           14714688  
_________________________________________________________________
gru_1 (GRU)                  (None, 64)                897408    
_________________________________________________________________
dense (Dense)                (None, 1024)              66560     
_________________________________________________________________
dropout (Dropout)            (None, 1024)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 512)               524800    
_________________________________________________________________
dropout_1 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 128)               65664     
_________________________________________________________________
dropout_2 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_3 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_4 (Dense)              (None, 3)                 195       
=================================================================
Total params: 16,277,571
Trainable params: 16,277,571
Non-trainable params: 0
_________________________________________________________________