为EfficientNetB3体系结构制作类激活映射(CAM)

Question

我想为在EfficientNet B3上构建的模型绘制一个类激活映射。但是，当我从不同的来源学习不同的教程和代码时，它就会失败。。。。

#load images
img = tf.keras.preprocessing.image.load_img(
base, target_size=(img_height, img_width))
img_array = tf.keras.preprocessing.image.img_to_array(img)
img_array = tf.expand_dims(img_array, 0) # Create a batch
predictions = model.predict(img_array)
score = tf.nn.softmax(predictions[0])

last_conv = model.layers[2].layers[-3]
grad_model = tf.keras.models.Model(
[model.inputs], [last_conv.output, model.output])

无法构建grad_model

ValueError：图已断开连接：无法获取张量的值KerasSensor(type_spec=TensorSpec(shape=(无，300300，3(，dtype=tf.float32，name='put_1'(，name='input_1'，description＝"；由层"input_1"创建(在层"处；stem_ cv"；。这个以下先前的层被访问而没有问题：[]

这是型号：

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
sequential (Sequential)      (None, 300, 300, 3)       0         
_________________________________________________________________
rescaling (Rescaling)        (None, 300, 300, 3)       0         
_________________________________________________________________
efficientnet-b3 (Functional) (None, 10, 10, 1536)      10783528  
_________________________________________________________________
global_average_pooling2d (Gl (None, 1536)              0         
_________________________________________________________________
dropout (Dropout)            (None, 1536)              0         
_________________________________________________________________
dense (Dense)                (None, 128)               196736    
_________________________________________________________________
dense_1 (Dense)              (None, 5)                 645       
=================================================================

Answer 1

要解决图形断开值错误，需要正确构建渐变凸轮模型。以下是为渐变凸轮建立模型的方法之一。

import tensorflow as tf 
from tensorflow import keras
from tensorflow.keras import layers
inputs = tf.keras.Input(shape=(300, 300, 3))
x = keras.applications.EfficientNetB3(
input_tensor=inputs, # pass input to input_tensor 
include_top=False,
weights=None
)
# flat the base model with x.output 
x = layers.GlobalAveragePooling2D()(x.output) 
# others 
x = layers.Dense(128)(x)
x = layers.Dropout(0.5)(x)
x = layers.Dense(5)(x)
model = keras.Model(inputs, x)
for i, layer in enumerate(model.layers[-10:]):
print(i, layer.name, layer.output_shape, layer.trainable)

0 block7b_project_bn (None, 10, 10, 384) True
1 block7b_drop (None, 10, 10, 384) True
2 block7b_add (None, 10, 10, 384) True
3 top_conv (None, 10, 10, 1536) True
4 top_bn (None, 10, 10, 1536) True
5 top_activation (None, 10, 10, 1536) True  # < - We will pick this 2D maps
6 global_average_pooling2d_2 (None, 1536) True
7 dense_2 (None, 128) True
8 dropout (None, 128) True
9 dense_3 (None, 5) True

构建渐变CAM模型

grad_model = keras.models.Model(
[model.inputs], 
[
model.get_layer('top_activation').output, 
model.output
]
)

检查

使用您的设置，您会收到以下代码的断开连接错误。但现在，它不会发生。

import numpy as np 
image = np.random.rand(1, 300, 300, 3).astype(np.float32) 
with tf.GradientTape() as tape:
convOutputs, predictions = grad_model(tf.cast(image, tf.float32))
loss = predictions[:, tf.argmax(predictions[0])]
grads = tape.gradient(loss, convOutputs)
print(grads.shape) 
(1, 10, 10, 1536) # NO DISCONNECTED ERROR

要从你的梯度凸轮模型中获得热图，请检查以下答案和来源作为参考。

• 渐变CAM
• 引导式GradCAM
• 渐变CAM多输出模型
• GradCAM回调
• Swin变压器：GradCAM