在使用迁移学习技术对一些图像进行分类训练机器学习模型后，我正在尝试运行Grad CAM。我使用的模型是MobilenetV2。我正在添加一个分类器部分。我已经完成了模型训练和微调部分，模型本身运行得很好。
然而，当我试图运行Grad CAM来生成热图时，它会给我一个错误消息，如下所示：
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.
以下是型号摘要：
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 200, 320, 3)] 0
mobilenetv2_1.00_224 (Funct (None, 7, 10, 1280) 2257984
ional)
flatten (Flatten) (None, 89600) 0
dense (Dense) (None, 6) 537606
=================================================================
Total params: 2,795,590
Trainable params: 537,606
Non-trainable params: 2,257,984
_________________________________________________________________
None
我使用以下代码来查看MobileNetV2的层，并获得我在GradCAM函数中需要提到的最后一个卷积层的名称：
model.get_layer('mobilenetv2_1.00_224').summary()
def make_gradcam_heatmap(img_array, model, base_model_name,last_conv_layer_name):
# First, we create a model that maps the input image to the activations
# of the last conv layer as well as the output predictions
grad_model = tf.keras.models.Model(
[model.inputs], [model.get_layer(base_model_name).get_layer(last_conv_layer_name).output, model.output]
)
# Then, we compute the gradient of the top predicted class for our input image
# with respect to the activations of the last conv layer
with tf.GradientTape() as tape:
last_conv_layer_output, preds = grad_model(img_array)
class_channel = preds[:, 0]
#print('preds: ',preds)
#print('last conv layer output:n',last_conv_layer_output)
#print('class channel:n',class_channel)
# This is the gradient of the output neuron (top predicted or chosen)
# with regard to the output feature map of the last conv layer
grads = tape.gradient(class_channel, last_conv_layer_output)
#print('grads:n',grads)
# This is a vector where each entry is the mean intensity of the gradient
# over a specific feature map channel
pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
# We multiply each channel in the feature map array
# by "how important this channel is" with regard to the top predicted class
# then sum all the channels to obtain the heatmap class activation
last_conv_layer_output = last_conv_layer_output[0]
heatmap = last_conv_layer_output @ pooled_grads[..., tf.newaxis]
heatmap = tf.squeeze(heatmap)
# For visualization purpose, we will also normalize the heatmap between 0 & 1
heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)
heatmap = heatmap.numpy()
heatmap_resized = resize(heatmap,(200, 320))
return heatmap_resized
model.layers[-1].activation = None
plt.figure(figsize = (6,12),dpi = 450)
for ii in range(6):
# Generate class activation heatmap
heatmap = make_gradcam_heatmap(X_test_pp[ii][np.newaxis,:,:], model, "mobilenetv2_1.00_224","Conv_1")
plt.subplot(6,4,ii+1)
plt.imshow(X_test_pp[ii])
plt.title('Label: '+class_names[Y_test[ii]]+'| Pred: '+class_names[np.argmax(Y_pred,axis=1)[ii]],fontdict = {'fontsize' : 4})
plt.imshow(heatmap, alpha = 0.35, cmap = "inferno")
#plt.colorbar()
plt.axis("off")
plt.show()
错误消息的详细信息：
ValueError Traceback (most recent call last)
~AppDataLocalTemp/ipykernel_5016/4168142392.py in <module>
8 for ii in range(6):
9 # Generate class activation heatmap
---> 10 heatmap = make_gradcam_heatmap(X_test_pp[ii][np.newaxis,:,:], model, "mobilenetv2_1.00_224","Conv_1")
11 plt.subplot(6,4,ii+1)
12 plt.imshow(X_test_pp[ii])
~AppDataLocalTemp/ipykernel_5016/3660858513.py in make_gradcam_heatmap(img_array, model, base_model_name, last_conv_layer_name)
22 # This is a vector where each entry is the mean intensity of the gradient
23 # over a specific feature map channel
---> 24 pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
25
26 # We multiply each channel in the feature map array
C:ProgramDataAnaconda3libsite-packagestensorflowpythonutiltraceback_utils.py in error_handler(*args, **kwargs)
151 except Exception as e:
152 filtered_tb = _process_traceback_frames(e.__traceback__)
--> 153 raise e.with_traceback(filtered_tb) from None
154 finally:
155 del filtered_tb
C:ProgramDataAnaconda3libsite-packagestensorflowpythonframeworkconstant_op.py in convert_to_eager_tensor(value, ctx, dtype)
100 dtype = dtypes.as_dtype(dtype).as_datatype_enum
101 ctx.ensure_initialized()
--> 102 return ops.EagerTensor(value, ctx.device_name, dtype)
103
104
ValueError: Attempt to convert a value (None) with an unsupported type (<class 'NoneType'>) to a Tensor.
<Figure size 2700x5400 with 0 Axes>
以下是我的问题，首先，我给Grad CAM的最后一个卷积层的名称正确吗？如果不是，应该是什么？当我自己定义整个模型结构时，出现这个错误消息的原因是什么？
到目前为止我做了什么：
我试着调试代码，看看这个错误消息背后的原因是什么。我意识到下面的行返回"；none"；作为一个梯度，但我不确定为什么？以及我如何解决这个问题。
grads = tape.gradient(class_channel, last_conv_layer_output)
我也为最后一个卷积层尝试了不同的层名称，但效果不太好。
我读了一些关于这个问题的文章，注意到有些人提到所有的图书馆都需要从tensorflow导入，而不是直接从keras导入。这是我要导入的：
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.model_selection import StratifiedShuffleSplit
from sklearn.model_selection import train_test_split
import tensorflow_datasets as tfds
import glob
from skimage import transform
import cv2
import seaborn as sns
from skimage.transform import resize
from sklearn.metrics import confusion_matrix
from tensorflow.keras import backend as K
提前感谢大家。
您的整个cnn模型保存在一个层中mobilenetv2_1.00_224-cnn型号
model = model.layers[1]
在加载模型时，指定输入形状以避免将模型保存在单层中，如下所示。
input = Input(shape=(640, 640, 3))
model = VGG16(include_top=False,input_tensor=input)