如何在经过训练的网络上实现GradCAM - pytorch

Question

我目前正在写我的论文，我正在处理医学图像。我想在模型的结果上添加一些GradCam可视化。我使用的是预训练的effentnet_b0与' features_only=True ' (timm库):

class EfficientNet(torch.nn.Module):
def __init__(self):
super().__init__()
# base model
self.feature_extractor = timm.create_model('efficientnet_b0', pretrained=True, features_only=True)
# Get the number input features that the classifier receive. 
# define the classifier. Note that the feature extractor keep 
self.classification_label = nn.Sequential(
nn.Linear(self.feature_extractor.feature_info.channels()[-1], 1280),
torch.nn.ReLU(True),
torch.nn.Dropout(),
torch.nn.Linear(1280, 1280),
torch.nn.ReLU(True),
torch.nn.Dropout(),
torch.nn.Linear(1280, 4) # the labels are 4 
)

self.classification_reason = nn.Sequential(
nn.Linear(self.feature_extractor.feature_info.channels()[-1], 1280),
torch.nn.ReLU(True),
torch.nn.Dropout(),
torch.nn.Linear(1280, 1280),
torch.nn.ReLU(True),
torch.nn.Dropout(),
torch.nn.Linear(1280, 3) # the reasons are 3
)


self.flat_gap = torch.nn.Sequential(
torch.nn.AdaptiveAvgPool2d(1),
torch.nn.Flatten()
)
def forward(self, x):
features = self.flat_gap(self.feature_extractor(x)[-1])
label = self.classification_label(features)
reason = self.classification_reason(features)

return label, reason

我的模型得到最后一个卷积层的特征，然后将它们转发给两个分类器。一个分类器用于图像的诊断(标签)，一个分类器用于图像质量差(bad_light，模糊，低分辨率)的原因。

在训练我的模型之后，我想加载保存的权重并呈现第二个分类器的GradCam可视化(由于质量原因)。我想要每个原因的热图表示。有人能帮我实现吗?

任何帮助将不胜感激。提前谢谢你:)

Answer 1

你可以在这个GitHub repo中使用GradCAM实现。有一些教程可以帮助你在自己的网络上应用GradCAM。这很简单。https://github.com/jacobgil/pytorch-grad-cam/