在测试数据加载器中创建一个大文件名字典，并将其中所有512x512补丁的预测分配为其值的列表

Question

我不知道为什么按照以下方式制作字典并不能创建所需的输出。我最终得到的不是一本有887个大文件名的词典，而是一本只有2个大文件号的词典。

快速介绍我的测试集。我有大的图像，我已经将它们平铺到512x512补丁中。下面你可以看到大量的大图像和每个正标签和负标签的512x512补丁：

--test
---pos_label 14, 11051
---neg_label 74, 45230

sample_fnames_labels = dataloaders_dict['test'].dataset.samples
test_large_images = {}
test_loss = 0.0
test_acc = 0

with torch.no_grad():

test_running_loss = 0.0
test_running_corrects = 0
print(len(dataloaders_dict['test']))
for i, (inputs, labels) in enumerate(dataloaders_dict['test']):

patch_name = sample_fname.split('/')[-1]
large_image_name = patch_name.split('_')[0]
test_inputs = inputs.to(device)
test_labels = labels.to(device)

test_outputs = saved_model_ft(test_inputs)


_, test_preds = torch.max(test_outputs, 1)

max_bs = len(test_preds)

for j in range(max_bs):

sample_file_name = sample_fnames_labels[i+j][0]
patch_name = sample_file_name.split('/')[-1]
large_image_name = patch_name.split('_')[0]

if large_image_name not in test_large_images.keys():

test_large_images[large_image_name] = list()
test_large_images[large_image_name].append(test_preds[j].item())

else:
test_large_images[large_image_name].append(test_preds[j].item())




#test_running_loss += test_loss.item() * test_inputs.size(0)
test_running_corrects += torch.sum(test_preds == test_labels.data)

#test_loss = test_running_loss / len(dataloaders_dict['test'].dataset)
test_acc = test_running_corrects / len(dataloaders_dict['test'].dataset)

这里testlarge_images字典只有两个大图像作为关键字，而不是88个测试大图像。谢谢你看。

本质上，我想将每个大图像的512x512个补丁的所有标签作为一个列表收集到一个以large_image_filename为关键字的字典中。所以，我可以稍后进行多数投票。

这是PyTorch使用的数据加载器，批量大小为512。

# Create training and validation datasets
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in ['train', 'val', 'test']}
# Create training and validation dataloaders
print('batch size: ', batch_size)
dataloaders_dict = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=batch_size, shuffle=True, num_workers=4) for x in ['train', 'val', 'test']}

最终，我希望得到这样的东西：

｛large_image_1:[0,1,1,0]，large_iimage_2:[1,1,1,0,0,0]，large_image_3:[0,0]，…｝

请注意，就512x512补丁的数量而言，我的大图像具有不同的大小。

我确实在下面看到了87个独特的大图像文件名。不知道为什么在字典中只有两个更新：

fnames = set()
for i in range(len(sample_fnames_labels)):
fname = sample_fnames_labels[i][0].split('/')[-1][:23]
fnames.add(fname)

print(len(fnames))

87

Answer 1

通过在测试的数据加载器中将批量大小设置为1来解决问题

# Create training and validation datasets
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x), data_transforms[x]) for x in ['test']}
# Create training and validation dataloaders
dataloaders_dict = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=1, shuffle=True, num_workers=4) for x in ['test']}
test_large_images = {}
test_loss = 0.0
test_acc = 0

with torch.no_grad():

test_running_loss = 0.0
test_running_corrects = 0
print(len(dataloaders_dict['test']))
for i, (inputs, labels) in enumerate(dataloaders_dict['test']):

print(i)
test_input = inputs.to(device)
test_label = labels.to(device)
test_output = saved_model_ft(test_input)
_, test_pred = torch.max(test_output, 1)
sample_fname, label = dataloaders_dict['test'].dataset.samples[i]
patch_name = sample_fname.split('/')[-1]
large_image_name = patch_name.split('_')[0]
if large_image_name not in test_large_images.keys():
test_large_images[large_image_name] = list()
test_large_images[large_image_name].append(test_pred.item())
else:
test_large_images[large_image_name].append(test_pred.item())

#print('test_large_images.keys(): ', test_large_images.keys())
test_running_corrects += torch.sum(test_preds == test_labels.data)

test_acc = test_running_corrects / len(dataloaders_dict['test'].dataset)
print(test_acc)