我正试图使用不同的图像从本文(模型1(中重建CNN图像识别模型。然而,拟合模型会在第一个epoch返回ResourceExhaustedError。批量已经相当小了,所以我猜问题出在我从论文中复制的模型定义上。如有任何关于该型号更改内容的建议,我们将不胜感激。非常感谢。
#Load dataset
BATCH_SIZE = 32
IMG_SIZE = (244,244)
train_set = tf.keras.preprocessing.image_dataset_from_directory(
main_dir,
shuffle = True,
image_size = IMG_SIZE,
batch_size = BATCH_SIZE)
val_set = tf.keras.preprocessing.image_dataset_from_directory(
main_dir,
shuffle = True,
image_size = IMG_SIZE,
batch_size = BATCH_SIZE)
class_names = train_set.class_names
print(class_names)
#Augment data by flipping image and random rotation
data_augmentation = tf.keras.Sequential([
tf.keras.layers.experimental.preprocessing.RandomFlip('horizontal'),
tf.keras.layers.experimental.preprocessing.RandomRotation(0.2),
])
#Model definition
model = Sequential([
data_augmentation,
tf.keras.layers.experimental.preprocessing.Rescaling(1./255),
Conv2D(filters=64,kernel_size=(4,4), activation='relu'),
Conv2D(filters=32,kernel_size=(3,3), activation='relu'),
AveragePooling2D(pool_size=(4,4)),
Conv2D(filters=32,kernel_size=(3,3), activation='relu'),
Conv2D(filters=32,kernel_size=(3,3), activation='relu'),
Conv2D(filters=32,kernel_size=(3,3), activation='relu'),
AveragePooling2D(pool_size=(2,2)),
Flatten(),
Dense(256, activation='relu'),
Dense(256, activation='relu'),
Dense(128, activation='relu'),
Dense(128, activation='relu'),
Dense(128, activation='tanh'),
Dense(1, activation='softmax')
])
model.compile(optimizer='RMSprop',
loss=keras.losses.CategoricalCrossentropy(from_logits=True),
metrics=[keras.metrics.CategoricalAccuracy()])
history = model.fit(train_set,validation_data=val_set, epochs=150)
拟合模型后的错误:
ResourceExhaustedError: OOM when allocating tensor with shape[32,32,239,239] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node gradient_tape/sequential_1/average_pooling2d/AvgPoolGrad (defined at <ipython-input-10-ef749d320491>:1) ]]
nvidia smi
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.91.03 Driver Version: 460.91.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce 940MX Off | 00000000:01:00.0 Off | N/A |
| N/A 46C P0 N/A / N/A | 1938MiB / 2004MiB | 2% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 959 G /usr/lib/xorg/Xorg 97MiB |
| 0 N/A N/A 1270 G /usr/bin/gnome-shell 25MiB |
| 0 N/A N/A 4635 G /usr/lib/firefox/firefox 212MiB |
| 0 N/A N/A 5843 C /usr/bin/python3 1595MiB |
+-----------------------------------------------------------------------------+
错误是由GPU内存不足引起的。这可能是因为1(即使对于你的GPU,你每个历元加载的数据也太多,或者2(如果你碰巧有足够的VRAM,另一个进程在历元之间保留了一些内存。这可能是因为tensorflow在运行时保留了100%的VRAM。您可以将保留的VRAM数量限制为所需数量。
import tensorflow as tf
for gpu in tf.config.list_physical_devices("GPU")
tf.config.experimental.set_memory_growth(gpu, True)
编辑:看看你的显卡,我很确定你没有足够的VRAM来满足你的批量大小和输入大小(所以问题1(。您应该降低批量大小。