我在尝试训练网络时遇到了很多内存分配问题。我有一个 GTX 1070，我正在使用 conda 来设置环境。目前我的 GPU 正在运行我的显示器，所以我想知道使用板载显卡是否有助于解决我的问题？此外，我收到的错误日志突然更改为略有不同，我完全不知道我更改了什么结果，但现在我得到的输出如下所示：
$ python training_bpm_only.py 100 temp2
2019-11-27 15:22:46.586350: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
2019-11-27 15:22:49.251739: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2019-11-27 15:22:49.362166: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.7465
pciBusID: 0000:01:00.0
2019-11-27 15:22:49.362539: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2019-11-27 15:22:49.363666: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-11-27 15:22:49.364438: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX
2019-11-27 15:22:49.366368: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.7465
pciBusID: 0000:01:00.0
2019-11-27 15:22:49.366684: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2019-11-27 15:22:49.367869: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-11-27 15:22:50.178519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-11-27 15:22:50.178786: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2019-11-27 15:22:50.178932: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2019-11-27 15:22:50.180129: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6382 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
2019-11-27 15:22:51.852085: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll
2019-11-27 15:22:53.857910: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudnn64_7.dll
2019-11-27 15:22:55.126031: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: Invoking ptxas not supported on Windows
Relying on driver to perform ptx compilation. This message will be only logged once.
2019-11-27 15:23:05.813409: W tensorflow/core/common_runtime/bfc_allocator.cc:419] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.91GiB (rounded to 2047868928). Current allocation summary follows.
2019-11-27 15:23:05.813823: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (256): Total Chunks: 31, Chunks in use: 30. 7.8KiB allocated for chunks. 7.5KiB in use in bin. 1.2KiB client-requested in use in bin.
2019-11-27 15:23:05.814212: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (512): Total Chunks: 6, Chunks in use: 6. 3.0KiB allocated for chunks. 3.0KiB in use in bin. 3.0KiB client-requested in use in bin.
2019-11-27 15:23:05.814598: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (1024): Total Chunks: 1, Chunks in use: 1. 1.3KiB allocated for chunks. 1.3KiB in use in bin. 1.0KiB client-requested in use in bin.
2019-11-27 15:23:05.814997: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (2048): Total Chunks: 2, Chunks in use: 2. 7.0KiB allocated for chunks. 7.0KiB in use in bin. 6.8KiB client-requested in use in bin.
2019-11-27 15:23:05.815362: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (4096): Total Chunks: 1, Chunks in use: 1. 4.0KiB allocated for chunks. 4.0KiB in use in bin. 3.4KiB client-requested in use in bin.
2019-11-27 15:23:05.815691: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.816017: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.816344: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.816676: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (65536): Total Chunks: 2, Chunks in use: 2. 144.0KiB allocated for chunks. 144.0KiB in use in bin. 144.0KiB client-requested in use in bin.
2019-11-27 15:23:05.817093: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (131072): Total Chunks: 1, Chunks in use: 1. 143.8KiB allocated for chunks. 143.8KiB in use in bin. 72.0KiB client-requested in use in bin.
2019-11-27 15:23:05.817519: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (262144): Total Chunks: 1, Chunks in use: 0. 388.5KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.817843: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (524288): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.818163: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (1048576): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.818487: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (2097152): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.818834: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (4194304): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.819154: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (8388608): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.819475: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.819791: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.820108: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (67108864): Total Chunks: 1, Chunks in use: 0. 95.62MiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2019-11-27 15:23:05.820447: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (134217728): Total Chunks: 4, Chunks in use: 4. 880.42MiB allocated for chunks. 880.42MiB in use in bin. 828.38MiB client-requested in use in bin.
2019-11-27 15:23:05.820813: I tensorflow/core/common_runtime/bfc_allocator.cc:869] Bin (268435456): Total Chunks: 5, Chunks in use: 4. 5.28GiB allocated for chunks. 3.84GiB in use in bin. 3.84GiB client-requested in use in bin.
2019-11-27 15:23:05.821159: I tensorflow/core/common_runtime/bfc_allocator.cc:885] Bin for 1.91GiB was 256.00MiB, Chunk State:
2019-11-27 15:23:05.821355: I tensorflow/core/common_runtime/bfc_allocator.cc:891] Size: 1.44GiB | Requested Size: 488.25MiB | in_use: 0 | bin_num: 20, prev: Size: 488.25MiB | Requested Size: 488.25MiB | in_use: 1 | bin_num: -1
2019-11-27 15:23:05.821712: I tensorflow/core/common_runtime/bfc_allocator.cc:898] Next region of size 6692228864
2019-11-27 15:23:05.821894: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00000 next 1 of size 1280
2019-11-27 15:23:05.822089: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00500 next 2 of size 256
2019-11-27 15:23:05.822280: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00600 next 5 of size 256
2019-11-27 15:23:05.822471: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00700 next 4 of size 256
2019-11-27 15:23:05.822660: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00800 next 8 of size 512
2019-11-27 15:23:05.822889: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00A00 next 15 of size 256
2019-11-27 15:23:05.823084: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00B00 next 11 of size 256
2019-11-27 15:23:05.823279: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00C00 next 14 of size 512
2019-11-27 15:23:05.823478: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00E00 next 13 of size 256
2019-11-27 15:23:05.823674: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E00F00 next 16 of size 256
2019-11-27 15:23:05.823863: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E01000 next 17 of size 256
2019-11-27 15:23:05.824054: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E01100 next 18 of size 256
2019-11-27 15:23:05.824246: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E01200 next 19 of size 256
2019-11-27 15:23:05.824444: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E01300 next 20 of size 256
2019-11-27 15:23:05.824649: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E01400 next 3 of size 4096
2019-11-27 15:23:05.824871: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E02400 next 6 of size 3584
2019-11-27 15:23:05.825084: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E03200 next 21 of size 256
2019-11-27 15:23:05.825292: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E03300 next 9 of size 147200
2019-11-27 15:23:05.825510: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E27200 next 7 of size 73728
2019-11-27 15:23:05.825707: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E39200 next 22 of size 256
2019-11-27 15:23:05.825897: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000504E39300 next 23 of size 255983616
2019-11-27 15:23:05.826105: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000514259300 next 24 of size 512
2019-11-27 15:23:05.826301: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000514259500 next 25 of size 512
2019-11-27 15:23:05.826504: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000514259700 next 26 of size 256
2019-11-27 15:23:05.826685: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000514259800 next 27 of size 3584
2019-11-27 15:23:05.826879: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051425A600 next 28 of size 256
2019-11-27 15:23:05.827097: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051425A700 next 29 of size 73728
2019-11-27 15:23:05.827299: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426C700 next 30 of size 256
2019-11-27 15:23:05.827493: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426C800 next 32 of size 512
2019-11-27 15:23:05.827704: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426CA00 next 33 of size 512
2019-11-27 15:23:05.827918: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426CC00 next 34 of size 256
2019-11-27 15:23:05.828136: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426CD00 next 35 of size 256
2019-11-27 15:23:05.828339: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426CE00 next 36 of size 256
2019-11-27 15:23:05.828632: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426CF00 next 37 of size 256
2019-11-27 15:23:05.828903: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D000 next 38 of size 256
2019-11-27 15:23:05.829156: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D100 next 39 of size 256
2019-11-27 15:23:05.829423: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D200 next 40 of size 256
2019-11-27 15:23:05.829678: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D300 next 41 of size 256
2019-11-27 15:23:05.829920: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D400 next 42 of size 256
2019-11-27 15:23:05.830131: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D500 next 43 of size 256
2019-11-27 15:23:05.830322: I tensorflow/core/common_runtime/bfc_allocator.cc:905] Free at 000000051426D600 next 44 of size 256
2019-11-27 15:23:05.830544: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D700 next 45 of size 256
2019-11-27 15:23:05.830747: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051426D800 next 46 of size 256
2019-11-27 15:23:05.830942: I tensorflow/core/common_runtime/bfc_allocator.cc:905] Free at 000000051426D900 next 69 of size 397824
2019-11-27 15:23:05.831138: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 00000005142CEB00 next 70 of size 256
2019-11-27 15:23:05.831332: I tensorflow/core/common_runtime/bfc_allocator.cc:905] Free at 00000005142CEC00 next 48 of size 100266240
2019-11-27 15:23:05.831552: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051A26DD00 next 49 of size 256
2019-11-27 15:23:05.831748: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 000000051A26DE00 next 12 of size 155235328
2019-11-27 15:23:05.831952: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000523679200 next 10 of size 255983616
2019-11-27 15:23:05.832178: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000532A99200 next 31 of size 255983616
2019-11-27 15:23:05.832408: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000541EB9200 next 50 of size 1048707072
2019-11-27 15:23:05.832633: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 00000005806D9200 next 51 of size 2047868928
2019-11-27 15:23:05.832849: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 00000005FA7D9200 next 52 of size 511967232
2019-11-27 15:23:05.833057: I tensorflow/core/common_runtime/bfc_allocator.cc:905] InUse at 0000000619019200 next 53 of size 511967232
2019-11-27 15:23:05.833269: I tensorflow/core/common_runtime/bfc_allocator.cc:905] Free at 0000000637859200 next 18446744073709551615 of size 1547549952
2019-11-27 15:23:05.833508: I tensorflow/core/common_runtime/bfc_allocator.cc:914] Summary of in-use Chunks by size:
2019-11-27 15:23:05.833704: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 30 Chunks of size 256 totalling 7.5KiB
2019-11-27 15:23:05.833896: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 6 Chunks of size 512 totalling 3.0KiB
2019-11-27 15:23:05.834090: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 1280 totalling 1.3KiB
2019-11-27 15:23:05.834281: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 2 Chunks of size 3584 totalling 7.0KiB
2019-11-27 15:23:05.834477: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 4096 totalling 4.0KiB
2019-11-27 15:23:05.834664: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 2 Chunks of size 73728 totalling 144.0KiB
2019-11-27 15:23:05.834875: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 147200 totalling 143.8KiB
2019-11-27 15:23:05.835081: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 155235328 totalling 148.04MiB
2019-11-27 15:23:05.835283: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 3 Chunks of size 255983616 totalling 732.38MiB
2019-11-27 15:23:05.835490: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 2 Chunks of size 511967232 totalling 976.50MiB
2019-11-27 15:23:05.835688: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 1048707072 totalling 1000.13MiB
2019-11-27 15:23:05.835888: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 2047868928 totalling 1.91GiB
2019-11-27 15:23:05.836080: I tensorflow/core/common_runtime/bfc_allocator.cc:921] Sum Total of in-use chunks: 4.70GiB
2019-11-27 15:23:05.836264: I tensorflow/core/common_runtime/bfc_allocator.cc:923] total_region_allocated_bytes_: 6692228864 memory_limit_: 6692228956 available bytes: 92 curr_region_allocation_bytes_: 13384458240
2019-11-27 15:23:05.836621: I tensorflow/core/common_runtime/bfc_allocator.cc:929] Stats:
Limit: 6692228956
InUse: 5044014592
MaxInUse: 6324449792
NumAllocs: 220
MaxAllocSize: 2659202560
2019-11-27 15:23:05.837111: W tensorflow/core/common_runtime/bfc_allocator.cc:424] ****_************************************************************************_______________________
2019-11-27 15:23:05.837429: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at pooling_ops_common.cc:309 : Resource exhausted: OOM when allocating tensor with shape[256,64,252,124] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
2019-11-27 15:23:05.837922: W tensorflow/core/common_runtime/base_collective_executor.cc:216] BaseCollectiveExecutor::StartAbort Resource exhausted: OOM when allocating tensor with shape[256,64,252,124] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node MaxPoolGrad}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
Train for 500 steps
Epoch 1/100
1/500 [..............................] - ETA: 2:04:47Traceback (most recent call last):
File "training_bpm_only.py", line 84, in <module>
run(*argv)
File "training_bpm_only.py", line 70, in run
model.fit(ds, epochs=int(epochs), steps_per_epoch=500)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythonkerasenginetraining.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 324, in fit
total_epochs=epochs)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythonkerasenginetraining_v2.py", line 123, in run_one_epoch
batch_outs = execution_function(iterator)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythonkerasenginetraining_v2_utils.py", line 86, in execution_function
distributed_function(input_fn))
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerdef_function.py", line 457, in __call__
result = self._call(*args, **kwds)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerdef_function.py", line 520, in _call
return self._stateless_fn(*args, **kwds)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerfunction.py", line 1823, in __call__
return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerfunction.py", line 1141, in _filtered_call
self.captured_inputs)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerfunction.py", line 1224, in _call_flat
ctx, args, cancellation_manager=cancellation_manager)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerfunction.py", line 511, in call
ctx=ctx)
File "C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythoneagerexecute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[256,64,252,124] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[node MaxPoolGrad (defined at C:UserscainyAnaconda3envsbeatnetlibsite-packagestensorflow_corepythonframeworkops.py:1751) ]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.
[Op:__inference_distributed_function_1122]
Function call stack:
distributed_function
取决于模型的复杂程度。1070具有8gb的vram，这对于上一代型号来说应该足够了。您也正在运行Windows，因此~1 GB的vram用于此(即使在空闲时(。
作为一般规则，您的 GPU 应该仅用于模型训练，因为大多数模型几乎不适合内存。 您会遇到一些 BIOS 配置问题，因为大多数 igp 在放入 GPU 时会关闭。