使用在马里GPU上完全委托BERT模型

Question

我正在尝试使用TensorflowLite在马里GPU上部署BERT和Transformer模型。但问题是TensorflowLite不支持这些模型中的一些操作，包括{CAST、GATHER、MUL、RESHAPE、UNPACK}。有人知道我如何在GPU上委派这些操作吗？是否有其他TensorflowLite库可以支持嵌入式GPU，特别是马里GPU？我只是想在GPU上测量它们的延迟。

STARTING!
Log parameter values verbosely: [0]
Min num runs: [1]
**Graph**: [mobilebert_float_384_gpu.tflite]
**Use gpu**: [1]
Loaded model mobilebert_float_384_gpu.tflite
**INFO**: Created TensorFlow Lite delegate for GPU.
**ERROR**: Following operations are not supported by GPU delegate:
CAST: Not supported cast case
GATHER: Operation is not supported.
MUL: MUL requires one tensor that not less than second in all dimensions.
RESHAPE: OP is supported, but tensor type isn't matched!
UNPACK: Operation is not supported.
**2661 operations will run on the GPU, and the remaining 81 operations will run on the CPU.**
INFO: Initialized OpenCL-based API.
INFO: Created 1 GPU delegate kernels.
Explicitly applied GPU delegate, and the model graph will be partially executed by the delegate w/ 1 delegate kernels.
The input model file size (MB): 100.239
Initialized session in 2491.9ms.
Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.
count=2 first=434022 curr=247839 min=247839 max=434022 avg=340930 std=93091

Answer 1

GPU代理不支持所有操作，只支持此列表。

如果NNAPI委托对您可用，您可能希望尝试它。它可以委托给不同的处理器作为"代理"；最好"；，不仅仅是GPU。支持的操作在这里。

最糟糕的情况是，如果这两项工作都不适合你，要么回退到CPU，要么编写自己的委托——但这是一大块工作。