案例1框架：Tensorflow 2.5.0，Intel Tensorflow 2.50
环境：谷歌Colab
我有一个通过LPOT量化的成功量化模型，该模型将在不使用LPOT API的情况下运行用于推理，因此我编写了以下推理代码：
with tf.compat.v1.Session() as sess:
tf.compat.v1.saved_model.loader.load(sess, ['serve'], model)
output = sess.graph.get_tensor_by_name(output_tensor_name)
predictions = sess.run(output, {input_tensor_name: x})
mse = tf.reduce_mean(tf.keras.losses.mean_squared_error(y, predictions))
print(mse.eval())
运行线路predictions = sess.run(output, {input_tensor_name: x}):时
---------------------------------------------------------------------------
InternalError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1374 try:
-> 1375 return fn(*args)
1376 except errors.OpError as e:
7 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1359 return self._call_tf_sessionrun(options, feed_dict, fetch_list,
-> 1360 target_list, run_metadata)
1361
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1452 fetch_list, target_list,
-> 1453 run_metadata)
1454
InternalError: Missing 0-th output from {{node model/layer_1/Conv2D_eightbit_requantize}}
During handling of the above exception, another exception occurred:
InternalError Traceback (most recent call last)
<ipython-input-6-2bddd853d111> in <module>()
2 tf.compat.v1.saved_model.loader.load(sess, ['serve'], model)
3 output = sess.graph.get_tensor_by_name(output_tensor_name)
----> 4 predictions = sess.run(output, {input_tensor_name: x[:64]}) # 64, 257, 60, 1
5 mse = tf.reduce_mean(tf.keras.losses.mean_squared_error(y[:64], predictions))
6 print(mse.eval())
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
966 try:
967 result = self._run(None, fetches, feed_dict, options_ptr,
--> 968 run_metadata_ptr)
969 if run_metadata:
970 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1189 if final_fetches or final_targets or (handle and feed_dict_tensor):
1190 results = self._do_run(handle, final_targets, final_fetches,
-> 1191 feed_dict_tensor, options, run_metadata)
1192 else:
1193 results = []
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1367 if handle is None:
1368 return self._do_call(_run_fn, feeds, fetches, targets, options,
-> 1369 run_metadata)
1370 else:
1371 return self._do_call(_prun_fn, handle, feeds, fetches)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1392 'nsession_config.graph_options.rewrite_options.'
1393 'disable_meta_optimizer = True')
-> 1394 raise type(e)(node_def, op, message)
1395
1396 def _extend_graph(self):
InternalError: Missing 0-th output from node model/layer_1/Conv2D_eightbit_requantize (defined at <ipython-input-6-2bddd853d111>:2)
无论是否安装了Intel-Tensorflow==2.5.0，都会发生此错误，当显式设置os.environ['TF_ENABLE_ONEDNN_OPTS'] = '1'时也不会解决此错误。
另一方面，当我在带有Python 3.6.8 64-bit base: Conda的VS code中运行相同的代码时，它会返回与情况2中相同的错误消息。
案例2
框架：Tensorflow 2.4.0，Intel Tensorflow 2.4.0
环境：谷歌Colab
这个案例运行良好，并打印出预测的MSE损失，但当我卸载Intel-Tensorflow 2.4.0并仅使用官方Tensorflow运行时，同时在案例1(predictions = sess.run(output, {input_tensor_name: x})(中运行同一行：
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1374 try:
-> 1375 return fn(*args)
1376 except errors.OpError as e:
7 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1357 # Ensure any changes to the graph are reflected in the runtime.
-> 1358 self._extend_graph()
1359 return self._call_tf_sessionrun(options, feed_dict, fetch_list,
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _extend_graph(self)
1397 with self._graph._session_run_lock(): # pylint: disable=protected-access
-> 1398 tf_session.ExtendSession(self._session)
1399
InvalidArgumentError: No OpKernel was registered to support Op 'QuantizedMatMulWithBiasAndDequantize' used by {{node model/dense/Tensordot/MatMul_eightbit_requantize}} with these attrs: [input_quant_mode="MIN_FIRST", T1=DT_QUINT8, Toutput=DT_FLOAT, T2=DT_QINT8, Tbias=DT_QINT32, transpose_a=false, transpose_b=false]
Registered devices: [CPU]
Registered kernels:
<no registered kernels>
[[model/dense/Tensordot/MatMul_eightbit_requantize]]
During handling of the above exception, another exception occurred:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-6-2bddd853d111> in <module>()
2 tf.compat.v1.saved_model.loader.load(sess, ['serve'], model)
3 output = sess.graph.get_tensor_by_name(output_tensor_name)
----> 4 predictions = sess.run(output, {input_tensor_name: x[:64]}) # 64, 257, 60, 1
5 mse = tf.reduce_mean(tf.keras.losses.mean_squared_error(y[:64], predictions))
6 print(mse.eval())
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
966 try:
967 result = self._run(None, fetches, feed_dict, options_ptr,
--> 968 run_metadata_ptr)
969 if run_metadata:
970 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1189 if final_fetches or final_targets or (handle and feed_dict_tensor):
1190 results = self._do_run(handle, final_targets, final_fetches,
-> 1191 feed_dict_tensor, options, run_metadata)
1192 else:
1193 results = []
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1367 if handle is None:
1368 return self._do_call(_run_fn, feeds, fetches, targets, options,
-> 1369 run_metadata)
1370 else:
1371 return self._do_call(_prun_fn, handle, feeds, fetches)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1392 'nsession_config.graph_options.rewrite_options.'
1393 'disable_meta_optimizer = True')
-> 1394 raise type(e)(node_def, op, message)
1395
1396 def _extend_graph(self):
InvalidArgumentError: No OpKernel was registered to support Op 'QuantizedMatMulWithBiasAndDequantize' used by node model/dense/Tensordot/MatMul_eightbit_requantize (defined at <ipython-input-6-2bddd853d111>:2) with these attrs: [input_quant_mode="MIN_FIRST", T1=DT_QUINT8, Toutput=DT_FLOAT, T2=DT_QINT8, Tbias=DT_QINT32, transpose_a=false, transpose_b=false]
Registered devices: [CPU]
Registered kernels:
<no registered kernels>
[[model/dense/Tensordot/MatMul_eightbit_requantize]]
即使显式设置了os.environ['TF_ENABLE_ONEDNN_OPTS'] = '1'，错误仍然存在。
结论我相信这两种情况都是由相同类型的错误引起的，即没有注册OpKernel来支持Op。。。
我了解到，在安装了官方Tensorflow v2.5并设置了环境变量TF_ENABLE_ONEDNN_OPTS=1(参考(的情况下，量化模型应该在支持oneDNN的情况下运行。但在v2.4和v2.5中似乎都不是这样。
我的问题是，如何在不安装Intel-Tensorflow的情况下获得支持oneDNN的官方Tensorflow 2.5环境？或者为什么Intel-Tensorflow 2.5不起作用？谢谢
LPOT在Intel®AI Analytics Toolkit中发布，可与Intel Optimization of TensorFlow配合使用。LPOT可以在任何英特尔CPU上运行，以量化人工智能模型。"英特尔优化TensorFlow 2.5.0"要求在运行LPOT量化或部署量化模型之前设置环境变量TF_ENABLE_MKL_NATIVE_FORMAT=0。
请参阅此了解更多信息。
你能检查一下你是否在2.4中量化了Tensorflow中的模型，并在Tensorflow 2.5上运行了推理吗？对未在Tensorflow 2.5中运行和在Tensorflow 2.4中运行的模型的合理解释是，支持Tensorflow 2.5的运营商可能不支持在Tensorflow2.4中创建的模型。