来自google ml引擎的JSON无效

Question

我在AI平台上部署了一个TensorFlow对象检测模型。型号配置如下：

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['image_bytes'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: encoded_image_string_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 300, 4)
name: detection_boxes:0
outputs['detection_classes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 300)
name: detection_classes:0
outputs['detection_features'] tensor_info:
dtype: DT_FLOAT
shape: (-1, -1, -1, -1, -1)
name: detection_features:0
outputs['detection_multiclass_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 300, 2)
name: detection_multiclass_scores:0
outputs['detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 300)
name: detection_scores:0
outputs['num_detections'] tensor_info:
dtype: DT_FLOAT
shape: (-1)
name: num_detections:0
outputs['raw_detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 300, 4)
name: raw_detection_boxes:0
outputs['raw_detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 300, 2)
name: raw_detection_scores:0
Method name is: tensorflow/serving/predict

我正在使用以下代码生成预测

def predict_json(project, model, request, version=None):
"""Send json data to a deployed model for prediction.
Args:
project (str): project where the Cloud ML Engine Model is deployed.
model (str): model name.
instances ([Mapping[str: Any]]): Keys should be the names of Tensors
your deployed model expects as inputs. Values should be datatypes
convertible to Tensors, or (potentially nested) lists of datatypes
convertible to tensors.
version: str, version of the model to target.
Returns:
Mapping[str: any]: dictionary of prediction results defined by the
model.
"""
# Create the ML Engine service object.
# To authenticate set the environment variable
# GOOGLE_APPLICATION_CREDENTIALS=<path_to_service_account_file>
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = <service-account-file>
service = build('ml', 'v1')
name = 'projects/{}/models/{}'.format(project, model)
if version is not None:
name += '/versions/{}'.format(version)
response = service.projects().predict(
name=name,
body=request
)
response.execute()
if 'error' in response:
raise RuntimeError(response['error'])
return response['predictions']
img = Image.open(image)
output_str = io.BytesIO()
img.save(output_str, "JPEG")
image_byte_array = output_str.getvalue()
image_base64 = base64.b64encode(image_byte_array).decode()
request = {"instances": [{"image_bytes": {"b64": image_base64}}]}
prediction = predict_json('handdetector', 'fastercnn', request)

预测返回，但缺少一个标签，即"detection_sscores"。此外，响应不是正确的JSON，这导致了以下错误：

Traceback (most recent call last):
File "/Users/syedmustufainabbasrizvi/PycharmProjects/sign-language/Sign-Language-Translation/detection/predict.py", line 60, in <module>
prediction = predict_json('handdetector', 'fastercnn', request)
File "/Users/syedmustufainabbasrizvi/PycharmProjects/sign-language/Sign-Language-Translation/detection/predict.py", line 36, in predict_json
response.execute()
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/sign-language/lib/python3.6/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/sign-language/lib/python3.6/site-packages/googleapiclient/http.py", line 857, in execute
return self.postproc(resp, content)
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/sign-language/lib/python3.6/site-packages/googleapiclient/model.py", line 216, in response
return self.deserialize(content)
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/sign-language/lib/python3.6/site-packages/googleapiclient/model.py", line 274, in deserialize
body = json.loads(content)
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/3.6.5/lib/python3.6/json/__init__.py", line 354, in loads
return _default_decoder.decode(s)
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/3.6.5/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/syedmustufainabbasrizvi/.pyenv/versions/3.6.5/lib/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 9 column 1976399 (char 1999826)

我手动检查了响应的主体，发现它没有返回有效的json，因为一些括号也丢失了，当它试图在内部加载回json时，这会导致json错误。有人遇到过类似的经历吗？

Answer 1

对此有任何解决方案吗@gogasca我也有同样的问题。

我所能猜测的是，人工智能平台上有一个规模阈值。我有一个对象检测模型，它被设置为预测100个类，它的saved_model.pb是~55MB，它工作得很好。。。我有第二个模型，它预测了近1000个类，其saved_model.pb约为160MB，该模型抑制了上述错误。

也许可以找到一种使用tf服务而不是AI平台的方法，或者制作一个更小的模型。但这只是一个猜测，因为人工智能平台对这个问题的文档很少。

在我的案例中，答案是一个较小的模型，这很不幸，因为1000个类更适合我的特定用例。尽管如此，你发布的问题的解决方案。