云数据流 找不到模型

Question

我正在运行一个云数据流作业，这给了我以下错误

java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -156: Traceback (most recent call last):
File "apache_beam/runners/common.py", line 813, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 610, in apache_beam.runners.common.PerWindowInvoker.invoke_process
File "apache_beam/runners/common.py", line 685, in apache_beam.runners.common.PerWindowInvoker._invoke_process_per_window
File "review_data_sim_pipeline_filter.py", line 47, in process
File "review_data_sim_pipeline_filter.py", line 31, in predict_json
File "/usr/local/lib/python3.5/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.5/site-packages/googleapiclient/http.py", line 856, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://ml.googleapis.com/v1/projects/<myprojectname>/models/xgb_cloudml_train:predict?alt=json returned "Field: name Error: The model resource: "xgb_cloudml_train" was not found. Please create the Cloud ML model resource first by using 'gcloud ml-engine models create xgb_cloudml_train'.". Details: "[{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'description': 'The model resource: "xgb_cloudml_train" was not found. Please create the Cloud ML model resource first by using 'gcloud ml-engine models create xgb_cloudml_train'.', 'field': 'name'}]}]">

但是我的模型确实存在，当我使用代码时

def predict_json(self, project, model, instances, version=None): 
import googleapiclient.discovery
service = googleapiclient.discovery.build('ml', 'v1', discoveryServiceUrl='https://storage.googleapis.com/cloud-ml/discovery/ml_v1_discovery.json',cache_discovery=True)
name = 'projects/{}/models/{}'.format(project, model)
if version is not None:
name += '/versions/{}'.format(version)
response = service.projects().predict(
name=name,
body={'instances': instances}
).execute()
if 'error' in response:
raise RuntimeError(response['error'])
return response['predictions']

我得到回应。但是，当我与云ml一起使用时，我收到此错误。

Answer 1

您可能需要将模型的服务代理ID 作为对象查看器添加到存储桶中。

Answer 2

根据日志，xgb_cloudml_train是否有活动且有效的默认模型版本？ 我记得在版本未定义时看到此错误...

https://ml.googleapis.com/v1/projects/<myprojectname>/models/xgb_cloudml_train:predict

否则，可以使用 curl 验证具有非默认版本的模型的 URL 的简单方法：

export PROJECT_ID=$(gcloud config get-value core/project 2> /dev/null)
export ACCESS_TOKEN=`gcloud auth print-access-token`
export MODEL_NAME="pretrained_model"
export MODEL_VERSION="cpu"
# Verify MODEL_NAME and MODEL_VERSION are defined
export URL=https://ml.googleapis.com/v1/projects/$PROJECT_ID/models/$MODEL_NAME/versions/$MODEL_VERSION:predict
## Send HTTPS Request
curl -X POST $URL -d @image_b64.json 
-H "Content-Type: application/json" 
-H "Authorization: Bearer $ACCESS_TOKEN" 

如果是默认版本，您可以使用：

URL=https://ml.googleapis.com/v1/projects/$PROJECT_ID/models/$MODEL_NAME:predict

这将导致这样的事情：

https://ml.googleapis.com/v1/projects/dpe-cloud-mle/models/pretrained_model:predict

此 Java 示例代码可能有助于验证数据流代码：

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.FileContent;
import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.HttpContent;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestFactory;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.UriTemplate;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.discovery.Discovery;
import com.google.api.services.discovery.model.JsonSchema;
import com.google.api.services.discovery.model.RestDescription;
import com.google.api.services.discovery.model.RestMethod;
import java.io.File;
/*
* Sample code for doing AI Platform online prediction in Java.
*/
public class OnlinePredictionSample {
public static void main(String[] args) throws Exception {
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
Discovery discovery = new Discovery.Builder(httpTransport, jsonFactory, null).build();
RestDescription api = discovery.apis().getRest("ml", "v1").execute();
RestMethod method = api.getResources().get("projects").getMethods().get("predict");
JsonSchema param = new JsonSchema();
String projectId = "YOUR_PROJECT_ID";
// You should have already deployed a model and a version.
// For reference, see https://cloud.google.com/ml-engine/docs/deploying-models.
String modelId = "YOUR_MODEL_ID";
String versionId = "YOUR_VERSION_ID";
param.set(
"name", String.format("projects/%s/models/%s/versions/%s", projectId, modelId, versionId));
GenericUrl url =
new GenericUrl(UriTemplate.expand(api.getBaseUrl() + method.getPath(), param, true));
System.out.println(url);
String contentType = "application/json";
File requestBodyFile = new File("input.txt");
HttpContent content = new FileContent(contentType, requestBodyFile);
System.out.println(content.getLength());
GoogleCredential credential = GoogleCredential.getApplicationDefault();
HttpRequestFactory requestFactory = httpTransport.createRequestFactory(credential);
HttpRequest request = requestFactory.buildRequest(method.getHttpMethod(), url, content);
String response = request.execute().parseAsString();
System.out.println(response);
}
}

另外，您使用哪个帐户通过数据流进行身份验证？