EventBridge触发器:Sagemaker处理作业已完成

Question

我目前正在用AWS为我的ML模型开发一些ETL。问题是，当Sagemaker处理作业完成时，我想触发Lambda。传递给Lambda的事件应该是Sagemaker处理作业的配置信息(作业名称、参数等(。

Q1：处理作业完成后，如何触发事件？

Q2：如何将处理作业配置作为Lambda的事件传递？

Answer 1

您可以使用以下EventBridge规则模式：

{
"source": ["aws.sagemaker"],
"detail-type": ["SageMaker Processing Job State Change"],
"detail": {
"ProcessingJobStatus": ["Failed", "Completed", "Stopped"]
}
}

ProcessingJobStatus列表可以根据要处理的状态进行修改。

您可以将Lambda函数设置为EventBridge规则的目标。

以下是一个将传递给您的Lambda的示例事件，取自AWS控制台：

{
"version": "0",
"id": "0a15f67d-aa23-0123-0123-01a23w89r01t",
"detail-type": "SageMaker Processing Job State Change",
"source": "aws.sagemaker",
"account": "123456789012",
"time": "2019-05-31T21:49:54Z",
"region": "us-east-1",
"resources": ["arn:aws:sagemaker:us-west-2:012345678987:processing-job/integ-test-analytics-algo-54ee3282-5899-4aa3-afc2-7ce1d02"],
"detail": {
"ProcessingInputs": [{
"InputName": "InputName",
"S3Input": {
"S3Uri": "s3://input/s3/uri",
"LocalPath": "/opt/ml/processing/input/local/path",
"S3DataType": "MANIFEST_FILE",
"S3InputMode": "PIPE",
"S3DataDistributionType": "FULLYREPLICATED"
}
}],
"ProcessingOutputConfig": {
"Outputs": [{
"OutputName": "OutputName",
"S3Output": {
"S3Uri": "s3://output/s3/uri",
"LocalPath": "/opt/ml/processing/output/local/path",
"S3UploadMode": "CONTINUOUS"
}
}],
"KmsKeyId": "KmsKeyId"
},
"ProcessingJobName": "integ-test-analytics-algo-54ee3282-5899-4aa3-afc2-7ce1d02",
"ProcessingResources": {
"ClusterConfig": {
"InstanceCount": 3,
"InstanceType": "ml.c5.xlarge",
"VolumeSizeInGB": 5,
"VolumeKmsKeyId": "VolumeKmsKeyId"
}
},
"StoppingCondition": {
"MaxRuntimeInSeconds": 2000
},
"AppSpecification": {
"ImageUri": "012345678901.dkr.ecr.us-west-2.amazonaws.com/processing-uri:latest"
},
"NetworkConfig": {
"EnableInterContainerTrafficEncryption": true,
"EnableNetworkIsolation": false,
"VpcConfig": {
"SecurityGroupIds": ["SecurityGroupId1", "SecurityGroupId2", "SecurityGroupId3"],
"Subnets": ["Subnet1", "Subnet2"]
}
},
"RoleArn": "arn:aws:iam::012345678987:role/SageMakerPowerUser",
"ExperimentConfig": {},
"ProcessingJobArn": "arn:aws:sagemaker:us-west-2:012345678987:processing-job/integ-test-analytics-algo-54ee3282-5899-4aa3-afc2-7ce1d02",
"ProcessingJobStatus": "Completed",
"LastModifiedTime": 1589879735000,
"CreationTime": 1589879735000
}
}

编辑：

如果您想将ProcessingJobName与特定前缀匹配：

{
"source": ["aws.sagemaker"],
"detail-type": ["SageMaker Processing Job State Change"],
"detail": {
"ProcessingJobStatus": ["Failed", "Completed", "Stopped"],
"ProcessingJobName": [{
"prefix": "standarize-data"
}]
}
}