如何在 SageMaker 中调用 fit 方法时防止 NoCredentialsError ?

Question

我是Python SageMaker的新手(我的背景是C#(。目前，我有一个问题，因为最后一个方法调用(我的意思是适合方法(导致"NoCredentialsError"。我不明白这一点。AWS 凭证已设置，我确实使用它们与 AWS 通信，例如与 S3 通信。如何防止此错误？

import io
import os
import gzip
import pickle
import urllib.request
import boto3
import sagemaker
import sagemaker.amazon.common as smac
DOWNLOADED_FILENAME = 'C:/Users/Daan/PycharmProjects/downloads/mnist.pkl.gz'
if not os.path.exists(DOWNLOADED_FILENAME):
urllib.request.urlretrieve("http://deeplearning.net/data/mnist/mnist.pkl.gz", DOWNLOADED_FILENAME)
with gzip.open(DOWNLOADED_FILENAME, 'rb') as f:
train_set, valid_set, test_set = pickle.load(f, encoding='latin1')
vectors = train_set[0].T
buf = io.BytesIO()
smac.write_numpy_to_dense_tensor(buf, vectors)
buf.seek(0)
key = 'recordio-pb-data'
bucket_name = 'SOMEKINDOFBUCKETNAME'
prefix = 'sagemaker/pca'
path = os.path.join(prefix, 'train', key)
print(path)
session = boto3.session.Session(aws_access_key_id='SECRET',aws_secret_access_key='SECRET',region_name='eu-west-1')
client = boto3.client('sagemaker',region_name='eu-west-1',aws_access_key_id='SECRET',aws_secret_access_key='SECRET')
region='eu-west-1'
sagemakerSession= sagemaker.Session(sagemaker_client=client,boto_session=session)
s3_resource=session.resource('s3')
bucket = s3_resource.Bucket(bucket_name)
current_bucket = bucket.Object(path)
train_data = 's3://{}/{}/train/{}'.format(bucket_name, prefix, key)
print('uploading training data location: {}'.format(train_data))
current_bucket.upload_fileobj(buf)
output_location = 's3://{}/{}/output'.format('SOMEBUCKETNAME', prefix)
print('training artifacts will be uploaded to: {}'.format(output_location))
region='eu-west-1'
containers = {'us-west-2': 'SOMELOCATION',
'us-east-1': 'SOMELOCATION',
'us-east-2': 'SOMELOCATION',
'eu-west-1': 'SOMELOCATION'}
container = containers[region]
role='AmazonSageMaker-ExecutionRole-SOMEVALUE'
pca = sagemaker.estimator.Estimator(container,
role,
train_instance_count=1,
train_instance_type='ml.c4.xlarge',
output_path=output_location,
sagemaker_session=sagemakerSession)

pca.set_hyperparameters(feature_dim=50000,
num_components=10,
subtract_mean=True,
algorithm_mode='randomized',
mini_batch_size=200)
pca.fit(inputs=train_data)
print('END')

Answer 1

我不确定您是否屏蔽了实际的访问 ID 和密钥，或者这就是您正在运行的内容。

session = boto3.session.Session(aws_access_key_id='SECRET',aws_secret_access_key='SECRET',region_name='eu-west-1')
client = boto3.client('sagemaker',region_name='eu-west-1',aws_access_key_id='SECRET',aws_secret_access_key='SECRET')

我希望您在上面的代码行中提供实际的aws_access_key_id和aws_secret_access_key。

在代码中指定相同但不硬编码的另一种方法是在配置文件目录中创建一个凭据文件，即

在 Mac 中 ~/.aws/

和视窗"%UserProfile%.aws"

该文件是名称为"凭据"(不带引号(的纯文本文件。 文件包含

[default]
aws_access_key_id=XXXXXXXXXXXXXX
aws_secret_access_key=YYYYYYYYYYYYYYYYYYYYYYYYYYY

AWS CLI 将从上述位置获取它并使用它。您还可以使用非默认配置文件并传递配置文件

os.environ["AWS_PROFILE"] = "profile-name"

希望这有帮助。