我正在尝试使用 AWS s3select 功能来查询镶木地板文件。根据文档,它受支持,但我尝试了各种配置,但无法使其工作。在我显示注释掉的每次 InputSerialization 尝试中,我都列出了我在尝试该版本时收到的错误。有人可以告诉我如何正确配置它吗?
import boto3
S3_BUCKET = 'myBucket'
KEY_LIST = "'0123','6789'"
S3_FILE = 'myFolder/myFile.parquet'
s3 = boto3.client('s3')
r = s3.select_object_content(
Bucket=S3_BUCKET,
Key=S3_FILE,
ExpressionType='SQL',
Expression="select "Record" from s3object s where s."Key" in [" + KEY_LIST + "]",
# InputSerialization={}, # (MissingRequiredParameter) when calling the SelectObjectContent operation: InputSerialization is required
# InputSerialization={'CompressionType': { 'NONE' }}, # Invalid type for parameter InputSerialization.CompressionType, value: {'NONE'}, type: <class 'set'>, valid types: <class 'str'>
# InputSerialization={'Parquet': {}}, # Unknown parameter in InputSerialization: "Parquet", must be one of: CSV, CompressionType, JSON
# InputSerialization={'CompressionType': { 'Snappy' }}, # Invalid type for parameter InputSerialization.CompressionType, value: {'Snappy'}, type: <class 'set'>, valid types: <class 'str'>
OutputSerialization={'JSON': {}},
)
for event in r['Payload']:
if 'Records' in event:
records = event['Records']['Payload'].decode('utf-8')
print(records)
我需要将我的 boto3 安装升级到最新版本。升级到 1.9.7 后,此版本有效:
InputSerialization={'Parquet': {}},