从JSON或CSV将50K条记录加载到DynamoDB



我有一个csv或50K记录以及50K记录的JSON文档。文件或文档是一组地理位置。

我的目标是将所有50万条记录加载到DynamoDB中。我不知道我哪里做错了。目前,我一次最多可以加载25条记录,这是一个将所有50万条记录加载到DynamoDB的漫长过程。

JSON文件大小约为70 MB。

有什么方法可以从保存对象数组的JSON中加载所有50K记录吗?或者以任何方式从CSV加载?我能够从java代码进行加载。但我主要是在寻找一些发电机命令或工具,我应该能够加载。

一个JSON对象如下所示:

"PutRequest": { 
"Item": { 
"id": {"S":"1"},
"zipcode": {"S":"501"},
"city": {"S":"HOLTSVILLE"},
"state": {"S":"NY"},
"county_name": {"S":"SUFFOLK"},
"area_codes": {"S":"631"},
"city_type": {"S":"Non Postal Community Name"},
"city_abbr": {"S":""},
"city_alias": {"S":"I R S SERVICE CENTER"},
"latitude": {"S":"40.81518"},
"longitude": {"S":"-73.0455"},
"time_zone_offset": {"S":"5"},
"elevation": {"S":"25"},
"county_fips": {"S":"103"},
"daylight_savings_flag": {"S":"Y"},
"preferred_last_line_key": {"S":"V13916"},
"classification_code": {"S":"U"},
"multi_county_flag": {"S":" "},
"state_fips": {"S":"36"},
"city_state_key": {"S":"V13995"},
"time_zone": {"S":"America/New_York"},
"daylight_savings": {"S":"1"},
"updated_at": {"S":"2/26/2009 0:00"},
"city_alias_code": {"S":"NULL"},
"primary_record": {"S":"NULL"},
"city_mixed_case": {"S":"NULL"},
"city_alias_mixed_case": {"S":"NULL"},
"state_ansi": {"S":"NULL"},
"county_ansi": {"S":"NULL"},
"facility_code": {"S":"NULL"},
"unique_zip_name": {"S":"NULL"},
"city_delivery_indicator": {"S":"NULL"},
"carrier_route_rate_sortation": {"S":"NULL"},
"finance_number": {"S":"NULL"},
"unique_id": {"S":"NULL"},
"created_at": {"S":"NULL"},
"country_code": {"S":"US"}
}
}

提前谢谢。

AWS几个月前发表了一篇关于这个主题的文章。你可以在这里查看。

最新更新