使用 Java 开发工具包列出 Amazon S3 中的所有对象

Question

我列出了存储桶中的所有对象。问题是当我列出所有对象时，批次中的最后一个对象再次考虑下一批对象，并在下一批中重复。为什么会发生这种情况，因为它应该只考虑批处理中的 1000 个对象，而不应该考虑以前的批处理对象。

BasicAWSCredentials credentials = new BasicAWSCredentials("foo", "bar");
client = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(http://(serviceEndpoint), null(signingRegion is null))
.withPathStyleAccessEnabled(true)
.withChunkedEncodingDisabled(true)
.build();
ObjectListing listing = client.listObjects( "bucketname");
System.out.println("Listing size "+listing.getObjectSummaries().size());
System.out.println("At 0 index "+ listing.getObjectSummaries().get(0).getKey());
System.out.println("At 999 index "+ listing.getObjectSummaries().get(999).getKey());
SomeFunction(listing);
while (listing.isTruncated()) {
System.out.println("-----------------------------------------------");
listing = client.listNextBatchOfObjects(listing);
System.out.println("Listing size "+listing.getObjectSummaries().size());
System.out.println("At 0 index "+ listing.getObjectSummaries().get(0).getKey());
System.out.println("At 999 index "+ listing.getObjectSummaries().get(1000).getKey());
someFunction(listing);
}

我的输出是：

Listing size 1000
At 0 index folder1/a.gz
At 999 index folder1/b.gz
---------------------------------------------------------------
Listing size 1001
At 0 index folder1/b.gz
At 1000 index folder1/d.gz
---------------------------------------------------------------
Listing size 1001
At 0 index folder1/d.gz
At 1000 index folder1/e.gz

如您所见，第一批999指数被认为是第二批(相同(为什么？它不应该发生吧？下一批正在获取 1001 个对象，包括上一个对象中的最后一个对象，因为它应该给出下一个 1000 个而不是 1001.帮助我解决这个问题。谢谢。

Answer 1

它不适合注释，所以我把我的代码放在这里：

BasicAWSCredentials credentials = new BasicAWSCredentials(accessKey, secretAccessKey);
AmazonS3 s3Client = AmazonS3ClientBuilder
.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("s3-us-west-1.amazonaws.com", region))
.withPathStyleAccessEnabled(true)
.withChunkedEncodingDisabled(true)
.build();
ObjectListing listing = s3Client.listObjects(bucket);
int size = listing.getObjectSummaries().size();
if (size > 0) {
System.out.println("Listing size " + size);
System.out.println("At 0 index " + listing.getObjectSummaries().get(0).getKey());
System.out.println("At " + (size - 1) + " index " + listing.getObjectSummaries().get(size - 1).getKey());
}
while(listing.isTruncated()) {
System.out.println("-----------------------------------------------");
listing = s3Client.listNextBatchOfObjects(listing);
size = listing.getObjectSummaries().size();
if (size > 0) {
System.out.println("Listing size " + size);
System.out.println("At 0 index " + listing.getObjectSummaries().get(0).getKey());
System.out.println("At " + (size - 1) + " index " + listing.getObjectSummaries().get(size - 1).getKey());
}
}

结果如下：

Listing size 1000
At 0 index file0001.txt
At 999 index file1000.txt
-----------------------------------------------
Listing size 1000
At 0 index file1001.txt
At 999 index file2000.txt
-----------------------------------------------
Listing size 1000
At 0 index file2001.txt
At 999 index file3000.txt
-----------------------------------------------
Listing size 100
At 0 index file3001.txt
At 99 index file3100.txt

我对如何输出大小和索引的代码进行了一些更改。顺便说一下，我使用的是 1.11.330 SDK 版本。