在Java中检查S3 bucket是否有键匹配通配符模式的对象

Question

从Scala或Java程序检查S3桶是否有匹配某个键模式的对象的好方法是什么?也就是说，如果我有一个名为"CsvBucket"的桶，我如何检查它是否包含一个对象，其中关键匹配模式"processed/files/2015/8/*.csv"?

谢谢

Answer 1

由于S3对象键只是String，您可以迭代它们并使用正则表达式测试每个键。也许是这样的(使用jets3t库):

Pattern pattern = Pattern.compile(".*\.csv");
// 'service' is an instance of S3Service
S3Bucket bucket = service.getBucket(bucketName);
S3Object[] files = service.listObjects(bucket, "processed/files/2015/8", null);
for (int i = 0; i < files.length; i++)
{
    if (pattern.matches(files[i].getKey()))
    {
        // ... work with the file ...
    }
}

Answer 2

另一种方法-http://docs.aws.amazon.com/AmazonS3/latest/dev/ListingObjectKeysUsingJava.html

import java.io.IOException;
import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.ListObjectsRequest;
import com.amazonaws.services.s3.model.ListObjectsV2Request;
import com.amazonaws.services.s3.model.ListObjectsV2Result;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.S3ObjectSummary;
public class ListKeys {
private static String bucketName = "***bucket name***";
public static void main(String[] args) throws IOException {
    AmazonS3 s3client = new AmazonS3Client(new ProfileCredentialsProvider());
    try {
        System.out.println("Listing objects");
        final ListObjectsV2Request req = new ListObjectsV2Request().withBucketName(bucketName).withMaxKeys(2);
        ListObjectsV2Result result;
        do {               
           result = s3client.listObjectsV2(req);
           for (S3ObjectSummary objectSummary : 
               result.getObjectSummaries()) {
               System.out.println(" - " + objectSummary.getKey() + "  " +
                       "(size = " + objectSummary.getSize() + 
                       ")");
           }
           System.out.println("Next Continuation Token : " + result.getNextContinuationToken());
           req.setContinuationToken(result.getNextContinuationToken());
        } while(result.isTruncated() == true ); 
     } catch (AmazonServiceException ase) {
        System.out.println("Caught an AmazonServiceException, " +
                "which means your request made it " +
                "to Amazon S3, but was rejected with an error response " +
                "for some reason.");
        System.out.println("Error Message:    " + ase.getMessage());
        System.out.println("HTTP Status Code: " + ase.getStatusCode());
        System.out.println("AWS Error Code:   " + ase.getErrorCode());
        System.out.println("Error Type:       " + ase.getErrorType());
        System.out.println("Request ID:       " + ase.getRequestId());
    } catch (AmazonClientException ace) {
        System.out.println("Caught an AmazonClientException, " +
                "which means the client encountered " +
                "an internal error while trying to communicate" +
                " with S3, " +
                "such as not being able to access the network.");
        System.out.println("Error Message: " + ace.getMessage());
    }
}

}