Amazon S3服务ConnectionPoolTimeoutException:等待连接池超时

Question

当我在Amazon上使用s3时，我一直得到丑陋的ConnectionPoolTimeoutException。

问题是，由于前端的应用程序，我不能关闭打开s3对象之前前端完成与他们，所以我已经实现了这个解决方案:

@Autowired
private AmazonS3 s3client; // credentials are set properly.
private static List<S3Object> openedObjects = new ArrayList<S3Object>();
// initialize bucket :
private String bucketName = "myShinyNewBucket";
private synchronized boolean initBucket(){
    try{
        Boolean exists = null;
        try{
            exists = s3client.doesBucketExist(bucketName);
        }catch(Exception e1){
            System.out.println("nntToo many opened objects ; closing...nn");
            deleteOpenedS3Objects();
            exists = s3client.doesBucketExist(bucketName);
        }
        if(exists!=null){
            if(!exists){
                s3client.createBucket(new CreateBucketRequest(bucketName));
            }
            return true;
        }
    }
    catch(Exception e){
        System.out.println("nntFailed to initialize bucket.n");
        e.printStackTrace();
    }
    return false;
}
private synchronized void deleteOpenedS3Objects(){
    System.out.println("ntClosing opened objects...");
    try{
        for(int i=0 ; i<openedObjects.size() ; i++){
            openedObjects.get(i).close();
            openedObjects.remove(i);
        }
    }catch(Exception e1){
        System.out.println("tCould not close all opened s3 objects, only the first "+i);
    }
    System.out.println("tTrying again :nn");
}
// GET :
public final String getFromAWS(final String amazonName){
    S3Object s3object = null;
    if(initBucket()){
        try{
            try{
                s3object = s3client.getObject(new GetObjectRequest(bucketName, amazonName));
            }catch(AmazonClientException e){
                deleteOpenedS3Objects();
                s3object = s3client.getObject(new GetObjectRequest(bucketName, amazonName));
            }
            openedObjects.add(s3object);
            return s3object.getObjectContent().getHttpRequest().getURI().toString();
        }catch(Exception e1){
            if (((AmazonS3Exception)e1).getStatusCode() == HttpStatus.SC_NOT_FOUND){
                System.out.println("nnNo such object in bucket.n");
            }
            else{
                System.out.println("nntCould not read bject from bucket.nn");
                e1.printStackTrace();
            }
        }
    }
    return null;
}

但是，异常仍然发生。

org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:286) ~[httpclient-4.5.1.jar!/:4.5.1]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar!/:4.5.1]
        at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
        at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) ~[aws-java-sdk-core-1.11.8.jar!/:na]
        at com.amazonaws.http.conn.$Proxy188.get(Unknown Source) ~[na:na]
...

只有当我在控制台中按ctrl+c时，它才会到达关闭打开的s3连接的部分:

... Caused by: java.lang.InterruptedException: Operation interrupted
        at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:142) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276) ~[httpclient-4.5.1.jar!/:4.5.1]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar!/:4.5.1]
        at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
        at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) ~[aws-java-sdk-core-1.11.8.jar!/:na]
        at com.amazonaws.http.conn.$Proxy188.get(Unknown Source) ~[na:na]
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190) ~[httpclient-4.5.1.jar!/:4.5.1]
        ... 148 common frames omitted

        Too many opened objects. // <-- This is where it catches it.

        Closing opened objects...
        Could not close all opened s3 objects.
        Trying again :


        Failed to initialize bucket.

再次，不幸的是，在我离开s3客户端类中的函数之前，我不能关闭打开的s3对象。我唯一的希望是等到TimeoutException发生，捕获它，然后关闭所有打开的对象并再次尝试。但是，我总是找不到合适的地方。

请帮助。

谢谢。

Answer 1

无论何时处理任何类型的文件读写，都必须对资源使用try。这将自动关闭所使用的资源，即使我们面临任何类型的异常。

在我的情况下:我在从S3读取批量操作时得到404，我在编写脚本时忽略了记录错误并忘记在最后块关闭连接。

示例代码片段:

static String readFirstLineFromFile(String path) throws IOException {
    try (BufferedReader br =
                   new BufferedReader(new FileReader(path))) {
        return br.readLine();
    }
}

注意:

1. 尝试使用JDK 8及以上版本的资源。
2. 所有在try()中声明的资源必须实现AutoCloseable接口。

Answer 2

我认为解决方案是在我的@ControllerAdvice类中捕获和处理丑陋的TimeoutException。我已经这样做了，到目前为止，我还没有发生在应用程序。我会发布确认一旦我确定。