ClosableHttpClient.ectecute尽管超时,每隔几周就会冻结一次



我们有一个Groovy Singleton,它使用PomingHttpClientConnectionManager(HTTPCLCLIENT:4.3.6),池尺寸为200,以处理与搜索服务的很高的并发连接,并处理XML响应。P>

尽管有指定的超时,但它每月大约冻结一次,但在剩下的时间内都可以很好地运行。

下面的Groovy Singleton。该方法检索froiveInputfromurl似乎在client.ecute(get);

上阻塞
@Singleton(strict=false)
class StreamManagerUtil {
   // Instantiate once and cache for lifetime of Signleton class
   private static PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
   private static CloseableHttpClient client;
   private static final IdleConnectionMonitorThread staleMonitor = new IdleConnectionMonitorThread(connManager);
   private int warningLimit;
   private int readTimeout;
   private int connectionTimeout;
   private int connectionFetchTimeout;
   private int poolSize;
   private int routeSize;
   PropertyManager propertyManager  = PropertyManagerFactory.getInstance().getPropertyManager("sebe.properties")
   StreamManagerUtil() {
      // Initialize all instance variables in singleton from properties file
      readTimeout = 6
      connectionTimeout = 6
      connectionFetchTimeout =6
      // Pooling
      poolSize = 200
      routeSize = 50
      // Connection pool size and number of routes to cache
      connManager.setMaxTotal(poolSize);
      connManager.setDefaultMaxPerRoute(routeSize);
      // ConnectTimeout : time to establish connection with GSA
      // ConnectionRequestTimeout : time to get connection from pool
      // SocketTimeout : waiting for packets form GSA
      RequestConfig config = RequestConfig.custom()
      .setConnectTimeout(connectionTimeout * 1000)
      .setConnectionRequestTimeout(connectionFetchTimeout * 1000)
      .setSocketTimeout(readTimeout * 1000).build();
      // Keep alive for 5 seconds if server does not have keep alive header
      ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
         @Override
         public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
            HeaderElementIterator it = new BasicHeaderElementIterator
               (response.headerIterator(HTTP.CONN_KEEP_ALIVE));
            while (it.hasNext()) {
               HeaderElement he = it.nextElement();
               String param = he.getName();
               String value = he.getValue();
               if (value != null && param.equalsIgnoreCase
                  ("timeout")) {
                  return Long.parseLong(value) * 1000;
               }
            }
            return 5 * 1000;
         }
      };
      // Close all connection older than 5 seconds. Run as separate thread.
      staleMonitor.start();
      staleMonitor.join(1000);
      client = HttpClients.custom().setDefaultRequestConfig(config).setKeepAliveStrategy(myStrategy).setConnectionManager(connManager).build();
   }     
   private retrieveInputFromURL (String categoryUrl, String xForwFor, boolean isXml) throws Exception {
      URL url = new URL( categoryUrl );
      GPathResult searchResponse = null
      InputStream inputStream = null
      HttpResponse response;
      HttpGet get;
      try {
         long startTime = System.nanoTime();
         get = new HttpGet(categoryUrl);
         response =  client.execute(get);
         int resCode = response.getStatusLine().getStatusCode();
         if (xForwFor != null) {
            get.setHeader("X-Forwarded-For", xForwFor)
         }
         if (resCode == HttpStatus.SC_OK) {
            if (isXml) {
               extractXmlString(response)
            } else {
               StringBuffer buffer = buildStringFromResponse(response)
               return buffer.toString();
            }
         }
      }
      catch (Exception e)
      {
         throw e;
      }
      finally {
         // Release connection back to pool
         if (response != null) {
            EntityUtils.consume(response.getEntity());
         }
      }
   }
   private extractXmlString(HttpResponse response) {
      InputStream inputStream = response.getEntity().getContent()
      XmlSlurper slurper = new XmlSlurper()
      slurper.setFeature("http://xml.org/sax/features/validation", false)
      slurper.setFeature("http://apache.org/xml/features/disallow-doctype-decl", false)
      slurper.setFeature("http://apache.org/xml/features/nonvalidating/load-dtd-grammar", false)
      slurper.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false)
      return slurper.parse(inputStream)
   }
   private StringBuffer buildStringFromResponse(HttpResponse response) {
      StringBuffer buffer= new StringBuffer();
      BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
      String line = "";
      while ((line = rd.readLine()) != null) {
         buffer.append(line);
         System.out.println(line);
      }
      return buffer
   }
public class IdleConnectionMonitorThread extends Thread {
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    public IdleConnectionMonitorThread
      (PoolingHttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    connMgr.closeExpiredConnections();
                    connMgr.closeIdleConnections(10, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // Ignore
        }
    }
    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
}

我还发现这是在日志中发现的,这使我相信它在等待响应数据

时发生

java.net.sockettimeoutexception:在Java.net.socketinputstream.socketRead0(本机方法)上阅读Java.net.net.socketInputSocketInputstream.java.java.java:150)的时间Java:121)在Sun.security.ssl.inputrecord..readly(inputrecord.java:465)

到目前为止的发现:

  • 我们正在使用Java 1.8U25。在类似情况下有一个空旷的问题https://bugs.openjdk.java.net/browse/jdk-8075484
  • httpclient也有类似的报告https://issues.apache.org/jira/browse/httpclient-1589,但这是修复的我们正在使用的4.3.6版本

问题

  • 这可以是同步问题吗?从我的理解中,即使多个线程访问了单顿,唯一共享的数据是缓存的CLASEABLEHTTPCLEINT
  • 此代码从根本上有其他错误,可能导致这种行为的方法?

我看不到您的代码明显错误。但是,我强烈建议在连接管理器上设置SO_TIMEOUT参数,以确保它适用于创建时间的所有新套接字,而不是在请求执行时。

我也将帮助知道"冻结"的含义。工人线程是否被阻止等待从池中获得连接或等待响应数据?

另外,如果服务器不断发送块编码数据,请注意,工作人员线程可能会出现"冷冻"。像往常一样http://hc.apache.org/httpcomponents-client-4.3.x/logging.html

最新更新