Apache bench和python BaseHTTPServer管道中断异常

Question

我有一个用python实现的简单HTTP服务器，如下所示：

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-Type', 'text/html')
        self.end_headers()
        self.wfile.write('Hello World!')
if __name__ == '__main__':
    try:
        server = HTTPServer(('', 8000), Handler)
        print 'Listening on 0.0.0.0:8000'
        server.serve_forever()
    except KeyboardInterrupt:
        print '^C, halting'
        server.socket.close()

然后我运行并发级别为50:的apache bench

ab -v 3 -c 50 -n 50000 http://localhost:8000/

我开始在apache bench中得到这个错误：

apr_socket_recv：对等（104）重置连接

在HTTP服务器控制台上，我看到了以下内容：

Exception happened during processing of request from ('127.0.0.1', 51545)
Traceback (most recent call last):
  File "/usr/lib/python2.6/SocketServer.py", line 281, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 307, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__
    self.handle()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "./test.py", line 12, in do_GET
    self.send_response(200)
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 384, in send_response
    self.send_header('Server', self.version_string())
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 390, in send_header
    self.wfile.write("%s: %srn" % (keyword, value))
  File "/usr/lib/python2.6/socket.py", line 300, in write
    self.flush()
  File "/usr/lib/python2.6/socket.py", line 286, in flush
    self._sock.sendall(buffer)
error: [Errno 32] Broken pipe

我还注意到HTTPServer的真的很慢

我试图通过这样做来增加HTTPServer的request_queue_size：

from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
class MyServer(HTTPServer):
    request_queue_size = 100
    timeout = None

class Handler(BaseHTTPRequestHandler):
    counter = 0
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-Type', 'text/html')
        self.end_headers()
        self.wfile.write(str(self.counter))
        Handler.counter += 1
if __name__ == '__main__':
    try:
        server = MyServer(('', 8000), Handler)
        print 'Listening on 0.0.0.0:8000'
        server.serve_forever()
    except KeyboardInterrupt:
        print '^C, halting'
        server.socket.close()

但我仍然会在测试运行结束时偶尔出现管道破裂的错误。如果我将request_queue_size增加到一个巨大的数字，那么错误就会消失。我不明白为什么，因为Apache工作台-c的值是50，request_queue_size的值应该是100就足够了。为什么request_queue_size需要比Apache工作台并发设置大得多？即使前一个连接还没有完成，Apache工作台是否会触发更多的连接，因此并发连接的总数可能会超过-c设置？

Answer 1

这是apachebench的问题，但不是httperf的问题。我真的不确定他们处理连接的方式有什么不同。

从默认值（5）增加到1000。我没有注意到内存中有任何巨大的漏洞，性能显著提高，同时消除了错误。