imaplib2: imap.gmail.com handler BYE响应:系统错误

Question

我正在更新一个python脚本，该脚本检查IMAP是否有新邮件，并在有新邮件时发送推送通知。问题是每隔几个小时我就会崩溃一次。一开始我真的不明白发生了什么，但后来我发现M.debug = 4给了我一个很好的输出，但我仍然不明白是什么导致了这个问题。我已经发布了我的脚本和调试输出从正常的行为，直到崩溃，希望有人在python更好的理解可以告诉我是怎么回事，我怎么能修复它。

编辑:

我已经按照这个问题的答案编辑了我的代码:

while True:
    try:
      [call function that does all your logic]
    except imaplib2.IMAP4.abort:
      print("Disconnected.  Trying again.")

或:

    while True:
        try:
          [call function that does all your logic]
except (imaplib2.IMAP4.abort, imaplib2.IMAP4.error) as e: 
      print("Disconnected.  Trying again.")

但是在一段不确定的时间后，我仍然会崩溃，并且异常永远不会被捕获。print("Disconnected. Trying again.")将永远不会执行。

代码:

    #!/usr/local/bin/python2.7
    print "Content-type: text/htmlrnrn";
    import socket, ssl, json, struct, re
    import imaplib2, time
    from threading import *
    # enter gmail login details here
    USER="[e-mail]"
    PASSWORD="password"
    # enter device token here
    deviceToken = 'my device token x x x x x'
    deviceToken = deviceToken.replace(' ','').decode('hex')
    currentBadgeNum = -1
    def getUnseen():
        (resp, data) = M.status("INBOX", '(UNSEEN)')
        print data
        return int(re.findall("UNSEEN (d)*)", data[0])[0])    
    def sendPushNotification(badgeNum):
        global currentBadgeNum, deviceToken
        if badgeNum != currentBadgeNum:
            currentBadgeNum = badgeNum
            thePayLoad = {
                 'aps': {
                      'alert':'Hello world!',
                      'sound':'',
                      'badge': badgeNum,
                      },
                 'test_data': { 'foo': 'bar' },
                 }
            theCertfile = 'certif.pem'
            theHost = ('gateway.push.apple.com', 2195)
            data = json.dumps(thePayLoad)
            theFormat = '!BH32sH%ds' % len(data)
            theNotification = struct.pack(theFormat, 0, 32, deviceToken, len(data), data)
            ssl_sock = ssl.wrap_socket(socket.socket(socket.AF_INET, socket.SOCK_STREAM), certfile=theCertfile)
            ssl_sock.connect(theHost)
            ssl_sock.write(theNotification)
            ssl_sock.close()
            print "Sent Push alert."
    # This is the threading object that does all the waiting on 
    # the event
    class Idler(object):
        def __init__(self, conn):
            self.thread = Thread(target=self.idle)
            self.M = conn
            self.event = Event()
        def start(self):
            self.thread.start()
        def stop(self):
            # This is a neat trick to make thread end. Took me a 
            # while to figure that one out!
            self.event.set()
        def join(self):
            self.thread.join()
        def idle(self):
            # Starting an unending loop here
            while True:
                # This is part of the trick to make the loop stop 
                # when the stop() command is given
                if self.event.isSet():
                    return
                self.needsync = False
                # A callback method that gets called when a new 
                # email arrives. Very basic, but that's good.
                def callback(args):
                    if not self.event.isSet():
                        self.needsync = True
                        self.event.set()
                # Do the actual idle call. This returns immediately, 
                # since it's asynchronous.
                self.M.idle(callback=callback)
                # This waits until the event is set. The event is 
                # set by the callback, when the server 'answers' 
                # the idle call and the callback function gets 
                # called.
                self.event.wait()
                # Because the function sets the needsync variable,
                # this helps escape the loop without doing 
                # anything if the stop() is called. Kinda neat 
                # solution.
                if self.needsync:
                    self.event.clear()
                    self.dosync()
        # The method that gets called when a new email arrives. 
        # Replace it with something better.
        def dosync(self):
            print "Got an event!"
            numUnseen = getUnseen()
            sendPushNotification(numUnseen)
    # Had to do this stuff in a try-finally, since some testing 
    # went a little wrong.....
while True:
    try:
        # Set the following two lines to your creds and server
        M = imaplib2.IMAP4_SSL("imap.gmail.com")
        M.login(USER, PASSWORD)
        M.debug = 4
        # We need to get out of the AUTH state, so we just select 
        # the INBOX.
        M.select("INBOX")
        numUnseen = getUnseen()
        sendPushNotification(numUnseen)
        typ, data = M.fetch(1, '(RFC822)')
        raw_email = data[0][1]
        import email
        email_message = email.message_from_string(raw_email)
        print email_message['Subject']
        #print M.status("INBOX", '(UNSEEN)')
        # Start the Idler thread
        idler = Idler(M)
        idler.start()

        # Sleep forever, one minute at a time
        while True:
            time.sleep(60)
    except imaplib2.IMAP4.abort:
      print("Disconnected.  Trying again.")   
    finally:
        # Clean up.
        #idler.stop() #Commented out to see the real error
        #idler.join() #Commented out to see the real error
        #M.close()    #Commented out to see the real error
        # This is important!
        M.logout()

………

  43:54.43 imap.gmail.com handler _request_pop(continuation, (True, 'idling')) = CHPJ127
  43:54.43 imap.gmail.com handler None:CHPJ127.ready.set
  43:54.43 Thread-4 continuation => True, idling
  43:54.43 Thread-4 server IDLE started, timeout in 1740.00 secs
  43:54.43 Thread-4 state_change_pending.release
  57:13.60 imap.gmail.com reader < * BYE System errorrn
  57:13.63 imap.gmail.com handler server IDLE finished
  57:13.63 imap.gmail.com handler BYE response: System error
  57:13.63 imap.gmail.com writer > DONErn
  **57:13.63 imap.gmail.com handler terminating: 'connection terminated'**
  57:13.63 imap.gmail.com writer finished
  57:13.63 imap.gmail.com handler last 20 log messages:
  51:49.77 Thread-4 [sync] IDLE ()
  20:50.18 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  20:50.51 Thread-4 [sync] IDLE ()
  49:50.79 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  49:51.02 Thread-4 [sync] IDLE ()
  18:51.33 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  18:51.49 Thread-4 [sync] IDLE ()
  47:51.80 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  47:51.96 Thread-4 [sync] IDLE ()
  16:52.26 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  16:52.63 Thread-4 [sync] IDLE ()
  45:53.08 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  45:53.24 Thread-4 [sync] IDLE ()
  14:53.54 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  14:53.69 Thread-4 [sync] IDLE ()
  43:53.96 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  43:54.17 Thread-4 [sync] IDLE ()
  57:13.63 imap.gmail.com handler BYE response: System error
  57:13.63 imap.gmail.com handler terminating: 'connection terminated'
  57:13.63 imap.gmail.com writer finished
Got an event!
  57:13.63 imap.gmail.com handler state_change_free.set
  57:13.63 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  57:13.63 imap.gmail.com handler finished
  57:13.63 Thread-4 state_change_pending.acquire
  57:13.63 Thread-4 state_change_pending.release
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "shaserver.py", line 111, in idle
    self.dosync()
  File "shaserver.py", line 117, in dosync
    numUnseen = getUnseen()
  File "shaserver.py", line 35, in getUnseen
    (resp, data) = M.status("INBOX", '(UNSEEN)')
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1121, in status
    return self._simple_command(name, mailbox, names, **kw)
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1607, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1295, in _command
    self._check_bye()
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1229, in _check_bye
    raise self.abort(bye[-1])
abort: System error
  57:13.70 imap.gmail.com reader finished

………

有时我得到这个:

03:09.29 Thread-4 [sync] IDLE ()
  05:53.25 imap.gmail.com reader socket error: <type 'exceptions.IOError'> - Error Hang up
  05:53.25 imap.gmail.com reader finished
  05:53.26 imap.gmail.com handler terminating: "socket error: <type 'exceptions.IOError'> - Error Hang up"
  05:53.26 imap.gmail.com handler last 20 log messages:
  07:07.66 Thread-4 [sync] IDLE ()
  36:07.78 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  36:07.83 Thread-4 [async] SEARCH ('ALL',)
  36:07.88 Thread-4 [async] FETCH ('1', '(RFC822.HEADER)')
  36:08.09 Thread-4 [sync] IDLE ()
  05:08.19 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  05:08.25 Thread-4 [async] SEARCH ('ALL',)
  05:08.42 Thread-4 [async] FETCH ('1', '(RFC822.HEADER)')
  05:08.48 Thread-4 [sync] IDLE ()
  34:08.58 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  34:08.68 Thread-4 [async] SEARCH ('ALL',)
  34:08.79 Thread-4 [async] FETCH ('1', '(RFC822.HEADER)')
  34:08.94 Thread-4 [sync] IDLE ()
  03:09.05 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  03:09.16 Thread-4 [async] SEARCH ('ALL',)
  03:09.21 Thread-4 [async] FETCH ('1', '(RFC822.HEADER)')
  03:09.29 Thread-4 [sync] IDLE ()
  05:53.25 imap.gmail.com reader socket error: <type 'exceptions.IOError'> - Error Hang up
  05:53.25 imap.gmail.com reader finished
  05:53.26 imap.gmail.com handler terminating: "socket error: <type 'exceptions.IOError'> - Error Hang up"
  05:53.26 imap.gmail.com writer finished
Got an event!
  05:53.26 Thread-4 [async] STATUS ('INBOX', '(UNSEEN)')
  05:53.26 Thread-4 state_change_pending.acquire
  05:53.26 Thread-4 server IDLE finished
  05:53.26 Thread-4 state_change_pending.release
  05:53.26 Thread-4 _get_untagged_response(READ-ONLY) => ['']
  05:53.26 imap.gmail.com handler state_change_free.set
  05:53.26 imap.gmail.com handler finished
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "shaserver.py", line 229, in idle
    self.dosync()
  File "shaserver.py", line 235, in dosync
    numUnseen = getUnseen()
  File "shaserver.py", line 150, in getUnseen
    (resp, data) = M.status("INBOX", '(UNSEEN)')
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1121, in status
    return self._simple_command(name, mailbox, names, **kw)
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1607, in _simple_command
    return self._command_complete(self._command(name, *args), kw)
  File "/home/boombe/lib/python2.7/imaplib2/imaplib2.py", line 1305, in _command
    raise self.abort('connection closed')
abort: connection closed

Answer 1

这是因为异常是在另一个线程中抛出的，并且在另一个不受except块保护的地方。只需查看反向跟踪—您将看到它在threading.py中开始(并且那里没有异常处理)。继续通过dosync和getUnseen方法——同样，没有异常处理。

所以这就解释了为什么你的try/except块无效——那是因为你明确地要求在另一个线程中执行该代码。需要解释的是，为什么首先会得到这个异常。一个原因可能是您正在达到超时时间，这通常与IDLE命令相关——详细信息请参阅RFC 2177(这是一个简短的阅读)，它将告诉您应该每29分钟打破您的空闲时间。另一种可能是，gmail根本不喜欢你——也许他们不喜欢维持与你联系太久的客户。不过，我没有数据来支持这种猜测。

但是最重要的问题仍然存在——为什么要使用线程呢?作为一个从头实现过IMAP客户端的人(顺便说一句)。(它开始于许多年前的Python)，我完全确定线程是而不是有效地谈论IMAP所必需的。在您的特定用例中，您似乎只是为IDLE命令运行一个线程，而将主线程的时间花在无休止的wait上。这本身是我不理解的，但也许你有一些重要的原因——也许你真的需要将一些IMAP处理卸载到另一个线程。但是，在这种情况下，请考虑将任何 IMAP相关活动移动到该线程中——因为现在，您的一个线程正在执行初始SELECT和一些数据获取，只有在此之后，它才会将IMAP连接传输到IDLE的另一个线程。这是邪恶的。

对于线程的基本需求——一个对我来说很有效的架构是利用IMAP协议的异步特性。我不确定它在imaplib2库中的表现如何，也许它不支持真正的异步IMAP处理。在这种情况下，我强烈考虑以单调、阻塞的同步方式使用IMAP，其中每个操作都真正等待命令的完成。是的，它们中的一些会运行很长时间——如果你绝对需要执行一些活动，而你正在等待IDLE在29分钟后完成，那么将IMAP处理推迟到另一个线程可能是一个好主意。但是在你这样做之前，你应该非常确定这是你真正需要的。

Answer 2

我一直在从头开始构建相同的应用程序，并收到完全相同的错误(BYE响应例外)。

如何在Thread-4中捕获异常

这可以通过包装

来解决

self.M.idle(callback=callback)

的try except，然后保存异常的地方(例如在Idler的属性)。

这个保存的异常在主线程中不断被扫描(即每秒钟睡眠和唤醒)，如果存在，则在主线程中引发以正确处理(即根据imaplib2库中的指示重新创建imap连接)。

如何避免服务器的BYE响应

为什么会发生这种情况?我的猜测是，当连接被不干净地关闭时(例如，使用kill信号，这样就没有清理完成)，服务器之前没有从连接接收NOOP信号，并决定关闭通道。但是现在是新的程序接收"关闭通道"信号。

我为什么要推测这个?一旦我实现了BYE imaplib2.abort异常捕获和kill信号捕获(SIGTERM等)，imaplib2.abort异常就完全停止了。

---

注意:您也可以在我的github中找到这两个解决方案的确切实现:https://www.github.com/Elijas/email-notifier

Answer 3

从您的跟踪中，它看起来远程端向您发送了一个未经请求的BYE命令，因为它想要关闭连接出于某种原因。

你可能需要让你的脚本更健壮一些来处理连接失败和BYE命令。

例如，你应该把你的顶层改成一个循环:

  while True:
    try:
      [call function that does all your logic]
    except imaplib2.IMAP4.abort:
      print("Disconnected.  Trying again.")

同样，您需要更新callback()来查看它的参数。如果IDLE返回一个错误，因为您异步使用它，它将向callback()报告错误。您将需要在那里处理异常。(注意它不会在你的回调中引发错误，只是在参数中返回错误。

来自imaplib2s文档:

If 'callback' is provided then the command is asynchronous, so after
the command is queued for transmission, the call returns immediately
with the tuple (None, None).
The result will be posted by invoking "callback" with one arg, a tuple:
callback((result, cb_arg, None))
or, if there was a problem:
callback((None, cb_arg, (exception class, reason)))

这意味着你的回调需要查看它的参数:

            def callback(args):
                result, arg, exc = args
                if result is None:
                    print("There was an error during IDLE:", str(exc))
                    self.error = exc
                    self.event.set()
                else:
                    self.needsync = True
                    self.event.set()

现在您可以检查您的空闲线程中是否有错误。然后在你的主线程中，你坐下来每次永远睡60秒，你可以设置一些标志来指示你是否已经断开连接。