多对象日志:IOError

Question

我有一个名为Job的类，它有一个记录器

class MyFileHandler(logging.FileHandler):
    def __init__(self, filename):
        self.filename = filename
        super(MyFileHandler, self).__init__(filename)
    def emit(self, record):
        log_text = self.format(record)
        try:
            fh = open(self.filename, "a")
            fh.write("%sn" % log_text)
            fh.close()
            return True
        except:
            return False
log_formatter = logging.Formatter('br: %(message)s')
class Job(object):
    def __init__(self, name):
        self.name = name
        self.logger = logging.getLogger(self.name)
        log_hdlr = MyFileHandler('/tmp/%s' % name)
        log_hdlr.setFormatter(log_formatter)
        self.logger.addHandler(log_hdlr)
        self.logger.setLevel(logging.INFO)
jobs = []
for i in range(100):
    j = Job(str(i))
    job.append(j)

和作业关闭做一些事情和日志通过job.logger.info()

但是当我有多个任务，也就是数千个任务时，它会抛出错误

IOError: [Errno 24]
    Too many open files:
      '/x/prototype_3885946_1608131132/cm/cm_conv/logs/20160827-195925.log'

我想每次我记录的东西，它会打开然后关闭文件，因为我已经覆盖了emit()

是否有一种模式/方法来拥有数千个记录器?

Answer 1

我猜你的操作系统文件句柄快用完了

Answer 2

仅供参考，而不是直接执行self.logger.info(msg)，我只是将其包装在以下代码周围，该代码打开文件处理程序并在每次写入日志时关闭它。

将self.logger.info(msg)改写为self.write_to_log(msg)

地点:

def write_to_log(self, msg):
    log_hdlr = MyFileHandler('/tmp/%s' % self.name)
    log_hdlr.setFormatter(log_formatter)
    self.logger.addHandler(log_hdlr)
    self.logger.setLevel(logging.INFO)
    self.logger.info(msg)      # <----- actually calling .info() here
    for handler in self.logger.handlers:
        handler.close()
        self.logger.removeHandler(handler)