我有一个将日志写入文件的应用程序，但我想扩展它以包括一些上下文信息，例如JobId。我不想把它添加到我拥有的每一条日志行中，所以我开始研究python日志框架提供了什么——LoggerAdapters
我试着做一个快速的POC，但不确定它是否满足我的用例：
import logging
import threading
import uuid
class MyLoggerAdapter(logging.LoggerAdapter):
def process(self, msg, kwargs):
return "[" + self.extra['JobId'] + "] " + msg, kwargs
class Job:
def __init__(self, logger):
self.job_id = str(uuid.uuid4())
job_id = {"JobId": self.job_id}
self.logger = MyLoggerAdapter(logger, extra=job_id)
def run(self):
self.logger.info("log line here")
if __name__ == '__main__':
# Initialize logger
logging.basicConfig(filename=LOG_FILE_PATH)
logger = logging.getLogger('LoggerAdapterTest')
# Create jobs and run in separate threads
...
记录器需要初始化无法全局使用的"额外"信息。我有两个主要的疑虑：
- 重复实例化LoggerAdapter实例可以吗
- 如何使用现有的框架来构建一个线程安全的记录器，并且可以使用本地函数中存在的一些元数据中的一些上下文信息？(例如：job.get_id(
最好使用带有线程本地的过滤器，如本例所示：
import concurrent.futures
import logging
import random
import threading
import time
import uuid
logger = logging.getLogger()
tld = threading.local()
def filter(record):
record.jobid = tld.jobid
return True
def job(which):
id = str(uuid.uuid4())
tld.jobid = id
count = int(3 + random.random() * 4)
while count > 0:
logger.debug('%d doing work', which)
delay = 1 + random.random() * 4 #delay between 1 and 5 secs
time.sleep(delay)
count -= 1
if __name__ == '__main__':
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(threadName)-10s %(jobid)s %(message)s')
logger.addFilter(filter)
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
for i in range(10):
executor.submit(job, i)
当我运行它时，它打印
2021-11-20 13:19:46,410 ThreadPoolExecutor-0_0 c97f6cff-9a7c-421a-8b7e-87a2dda71f8f 0 doing work
2021-11-20 13:19:46,410 ThreadPoolExecutor-0_1 830d8302-55b0-40b7-aef9-699b0c3b470a 1 doing work
2021-11-20 13:19:46,410 ThreadPoolExecutor-0_2 a49b4cc5-7f62-4956-830a-ca1c77e05c30 2 doing work
2021-11-20 13:19:46,425 ThreadPoolExecutor-0_3 d10b1424-f76f-47f4-b246-83e82c283ec3 3 doing work
2021-11-20 13:19:46,425 ThreadPoolExecutor-0_4 b185f470-0590-4f14-9554-b585b9ae45c6 4 doing work
2021-11-20 13:19:47,949 ThreadPoolExecutor-0_4 b185f470-0590-4f14-9554-b585b9ae45c6 4 doing work
2021-11-20 13:19:48,660 ThreadPoolExecutor-0_2 a49b4cc5-7f62-4956-830a-ca1c77e05c30 2 doing work
2021-11-20 13:19:49,270 ThreadPoolExecutor-0_4 b185f470-0590-4f14-9554-b585b9ae45c6 4 doing work
2021-11-20 13:19:49,946 ThreadPoolExecutor-0_3 d10b1424-f76f-47f4-b246-83e82c283ec3 3 doing work
2021-11-20 13:19:50,525 ThreadPoolExecutor-0_0 c97f6cff-9a7c-421a-8b7e-87a2dda71f8f 0 doing work
2021-11-20 13:19:50,868 ThreadPoolExecutor-0_1 830d8302-55b0-40b7-aef9-699b0c3b470a 1 doing work
2021-11-20 13:19:51,553 ThreadPoolExecutor-0_3 d10b1424-f76f-47f4-b246-83e82c283ec3 3 doing work
2021-11-20 13:19:52,642 ThreadPoolExecutor-0_2 a49b4cc5-7f62-4956-830a-ca1c77e05c30 2 doing work
2021-11-20 13:19:52,814 ThreadPoolExecutor-0_4 b185f470-0590-4f14-9554-b585b9ae45c6 4 doing work
2021-11-20 13:19:53,142 ThreadPoolExecutor-0_1 830d8302-55b0-40b7-aef9-699b0c3b470a 1 doing work
2021-11-20 13:19:53,505 ThreadPoolExecutor-0_3 d10b1424-f76f-47f4-b246-83e82c283ec3 3 doing work
2021-11-20 13:19:54,256 ThreadPoolExecutor-0_2 cee0c924-dae1-4d77-ae5a-36ecbfac2661 5 doing work
2021-11-20 13:19:55,366 ThreadPoolExecutor-0_0 c97f6cff-9a7c-421a-8b7e-87a2dda71f8f 0 doing work
2021-11-20 13:19:55,522 ThreadPoolExecutor-0_3 d10b1424-f76f-47f4-b246-83e82c283ec3 3 doing work
2021-11-20 13:19:55,742 ThreadPoolExecutor-0_4 b185f470-0590-4f14-9554-b585b9ae45c6 4 doing work
2021-11-20 13:19:56,414 ThreadPoolExecutor-0_1 830d8302-55b0-40b7-aef9-699b0c3b470a 1 doing work
2021-11-20 13:19:56,679 ThreadPoolExecutor-0_0 c97f6cff-9a7c-421a-8b7e-87a2dda71f8f 0 doing work
2021-11-20 13:19:56,982 ThreadPoolExecutor-0_2 cee0c924-dae1-4d77-ae5a-36ecbfac2661 5 doing work
2021-11-20 13:19:58,399 ThreadPoolExecutor-0_3 d10b1424-f76f-47f4-b246-83e82c283ec3 3 doing work
2021-11-20 13:19:58,745 ThreadPoolExecutor-0_1 830d8302-55b0-40b7-aef9-699b0c3b470a 1 doing work
2021-11-20 13:19:58,823 ThreadPoolExecutor-0_0 c97f6cff-9a7c-421a-8b7e-87a2dda71f8f 0 doing work
2021-11-20 13:19:59,057 ThreadPoolExecutor-0_2 cee0c924-dae1-4d77-ae5a-36ecbfac2661 5 doing work
2021-11-20 13:19:59,699 ThreadPoolExecutor-0_4 61cb18f5-30d5-4382-bce1-4f1b4f06a2b7 6 doing work
2021-11-20 13:20:01,218 ThreadPoolExecutor-0_2 cee0c924-dae1-4d77-ae5a-36ecbfac2661 5 doing work
2021-11-20 13:20:01,421 ThreadPoolExecutor-0_3 8dc93b28-6bd8-4183-808f-7bd61e8e425c 7 doing work
2021-11-20 13:20:01,483 ThreadPoolExecutor-0_0 7d2f88fa-4d05-40bc-b3c1-607e9c3505bf 8 doing work
2021-11-20 13:20:02,272 ThreadPoolExecutor-0_4 61cb18f5-30d5-4382-bce1-4f1b4f06a2b7 6 doing work
2021-11-20 13:20:02,866 ThreadPoolExecutor-0_1 830d8302-55b0-40b7-aef9-699b0c3b470a 1 doing work
2021-11-20 13:20:03,470 ThreadPoolExecutor-0_2 cee0c924-dae1-4d77-ae5a-36ecbfac2661 5 doing work
2021-11-20 13:20:04,129 ThreadPoolExecutor-0_3 8dc93b28-6bd8-4183-808f-7bd61e8e425c 7 doing work
2021-11-20 13:20:04,706 ThreadPoolExecutor-0_4 61cb18f5-30d5-4382-bce1-4f1b4f06a2b7 6 doing work
2021-11-20 13:20:05,695 ThreadPoolExecutor-0_1 82e44096-19e1-48a7-991a-8e96d6e41a9a 9 doing work
2021-11-20 13:20:05,746 ThreadPoolExecutor-0_0 7d2f88fa-4d05-40bc-b3c1-607e9c3505bf 8 doing work
2021-11-20 13:20:07,190 ThreadPoolExecutor-0_0 7d2f88fa-4d05-40bc-b3c1-607e9c3505bf 8 doing work
2021-11-20 13:20:07,224 ThreadPoolExecutor-0_2 cee0c924-dae1-4d77-ae5a-36ecbfac2661 5 doing work
2021-11-20 13:20:07,872 ThreadPoolExecutor-0_3 8dc93b28-6bd8-4183-808f-7bd61e8e425c 7 doing work
2021-11-20 13:20:07,876 ThreadPoolExecutor-0_4 61cb18f5-30d5-4382-bce1-4f1b4f06a2b7 6 doing work
2021-11-20 13:20:08,658 ThreadPoolExecutor-0_1 82e44096-19e1-48a7-991a-8e96d6e41a9a 9 doing work
2021-11-20 13:20:08,699 ThreadPoolExecutor-0_0 7d2f88fa-4d05-40bc-b3c1-607e9c3505bf 8 doing work
2021-11-20 13:20:10,012 ThreadPoolExecutor-0_4 61cb18f5-30d5-4382-bce1-4f1b4f06a2b7 6 doing work
2021-11-20 13:20:11,311 ThreadPoolExecutor-0_1 82e44096-19e1-48a7-991a-8e96d6e41a9a 9 doing work
2021-11-20 13:20:11,763 ThreadPoolExecutor-0_4 61cb18f5-30d5-4382-bce1-4f1b4f06a2b7 6 doing work
2021-11-20 13:20:12,315 ThreadPoolExecutor-0_3 8dc93b28-6bd8-4183-808f-7bd61e8e425c 7 doing work
2021-11-20 13:20:12,677 ThreadPoolExecutor-0_0 7d2f88fa-4d05-40bc-b3c1-607e9c3505bf 8 doing work
2021-11-20 13:20:13,883 ThreadPoolExecutor-0_1 82e44096-19e1-48a7-991a-8e96d6e41a9a 9 doing work
2021-11-20 13:20:15,333 ThreadPoolExecutor-0_0 7d2f88fa-4d05-40bc-b3c1-607e9c3505bf 8 doing work
2021-11-20 13:20:17,137 ThreadPoolExecutor-0_1 82e44096-19e1-48a7-991a-8e96d6e41a9a 9 doing work
2021-11-20 13:20:17,295 ThreadPoolExecutor-0_3 8dc93b28-6bd8-4183-808f-7bd61e8e425c 7 doing work
2021-11-20 13:20:19,960 ThreadPoolExecutor-0_1 82e44096-19e1-48a7-991a-8e96d6e41a9a 9 doing work
2021-11-20 13:20:21,940 ThreadPoolExecutor-0_3 8dc93b28-6bd8-4183-808f-7bd61e8e425c 7 doing work
请注意线程和作业ID之间的一致性。该池只有5个工人，因此最初显示的是作业0-4，当它们完成时，将显示作业5-9。