Kafka发送值作为一个字符串:我如何用Python反序列化它并把它变成一个JSON对象

Question

我正在尝试从lambda函数中消费MSK(托管的Amazon Kafka服务)消息- MSK是我的lambda的触发器。

生产者是这样的:

data = {'time': 1611215510000000000, 'tags': {'tag1': 'tagvalue'}, 'fields': {'value': 12345}}
self.producer = KafkaProducer(
security_protocol=self.security_protocol,
bootstrap_servers=self.kafka_servers,
value_serializer=lambda x: dumps(x).encode('utf-8'))
self.producer.send(kafka_topic, value=data)

在lambda函数中，我收到以下内容:

{
"eventSource":"aws:kafka",
"eventSourceArn":"<arn....>",
"bootstrapServers":"<serverlist...>",
"records":{
"topic-0":[
{
"topic":"topic",
"partition":0,
"offset":0,
"timestamp":1611138328871,
"timestampType":"CREATE_TIME",
"value":"eyJ0aW1lIjogMTYxMTEzODI4MDAwMDAwMDAwMCwgInRhZ3MiOiB7InN0YXR1cyI6ICJHb29kIn0sICJmaWVsZHMiOiB7InZhbHVlX251bSI6IDAuMCwgInZhbHVlIjogZmFsc2V9fQ=="
},
{
"topic":"topic",
"partition":0,
"offset":1,
"timestamp":1611138330033,
"timestampType":"CREATE_TIME",
"value":"eyJ0aW1lIjogMTYxMTEzODI4MDAwMDAwMDAwMCwgInRhZ3MiOiB7InN0YXR1cyI6ICJHb29kIn0sICJmaWVsZHMiOiB7InZhbHVlIjogMTQxMzUuMH19"
}
]
}
}

我想将值字符串转换为JSON对象。我怎么能做到呢?我尝试了很多版本，我认为应该工作抛出一个异常(Exception: Expecting value: line 1 column 1 (char 0))

records = event['records']['topic-0']
for record in records:
print(json.loads(record['value']).decode('utf-8'))

Answer 1

值字符串似乎是base64编码的，所以您需要找到一种方法来解码它们。然后你可以加载它们。

使用https://www.base64decode.org/:

解码的第一个字符串

{"time": 1611138280000000000, "tags": {"status": "Good"}, "fields": {"value_num": 0.0, "value": false}}