Python Avro,如何将数据写入修改的模式?



我是Avro的新手,我正在尝试执行基本任务,如从data.avro读取数据,现在我想将数据写入data.avro

我的问题是:...is not an example of the schema...

我不明白我的错误在哪里,谢谢你的帮助:

from avro import schema, datafile, io
from avro.datafile import DataFileWriter, DataFileReader
from avro.io import DatumWriter, DatumReader
OUTFILE_NAME = '4_2.avro'
SCHEMA_STR = """{
"namespace": "advdaba",
"type": "record",
"name": "Conference",
"fields": [
{ "name": "name",       "type": "string" },
{ "name": "time",       "type": "long" },
{ "name": "location",   "type": "string" },
{ "name": "speakers",   "type": {"type":"array","items":"string"} },
{ "name": "participants", "type": {"type": "array", "items": "string"} },
{ "name": "seating", "type": {"type": "map", "values": "int"} }
]
}"""
SCHEMA = schema.parse(SCHEMA_STR)
# read data writtent with the old schema
reader = DataFileReader(open("example.avro", "rb"), DatumReader())
#data = []
for example in reader:
print(example)
#data.append(example)
reader.close()
# generate data for new schema
data = {
'name': 'Foo',
'time': 25612345,
'location': 'Berne',
'speakers': ['Jean', 'Elton'],
'participants': ['John', 'Michel', 'Jacques'],
'seating': [{'John': 1}, {'Michel': 2}, {'Jacques': 3}]
}
rec_writer = io.DatumWriter(SCHEMA)
df_writer = datafile.DataFileWriter(
open(OUTFILE_NAME, 'wb'),
rec_writer,
writers_schema=SCHEMA,
codec='deflate'
)
df_writer.append(data)
df_writer.close()

我认为就我所理解的,甚至可以在同一个.avro文件中写入旧数据和新数据

[编辑]调试后问题来自seating

问题出在seating

解决方案是'seating': {"John": 1, "Michel": 2, "Jacques": 3}

相关内容

  • 没有找到相关文章

最新更新