KafkaConnect和Debezium正在同一模式下为Key和Value创建模式

Question

Am使用Debezium MySQL连接器将更改从数据库流式传输到Kafka。我想使用AWS Glue模式注册表来存储模式。下面的配置是以相同的模式名称"为Key和Value创建模式；database1.tutorial.movies"；根据注册处"；msk-cdc"；作为两个不同的版本。默认情况下，我期望为"创建不同的模式；database1.tutorial.movies键"；以及"；database1.tutorial.movies Value"；。我在这里做错了什么？

name=debezium-mysql-sample
connector.class=io.debezium.connector.mysql.MySqlConnector
database.hostname=<hostname>
database.port=3306
database.user=<username>
database.password=<passowrd>
database.server.id=42
database.server.name=database1
table.whitelist=tutorial.movies
database.history.kafka.bootstrap.servers=<bootstrap-server>
database.history.kafka.topic=dbhistory.demo1
key.converter=com.amazonaws.services.schemaregistry.kafkaconnect.AWSKafkaAvroConverter
value.converter=com.amazonaws.services.schemaregistry.kafkaconnect.AWSKafkaAvroConverter
key.converter.compressionType=NONE
value.converter.compressionType=NONE
key.converter.endpoint=https://glue.us-east-1.amazonaws.com
value.converter.endpoint=https://glue.us-east-1.amazonaws.com
key.converter.region=us-east-1
value.converter.region=us-east-1
key.converter.timeToLiveMillis=3600000
value.converter.timeToLiveMillis=3600000
key.converter.cacheSize=100
value.converter.cacheSize=100
key.converter.avroRecordType=GENERIC_RECORD
value.converter.avroRecordType=GENERIC_RECORD
key.converter.registry.name=CDC
value.converter.registry.name=CDC
key.converter.compatibility=NONE
value.converter.compatibility=NONE
key.converter.description=none
value.converter.description=none
key.converter.schemaAutoRegistrationEnabled=true
value.converter.schemaAutoRegistrationEnabled=true
transforms=unwrap
transforms.unwrap.type=io.debezium.transforms.ExtractNewRecordState
transforms.unwrap.drop.tombstones=false
transforms.unwrap.delete.handling.mode=rewrite
transforms.unwrap.add.fields=op,source.ts_ms

Answer 1

您可能需要考虑设置特定的命名策略。

默认情况下，它似乎只使用主题名称，不区分键和值。

相关问题-https://github.com/awslabs/aws-glue-schema-registry/issues/93
序列化程序源-https://github.com/awslabs/aws-glue-schema-registry/blob/master/serializer-deserializer/src/main/java/com/amazonaws/services/schemaregistry/serializers/avro/AWSKafkaAvroSerializer.java#L148

此外，理想情况下，您的记录密钥应该只是数据库的主键类型，因此String、Int或Long Converter将比Avro更合适，在这种情况下是