在创建和删除源连接器时,会创建哪些额外的主题



Q1(以下是我在为MySQL源创建kafka连接器时使用的配置。

{
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"snapshot.locking.mode": "minimal",
"database.user": "cdc_user",
"tasks.max": "3",
"database.history.kafka.bootstrap.servers": "10.49.115.X:9092,10.48.X.211:9092,10.X.178.121:9092,10.53.4.X:9092",
"database.history.kafka.topic": "history.cdc.fkw.supply.mp.seller_platform",
"database.server.name": "cdc.fkw.supply.mp",
"heartbeat.interval.ms": "5000",
"database.port": "3306",
"table.whitelist": "seller_platform.Contacts, seller_platform.EmailVerificationConfigs, seller_platform.financial_account_tag, seller_platform.HolidayConfigs, seller_platform.Preferences, seller_platform.Sellers",
"database.hostname": "something.cloud.in",
"database.password": "ABCDE",
"database.history.kafka.recovery.poll.interval.ms": "5000",
"name": "cdc.fkw.supply.mp.seller_platform.connector",
"database.history.skip.unparseable.ddl": "true",
"errors.tolerance": "all",
"database.whitelist": "seller_platform",
"snapshot.mode": "when_needed"
}

curl -s --location --request GET "http://10.24.18.167:80/connectors/cdc.fkw.supply.mp.seller_platform.connector/topics" | jq '.'
{
"cdc.fkw.supply.mp.seller_platform.connector": {
"topics": [
"cdc.fkw.supply.mp.seller_platform.Sellers",
"cdc.fkw.supply.mp",
"cdc.fkw.supply.mp.seller_platform.HolidayConfigs",
"cdc.fkw.supply.mp.seller_platform.EmailVerificationConfigs",
"cdc.fkw.supply.mp.seller_platform.Contacts",
"cdc.fkw.supply.mp.seller_platform.Preferences",
"__debezium-heartbeat.cdc.fkw.supply.mp",
"cdc.fkw.supply.mp.seller_platform.financial_account_tag"
]
}
}

为什么创建cdc.fkw.supply.mp和__debezium-heartbeat.cdc.fw.supply.msp主题?

我在这两个主题中看到了一些垃圾数据。

Q2(

是否有任何rest api可以了解工作服务器上的kafka-connect转换器配置?若并没有API,那个么存储所有工作者属性的配置文件的路径是什么?

这是工作者属性的链接:https://docs.confluent.io/platform/current/connect/references/allconfigs.html

curl -s --location --request GET "http://10.24.18.167:80"         

{"version":"6.1.1-ccs","commit":"c209f70c6c2e52ae","kafka_cluster_id":"snBlf-kfTdCYWEO9IIEXTA"}%

A1(

心跳主题存储连接器正在使用的所有kafka主题的详细信息,以便连接器可以向其发送心跳。

创建名为topic的database.server.name值是为了存储数据库中发生的任何架构更改。

https://debezium.io/documentation/reference/1.7/connectors/mysql.html#mysql-架构更改主题

最新更新