使用 debezium link postgresql 11 无法获取用于数据库测试的编码



我使用debezium cdc连接pg,并构建了docker使用的pg 11,pg运行良好。当我在kafka连接器中使用debezium时,它报告:

无法获得数据库测试的编码

旋度为:

curl -H "Accept: application/json" -H "Content-type: application/json" -X POST http://localhost:8083/connectors/ -d '{
"name": "debezium",
"config": {
"name": "debezium",
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "localhost",
"database.port": "5432",
"database.dbname": "test",
"database.user": "pg",
"database.password": "135790",
"database.server.name": "ls",
"table.whitelist": "public.test",
"plugin.name": "pgoutput"
}
}'

卡夫卡的例外是:

[2020-07-08 09:24:35,076] ERROR Uncaught exception in REST call to /connectors/ (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper:61)
java.lang.RuntimeException: Couldn't obtain encoding for database test
at io.debezium.connector.postgresql.connection.PostgresConnection.determineDatabaseCharset(PostgresConnection.java:434)
at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:77)
at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:87)
at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:102)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:277)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:534)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:531)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:267)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:216)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: FATAL: database "test" does not exist
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
at org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2644)
at org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:137)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:255)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217)
at org.postgresql.Driver.makeConnection(Driver.java:458)
at org.postgresql.Driver.connect(Driver.java:260)
at io.debezium.jdbc.JdbcConnection.lambda$patternBasedFactory$1(JdbcConnection.java:190)
at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:788)
at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:783)
at io.debezium.connector.postgresql.connection.PostgresConnection.determineDatabaseCharset(PostgresConnection.java:431)
... 13 more
[2020-07-08 09:24:35,128] INFO 127.0.0.1 - - [08/Jul/2020:01:24:34 +0000] "POST /connectors/ HTTP/1.1" 500 73  330 (org.apache.kafka.connect.runtime.rest.RestServer:60)

在我看来,名为test的数据库要么不存在,要么对用户pg不可见。

负载中有一些不准确的地方。保留可解析名称,而不是localhost。其次,放置正确的数据库名称空间。

"database.hostname": "FQDN",
"database.server.name": "test_table_name",

可能是PostgreSQL主机没有验证与pg用户的身份验证的情况。它需要在pg_hba.conf(在PostgreSQL服务器上(中有一个条目,并从客户端机器(即Kafka连接器(建立信任/auth。

# host       DATABASE  USER  ADDRESS                       METHOD  [OPTIONS]
# hostssl    DATABASE  USER  ADDRESS                       METHOD  [OPTIONS]
host         test      pg    Kafka.connector.server.ip/32  md5
hostssl      test      pg    Kafka.connector.server.ip/32  md5

然后,重新启动PostgreSQL服务器,使pg用户的身份验证生效,在我的例子中是pg_ctl reload

正如,curl作为一个REST API调用发生,在PostgreSQL服务器的防火墙设置中添加Kafka-REST(8082(和Kafka-connect-est(8083(端口。

是的,该消息具有不同的误导性。在我的案例中,问题出现在kafka连接和数据库服务器之间的封闭端口中。

最新更新