我目前正在尝试使用管道的JDBCIO步骤从数据流作业连接到SQL Server。我得到以下错误:
2022-12-21T15:06:16.7965222Z SEVERE: 2022-12-21T15:04:31.071Z: java.lang.AbstractMethodError
2022-12-21T15:06:16.7966124Z at net.sourceforge.jtds.jdbc.JtdsConnection.isValid(JtdsConnection.java:2833)
2022-12-21T15:06:16.7967191Z at org.apache.commons.dbcp2.DelegatingConnection.isValid(DelegatingConnection.java:895)
2022-12-21T15:06:16.7968254Z at org.apache.commons.dbcp2.PoolableConnection.validate(PoolableConnection.java:273)
2022-12-21T15:06:16.7969451Z at org.apache.commons.dbcp2.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:644)
2022-12-21T15:06:16.7970625Z at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:106)
2022-12-21T15:06:16.7971833Z at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:652)
2022-12-21T15:06:16.7973522Z at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:534)
2022-12-21T15:06:16.7974830Z at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:734)
2022-12-21T15:06:16.7976091Z at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.processElement(JdbcIO.java:1354)
我在网上发现我需要能够在数据源上设置验证查询，但我不知道如何从数据流作业中做到这一点。下面是我用来设置连接的代码——我可以添加一个验证查询吗?有其他的解决方法吗?
PCollection<Row> Coll = pipeline
.apply("Connect", JdbcIO.<TableRow>read()
.withDataSourceConfiguration(
buildDataSourceConfig(options, URL))
.withQuery(query)
.withRowMapper(new JdbcIO.RowMapper<TableRow>() {
// Convert ResultSet to PCollection
public TableRow mapRow(ResultSet rs) throws Exception {
String ipAddress = rs.getString("IP");
return trOf(ipAddress);
}
}))
private static DataSourceConfiguration buildDataSourceConfig(Options options, String url)
throws Exception {
return DataSourceConfiguration
.create("net.sourceforge.jtds.jdbc.Driver", url)
.withUsername(user)
.withPassword(pass);
}
您可以单独创建DataSource并将其传递给datasourcecconfigurationconstructor (https://beam.apache.org/releases/javadoc/2.0.0/org/apache/beam/sdk/io/jdbc/JdbcIO.DataSourceConfiguration.html)
创建DataSource时，您应该能够设置验证查询:
https://commons.apache.org/proper/commons-dbcp/apidocs/org/apache/commons/dbcp2/BasicDataSource.html setValidationQuery-java.lang.String——