我正在尝试使用Flink 2.1.0从MySQL日志表读取流数据，但是它只会读取一次，然后它将停止该过程。如果有传入的数据并打印它，我希望它可以阅读。以下是我的代码
public class Database {
public static void main(String[] args) throws Exception {
// get the execution environment
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
TypeInformation[] fieldTypes = new TypeInformation[] { LONG_TYPE_INFO, STRING_TYPE_INFO };
RowTypeInfo rowTypeInfo = new RowTypeInfo(fieldTypes);
DataStreamSource source = env.createInput(
JDBCInputFormat.buildJDBCInputFormat()
.setDrivername("com.mysql.jdbc.Driver")
.setDBUrl("jdbc:mysql://localhost/log_db")
.setUsername("root")
.setPassword("pass")
.setQuery("select id, SERVER_NAME from ERRORLOG")
.setRowTypeInfo(rowTypeInfo)
.finish()
);
source.print().setParallelism(1);
env.execute("Error Log Data");
}
}
我正在使用Maven的本地内部运行：
mvn exec:java -Dexec.mainClass=com.test.Database
结果：
09:15:56,394 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Source: Custom Source (1$
4) (41c66a6dfb97e1d024485f473617a342).
09:15:56,394 INFO org.apache.flink.core.fs.FileSystem - Ensuring all FileSystem streams are closed for Sour$
e: Custom Source (1/4)
09:15:56,394 INFO org.apache.flink.runtime.taskmanager.Task - Sink: Unnamed (1/1) (5212fc2a570152c58ffe3d39d3d805$
0) switched from RUNNING to FINISHED.
09:15:56,394 INFO org.apache.flink.runtime.taskmanager.Task - Freeing task resources for Sink: Unnamed (1/1) (521$
fc2a570152c58ffe3d39d3d805b0).
09:15:56,394 INFO org.apache.flink.runtime.taskmanager.TaskManager - Un-registering task and sending final execution sta$
e FINISHED to JobManager for task Source: Custom Source (41c66a6dfb97e1d024485f473617a342)
09:15:56,396 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: Custom Source (1/4) (41c66a6dfb97e1d024485f$
73617a342) switched from RUNNING to FINISHED.
09:15:56,396 INFO org.apache.flink.runtime.client.JobSubmissionClientActor - 02/22/2017 09:15:56 Source: Custom Source(1/4) swi$
ched to FINISHED
02/22/2017 09:15:56 Source: Custom Source(1/4) switched to FINISHED
09:15:56,396 INFO org.apache.flink.core.fs.FileSystem - Ensuring all FileSystem streams are closed for Sink$
Unnamed (1/1)
09:15:56,397 INFO org.apache.flink.runtime.taskmanager.TaskManager - Un-registering task and sending final execution sta$
e FINISHED to JobManager for task Sink: Unnamed (5212fc2a570152c58ffe3d39d3d805b0)
09:15:56,398 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Sink: Unnamed (1/1) (5212fc2a570152c58ffe3d39d3d805$
0) switched from RUNNING to FINISHED.
09:15:56,398 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Socket Window Data (0eb15d61031ede785e7ed21ead2$
ceea) switched from state RUNNING to FINISHED.
09:15:56,398 INFO org.apache.flink.runtime.client.JobSubmissionClientActor - 02/22/2017 09:15:56 Sink: Unnamed(1/1) switched to
FINISHED
02/22/2017 09:15:56 Sink: Unnamed(1/1) switched to FINISHED
09:15:56,405 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 0eb15d61031$
de785e7ed21ead21ceea
09:15:56,406 INFO org.apache.flink.runtime.client.JobSubmissionClientActor - Terminate JobClientActor.
09:15:56,406 INFO org.apache.flink.runtime.client.JobClient - Job execution complete
09:15:56,408 INFO org.apache.flink.runtime.minicluster.FlinkMiniCluster - Stopping FlinkMiniCluster.
09:15:56,405 INFO org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down
MySQL的表数据已开始查询时已固定，因此该作业应该是flink批处理作业。
如果您想读取传入的数据，如果有传入的数据，Flink将无法处理这种情况，因为Flink不知道传入的数据，除非您监视Binlog。
您必须使用运河将Binlog从MySQL同步到Kafka，并运行来自Kafka的Flink流媒体读取数据。这是最好的解决方案。