列'nextfetchdate'的风暴爬网程序 SQL 错误



我的设置与此相同。当我在爬网模式下运行爬虫时,出现以下错误:

[Thread-130-status-executor[109 109]] 错误 c.d.s.p.AbstractStatusUpdaterBolt - 存储时捕获异常 com.mysql.jdbc.MysqlDataTruncation: 数据截断: 第 1 行的"nextfetchdate"列的日期时间值不正确:"2099-12-31 00:00:00" at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3964( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1192( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.digitalpebble.stormcrawler.sql.StatusUpdaterBolt.store(StatusUpdaterBolt.java:132( ~[stromcrawler-1.0-SNAPSHOT.jar:?] at com.digitalpebble.stormcrawler.persistence.AbstractStatusUpdaterBolt.execute(AbstractStatusUpdaterBolt.java:196( [stromcrawler-1.0-SNAPSHOT.jar:?] at org.apache.storm.daemon.executor$fn__5043$tuple_action_fn__5045.invoke(executor.clj:739( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.daemon.executor$mk_task_receiver$fn__4964.invoke(executor.clj:468( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:74( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.daemon.executor$fn__5043$fn__5056$fn__5109.invoke(executor.clj:861( [storm-core-1.2.1.jar:1.2.1] at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484( [storm-core-1.2.1.jar:1.2.1] at clojure.lang.AFn.run(AFn.java:22( [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745( [?:1.8.0_121]

请建议在哪里寻找修复它

这是由于错误的获取间隔已设置为 -1,这意味着"永不重新访问"。这实际上是由 DefaultScheduler 转换为一个遥远的未来日期。不清楚为什么mysql会窒息。您可以尝试设置一个更合理的值,例如 43200 一个月,看看是否有效。

相关内容

最新更新