当我尝试在Apache Flink独立集群上运行我的流程时,我看到以下异常:
java.lang.IllegalStateException: Update task on instance aaa0859f6af25decf1f5fc1821ffa55d @ app-2 - 4 slots - URL: akka.tcp://flink@192.168.38.98:46369/user/taskmanager failed due to:
at org.apache.flink.runtime.executiongraph.Execution$6.onFailure(Execution.java:954)
at akka.dispatch.OnFailure.internal(Future.scala:228)
at akka.dispatch.OnFailure.internal(Future.scala:227)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:174)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:171)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:28)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:136)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:134)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://flink@192.168.38.98:46369/user/taskmanager#1804590378]] after [10000 ms]
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:333)
at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117)
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597)
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467)
at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419)
at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423)
at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375)
at java.lang.Thread.run(Thread.java:745)
似乎端口 46369 被防火墙阻止。这是真的,因为我阅读了配置部分并仅打开这些端口:
6121:
comment: Apache Flink TaskManager (Data Exchange)
6122:
comment: Apache Flink TaskManager (IPC)
6123:
comment: Apache Flink JobManager
6130:
comment: Apache Flink JobManager (BLOB Server)
8081:
comment: Apache Flink JobManager (Web UI)
flink-conf.yaml
中所述的相同端口:
jobmanager.rpc.address: app-1.stag.local
jobmanager.rpc.port: 6123
jobmanager.heap.mb: 1024
taskmanager.heap.mb: 2048
taskmanager.numberOfTaskSlots: 4
taskmanager.memory.preallocate: false
blob.server.port: 6130
parallelism.default: 4
jobmanager.web.port: 8081
state.backend: jobmanager
restart-strategy: none
restart-strategy.fixed-delay.attempts: 2
restart-strategy.fixed-delay.delay: 60s
所以,我有两个问题:
- 此异常与阻塞的端口有关。右?
- 我应该在防火墙上为独立的 Apache Flink 集群打开哪些端口?
更新 1
我在masters
和slaves
文件中发现了配置问题(我跳过了这些文件中所述的主机之间的新行分隔符)。我修复了它,现在我看到其他异常:
- flink--taskmanager-0-app-1.stag.local.log
- flink--taskmanager-0-app-2.stag.local.log
我有 2 个节点:
- app-1.stag.local(使用正在运行的作业和任务管理器)
- app-2.stag.local(运行任务管理器)
从这些日志中可以看出,app-1.stag.local 任务管理器无法连接到其他任务管理器:
java.io.IOException:连接通道失败:连接到远程任务管理器 + 'app-2.stag.local/192.168.38.98:35806' 失败。这可能表示远程任务管理器已丢失。
但是app-2.stag.local有开放端口:
2016-03-18 16:24:14,347 INFO org.apache.flink.runtime.io.network.netty.NettyServer - 初始化成功(耗时 39 毫秒)。侦听套接字地址/192.168.38.98:35806
所以,我认为问题与防火墙有关,但我不明白我可以在 Apache Flink 中在哪里配置此端口(或端口范围)。
我发现了一个问题:默认情况下taskmanager.data.port
参数设置为 0(但文档说明它应该设置为 6121
)。
因此,我将此端口设置为flink-conf.yaml
,现在一切正常。