我正在使用 EMR 5.30.0 并尝试使用以下命令提交 Flink (1.10.0( 作业
flink run -m yarn-cluster/home/hadoop/flink--test-0.0.1-SNAPSHOT
.jar并且我收到以下错误：
Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.
在浏览工作器节点和作业管理器日志上的日志后，看起来存在端口冲突
2020-06-17 21:40:51,199 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not start cluster entrypoint YarnJobClusterEntrypoint.
org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint YarnJobClusterEntrypoint.
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:187)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:518)
at org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint.main(YarnJobClusterEntrypoint.java:119)
Caused by: org.apache.flink.util.FlinkException: Could not create the DispatcherResourceManagerComponent.
at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:261)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:215)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:169)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:168)
... 2 more
Caused by: java.net.BindException: Could not start rest endpoint on any port in port range 8081
at org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:165)
... 9 more
似乎为此打开了 JIRA 票证 (https://issues.apache.org/jira/browse/FLINK-15394((尽管它是针对 1.9 版本的 Flink(，建议的解决方案是在 Flink 配置文件中使用rest.bind-port的端口范围。
在 Flink 的 1.10 版本中，我们只遵循 Yan Conf YML 文件
rest.port: 8081
我面临的另一个问题是，我使用 AWS 控制台和添加步骤 UI 提交了多个 Flink 作业(多次提交相同的作业(。只有一个作业成功，其余作业因上面发布的错误而失败。当我转到 Flink UI 时，它根本不显示任何作业。
想知道每个提交的作业是否都尝试创建 Flink Yarn 会话而不是使用现有会话。
谢谢 萨蒂什
我能够解决它。似乎有端口冲突，我必须使用一系列端口并注释掉 rest.port：8081
#rest.port: 8081
rest.bind-port: 50100-50200
谢谢