我正在尝试使用flink 1.14的自适应调度来运行基于可用资源的flink作业,而不是等待所需的并行性(缩放),但我没有看到flink正在获得识别自适应调度
Ex: flink run -m yarn-cluster -ynm jobName -p 128 -D jobmanager.scheduler=Adaptive -D cluster.declarative-resource-management.enabled=true -c className JarName
参考:https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/elastic_scaling/
Caused by: java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
at org.apache.flink.runtime.scheduler.DefaultScheduler.lambda$assignResource$8(DefaultScheduler.java:515)
... 37 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
... 35 more
问候,马丹
请参阅文档中描述弹性缩放限制的部分。特别是这部分,说明纱线不支持:
部署只支持作为独立的应用程序部署。不支持活动资源提供程序(如本机Kubernetes, YARN)。也不支持独立会话集群。应用程序部署仅限于单个作业应用程序。
唯一支持的部署选项是单机应用模式,Docker应用模式和单机Kubernetes应用集群。