我现在很困惑。请检查这是否正确。
4种情况命令如下:
# It mean, yarn is cluster mode and deploy cluster mode.
# cluster have YARN Container(have Spark AM, Spark Driver) and YARN node manager.
spark-submit --master yarn --deploy-mode cluster
# It mean, yarn is cluster mode and deploy client mode.
# client have Spark Driver.
# cluster have YARN Container(have Spark AM, Spark Driver) and YARN node manager.
spark-submit --master yarn --deploy-mode client
# It mean, yarn is client mode and deploy cluster mode.
# cluster have YARN Container(have Spark AM) and YARN node manager.
spark-submit --master yarn-client --deploy-mode cluster
# It mean, yarn is client mode and deploy client mode.
# client have Spark Driver.
# cluster have YARN Container(have Spark AM) and YARN node manager.
spark-submit --master yarn-client --deploy-mode client
以上代码的解释是否正确?
#使用yarn,将驱动程序部署到yarn集群中。
spark-submit --master yarn --deploy-mode cluster
#使用纱线,在我的本地机器上部署驱动程序(正在启动代码的机器(
spark-submit --master yarn --deploy-mode client # this is the default if you don't specify --deploy-mode
这些不再是实际的选择,所以它们真的不值得讨论:
spark-submit --master yarn-client --deploy-mode cluster
spark-submit --master yarn-client --deploy-mode client
--master yarn-client
可能是早期版本spark的一个选项,但现在没有使用。(如上述文件中所述(