火花中的纱线模式和展开模式有什么不同



我现在很困惑。请检查这是否正确。

4种情况命令如下:

# It mean, yarn is cluster mode and deploy cluster mode.
# cluster have YARN Container(have Spark AM, Spark Driver) and YARN node manager.
spark-submit --master yarn --deploy-mode cluster
# It mean, yarn is cluster mode and deploy client mode.
# client have Spark Driver.
# cluster have YARN Container(have Spark AM, Spark Driver) and YARN node manager.
spark-submit --master yarn --deploy-mode client
# It mean, yarn is client mode and deploy cluster mode.
# cluster have YARN Container(have Spark AM) and YARN node manager.
spark-submit --master yarn-client --deploy-mode cluster
# It mean, yarn is client mode and deploy client mode.
# client have Spark Driver.
# cluster have YARN Container(have Spark AM) and YARN node manager.
spark-submit --master yarn-client --deploy-mode client

以上代码的解释是否正确?

#使用yarn,将驱动程序部署到yarn集群中。

spark-submit --master yarn --deploy-mode cluster

#使用纱线,在我的本地机器上部署驱动程序(正在启动代码的机器(

spark-submit --master yarn --deploy-mode client # this is the default if you don't specify --deploy-mode

这些不再是实际的选择,所以它们真的不值得讨论:

spark-submit --master yarn-client --deploy-mode cluster
spark-submit --master yarn-client --deploy-mode client 

--master yarn-client可能是早期版本spark的一个选项,但现在没有使用。(如上述文件中所述(

相关内容

  • 没有找到相关文章

最新更新