我在Windows机器上安装了最新的Hadoop和Spark版本。我试图启动提供的例子之一，但它失败了，我不知道诊断意味着什么。这似乎与标准输出有关，但我无法找出根本原因。
我启动以下命令:
spark-submit --master yarn --class org.apache.spark.examples.JavaSparkPi C:spark-3.0.1-bin-hadoop3.2examplesjarsspark-examples_2.12-3.0.1.jar 100
我的例外是:
21/01/25 10:53:53 WARN MetricsSystem: Stopping a MetricsSystem that is not running .日志含义21/01/25 10:53:53 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!21/01/25 10:53:53 INFO SparkContext: Successfully stopped SparkContext .日志含义线程"main"中的异常org.apache.spark.SparkException: Application application_1611568137841_0002 failed 2 times due to AM Container for appattempt_1611568137841_0002_000002 exitCode: -1尝试失败。诊断:[2010-01-25 10:53:53.381]标准输出路径必须是绝对的
有关更详细的输出，请查看应用程序跟踪页面:http://xxxx-PC:8088/cluster/app/application_1611568137841_0002然后单击指向每次尝试的日志的链接。。应用程序失败。org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication (YarnClientSchedulerBackend.scala: 95)org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start (YarnClientSchedulerBackend.scala: 62)org.apache.spark.scheduler.TaskSchedulerImpl.start (TaskSchedulerImpl.scala: 201)在org.apache.spark.SparkContext。(SparkContext.scala: 555)org.apache.spark.SparkContext .getOrCreate美元(SparkContext.scala: 2574)在org.apache.spark.sql.SparkSession建设者。美元anonfun getOrCreate 2美元(SparkSession.scala: 934)scala.Option.getOrElse (Option.scala: 189)org.apache.spark.sql.SparkSession Builder.getOrCreate美元(SparkSession.scala: 928)在org.apache.spark.examples.JavaSparkPi.main (JavaSparkPi.java: 37)在sun.reflect.NativeMethodAccessorImpl。invoke0(本地方法)sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java: 62)sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java: 43)java.lang.reflect.Method.invoke (Method.java: 498)org.apache.spark.deploy.JavaMainApplication.start (SparkApplication.scala: 52)在org.apache.spark.deploy.SparkSubmit.org apache火花部署SparkSubmit $ $美元美元runMain (SparkSubmit.scala: 928)org.apache.spark.deploy.SparkSubmit.doRunMain 1美元(SparkSubmit.scala: 180)org.apache.spark.deploy.SparkSubmit.submit (SparkSubmit.scala: 203)org.apache.spark.deploy.SparkSubmit.doSubmit (SparkSubmit.scala: 90)在另一次2.美元美元org.apache.spark.deploy.SparkSubmit dosubmit (SparkSubmit.scala: 1007)org.apache.spark.deploy.SparkSubmit .main美元(SparkSubmit.scala: 1016)org.apache.spark.deploy.SparkSubmit.main (SparkSubmit.scala)21/01/25 10:53:53 INFO ShutdownHookManager: Shutdown hook called21/01/25 10:53:53 INFO ShutdownHookManager:删除目录C:UsersxxxAppDataLocalTempspark-b28ecb32-5e3f-4d6a-973a-c03a7aae0da921/01/25 10:53:53 INFO ShutdownHookManager:删除目录C:Users/xxxAppDataLocalTempspark-3665ba77-d2aa-424a-9f75-e772bb5b9104
对于诊断:
诊断:
Application application_1611562870926_0004 failed 2 times due to AM Container for appattempt_1611562870926_0004_000002 exitCode: -1尝试失败。Diagnostics: [2021-01-25 10:29:19.734]Stdout path必须是绝对的要获得更详细的输出，请查看应用程序跟踪页面:http://****-PC:8088/cluster/app/application_1611562870926_0004然后单击指向每次尝试日志的链接。。应用程序失败。
谢谢你!
所以我不确定根本原因，这可能是由于我在windows下运行，一些默认属性对Yarn是错误的。当我在yarn-site.xml上添加以下2个属性时，它工作得很好:
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/tmp</value>
</property>
<property>
<name>yarn.log.dir</name>
<value>/tmp</value>
</property>
希望它能帮助到将来的人!