线程"main" java.lang.NoClassDefFoundError中的异常： akka/util/Helpers$ConfigOps$

Question

我正在关注下面链接中的一篇文章，用java创建一个简单的spark作业。

https://www.datasciencebytes.com/bytes/2016/04/18/getting-started-with-spark-running-a-simple-spark-job-in-java/

下面是我的Java文件：

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import java.io.IOException;
public class SparkAppMain {
public static void main(String[] args) throws IOException {
SparkConf sparkConf = new SparkConf()
.setAppName("Example Spark App")
.setMaster("local[*]"); // Delete this line when submitting to a cluster
JavaSparkContext sparkContext = new JavaSparkContext(sparkConf);
JavaRDD<String> stringJavaRDD = sparkContext.textFile("/tmp/nationalparks.csv");
System.out.println("Number of lines in file = " + stringJavaRDD.count());
}
}

下面是我的POM.xml文件：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>edu.berkeley</groupId>
<artifactId>simple-project</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.3</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.5.Final</version>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-actor_2.10</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.6.1</version>
</dependency>
</dependencies>
</project>

我已经尝试过类似spark-streaming的解决方案，该版本应该与spark-core(1.5.2(相同。spark-streaming-kafka_2.10也是如此。

Answer 1

添加了以下pom.xml并解决了问题

<?xml version="1.0" encoding="UTF-8"?>

http://maven.apache.org/xsd/maven-4.0.0.xsd">4.0.0

<groupId>edu.berkeley</groupId>
<artifactId>simple-project</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>com.codahale.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>com.codahale.metrics</groupId>
<artifactId>metrics-json</artifactId>
<version>3.0.2</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.5.Final</version>
</dependency>
</dependencies>