创建RDD时未找到火花错误RDD类型

Question

我正在尝试创建case类对象的RDD。例如，

// sqlContext from the previous example is used in this example.
// createSchemaRDD is used to implicitly convert an RDD to a SchemaRDD.
import sqlContext.createSchemaRDD
val people: RDD[Person] = ... // An RDD of case class objects, from the previous example.
// The RDD is implicitly converted to a SchemaRDD by createSchemaRDD, allowing it to be stored using        Parquet.
people.saveAsParquetFile("people.parquet")

我正试图通过给出来完成上一个例子中的部分

    case class Person(name: String, age: Int)
    // Create an RDD of Person objects and register it as a table.
    val people: RDD[Person] = sc.textFile("/user/root/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))
    people.registerAsTable("people")

我得到以下错误：

<console>:28: error: not found: type RDD
       val people: RDD[Person] =sc.textFile("/user/root/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))

你知道出了什么问题吗？提前感谢！

Answer 1

这里的问题是显式的RDD[String]类型注释。在spark-shell中，似乎默认情况下没有导入RDD，这就是Scala抱怨找不到RDD类型的原因。请先尝试运行import org.apache.spark.rdd.RDD。