Spark 中是否有类似的 API 和 twitter.scalding.addTrap 来处理异常

Question

我正在将一个项目从Hadoop移植到Spark 2.1.0。以前，它使用 twitter.scalding.addTrap 来处理异常，例如： https://github.com/scalding-io/ProgrammingWithScalding/blob/master/chapter3/src/main/scala/addTrap.scala

对于Spark，我使用sc.textFile(InputPath(读取输入，但我不知道如何执行以前的异常处理功能。

Answer 1

您可以使用 Try。

import scala.io.StdIn
import scala.util.{Try, Success, Failure}
def divide: Try[Int] = {
val dividend = Try(StdIn.readLine("Enter an Int that you'd like to divide:n").toInt)
val divisor = Try(StdIn.readLine("Enter an Int that you'd like to divide by:n").toInt)
val problem = dividend.flatMap(x => divisor.map(y => x/y))
problem match {
case Success(v) =>
println("Result of " + dividend.get + "/"+ divisor.get +" is: " + v)
Success(v)
case Failure(e) =>
println("You must've divided by zero or entered something that's not an Int. Try again!")
println("Info from the exception: " + e.getMessage)
divide
}
}