在Spark Databricks (scala)中写入/读取/删除二进制数据

Question

我对Databricks (Scala)上的Spark很陌生，我想知道如何将类型为Array[Byte]的变量内容写入挂载存储mtn/somewhere/tmp/(Azure数据湖)或file:/tmp/中的临时文件data.bin。然后我想知道如何将其读取为InputStream，然后在我完成它时删除它。

到目前为止，我读过的所有方法都不能工作或不适用于二进制数据。

谢谢。

Answer 1

结果证明这段代码运行良好:

import java.io._
import org.apache.commons.io.FileUtils
// Create or collect the data
val bytes: Array[Byte] = <some_data>
try {
// Write data to temp file
// Note : Here I use GRIB2 file as I manipulate forecast data,
//        but you can use .bin or .png/.jpg (if it's an image data)
//        extensions or no extension at all. It doesn't matter.
val path: String = "mf-data.grib"
val file: File = new File(path)
FileUtils.writeByteArrayToFile(file, bytes)

// Read the temp file
val input = new FileInputStream(path)
////////// Do something with it //////////
// Remove the temp file
if (!file.delete()) {
println("Cannot delete temporary file !")
}
} catch {
case _: Throwable => println("An I/O error occured")