FileNotFoundError:[WinError 2]当我在cmd/Pycharm中运行pyspark时,系统找不



我正试图用下面的代码在pycharm中运行一个python文件。当我在cmd中提供pyspark和pycharm时,我也面临着同样的问题,有人能帮我吗。提前谢谢。

代码:

from pyspark.sql import SparkSession
from pyspark.sql.types import StructType, StructField, StringType, DateType, DecimalType, IntegerType
spark = SparkSession.builder.master("local[*]").appName("ETL").getOrCreate()
spark.sparkContext.setLogLevel("WARN")
source_data_file = "C:Python_pgmsapache-spark-etl-pipeline-example-masterapache-spark-etl-pipeline-example-masterdata20160104*"
print("Fetching")

面临以下问题:

Traceback (most recent call last):
File "C:/Python_pgms/apache-spark-etl-pipeline-example-master/apache-spark-etl-pipeline-example-master/src/etl.py", line 5, in <module>
spark = SparkSession.builder.master("local[*]").appName("ETL").getOrCreate()
File "C:Sparkspark-3.0.1-bin-hadoop2.7pythonlibpyspark.zippysparksqlsession.py", line 186, in getOrCreate
File "C:Sparkspark-3.0.1-bin-hadoop2.7pythonlibpyspark.zippysparkcontext.py", line 376, in getOrCreate
File "C:Sparkspark-3.0.1-bin-hadoop2.7pythonlibpyspark.zippysparkcontext.py", line 133, in __init__
File "C:Sparkspark-3.0.1-bin-hadoop2.7pythonlibpyspark.zippysparkcontext.py", line 325, in _ensure_initialized
File "C:Sparkspark-3.0.1-bin-hadoop2.7pythonlibpyspark.zippysparkjava_gateway.py", line 98, in launch_gateway
File "C:UserscompAppDataLocalProgramsPythonPython36libsubprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:UserscompAppDataLocalProgramsPythonPython36libsubprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Process finished with exit code 1

pycharm->运行->编辑配置->环境变量

根据您的路径添加PYTHONPATH和SPARK_HOME

最新更新