语法错误行 1 位于位置 15 意外'copy' pyspark

Question

我正试图在pyspark中运行一个复制命令。如下所示。如何消除这个错误？

spark.write.format("snowflake") 
.options(**config.sfparams) 
.option("query",  "copy into people_data from (select $1:Company_ID::varchar as Company_ID from  @company_stage/pitchbook/"+config.todays_date+"/Company/")
.load()

错误：

py4j.protocol.Py4JJavaError: An error occurred while calling o49.load.
: net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation error:
syntax error line 1 at position 15 unexpected 'copy'.

Answer 1

复制在pyspark中不起作用。创建一个外部表直接从s3读取数据，并启用autorefresh=true