GCP Cloud函数到BigQuery-拼花地板支持错误

Question

我正在尝试运行一个简单的测试cloud function，在这里我创建一个BigQuery表并插入一个值。我收到的错误听起来像是我需要导入pyarrow，所以我尝试过这样做，但我一直收到同样的错误。当我在本地运行等效的脚本时，没有任何问题，表已经创建，我甚至不需要导入pyarrow。我在这里错过了什么？

error:

ImportError: Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'. pyarrow or fastparquet is required for parquet support

main.py:

import pandas as pd
from google.cloud import bigquery
import pyarrow
def main_func(data, context):
    df = pd.DataFrame({'Test': ['Success']})
    client = bigquery.Client()
    dataset_id = #removed here but specified in the real code
    dataset = bigquery.Dataset(dataset_id)
    dataset.location = #removed here but specified in the real code
    dataset = client.create_dataset(dataset, exists_ok=True)
    print("Created dataset {}.{}".format(client.project, dataset.dataset_id))
    table_id = #removed here but specified in the real code
    job_config = bigquery.LoadJobConfig(
        schema=[
            bigquery.SchemaField("Test", bigquery.enums.SqlTypeNames.STRING),
        ],
        write_disposition="WRITE_TRUNCATE",
    )
    job = client.load_table_from_dataframe(
        df, table_id, job_config = job_config
    )
    job.result()

requirements.txt:

pandas
google-cloud-bigquery
pyarrow

Answer 1

pyarrow版本有问题。由于兼容性问题，Pandas没有检测到任何pyarrow<0.4，因此您应该尝试在requirements.txt中添加pyarrow>=0.4。

导入射线后Pyarrow未正确检测到