我有一个pyarrow表的标题:['column1','column2','column3','column4','column5']我想交换和模式列标题和数据:(‘column1’,‘column2’,‘column5’,‘column3’,‘column4’)如何使用pandas或pyarrow
df = df[['column1','column2','column5','column3','column4' ]]
这将重新排列
在PyArrow中,您可以尝试使用select()
方法:
import pyarrow as pa
# Define an example pa.Table
n_legs = pa.array([2, 4, 5, 100])
animals = pa.array(["Flamingo", "Horse", "Brittle stars", "Centipede"])
names = ["n_legs", "animals"]
table = pa.table([n_legs, animals], names=names)
# Select columns
table.select([1,0])
你会得到:
>>> table.select([1,0])
pyarrow.Table
animals: string
n_legs: int64
----
animals: [["Flamingo","Horse","Brittle stars","Centipede"]]
n_legs: [[2,4,5,100]]
与原文:
>>> table
pyarrow.Table
n_legs: int64
animals: string
----
n_legs: [[2,4,5,100]]
animals: [["Flamingo","Horse","Brittle stars","Centipede"]]
在将Pandas数据框转换为PyArrow表时,也可以使用自定义模式:
import pyarrow as pa
import pandas as pd
# Define Pandas dataframe
df = pd.DataFrame({'year': [2020, 2022, 2019, 2021],
'n_legs': [2, 4, 5, 100],
'animals': ["Flamingo", "Horse", "Brittle stars", "Centipede"]})
# Define custom PyArrow schema
my_schema = pa.schema([
pa.field('n_legs', pa.int64()),
pa.field('animals', pa.string()),
pa.field('year', pa.int64())
])
# Read Pandas dataframe as PyArrow table with specified schema
table = pa.table(df, my_schema)
你会得到:
>>> table.to_pandas()
n_legs animals year
0 2 Flamingo 2020
1 4 Horse 2022
2 5 Brittle stars 2019
3 100 Centipede 2021
>>> df
year n_legs animals
0 2020 2 Flamingo
1 2022 4 Horse
2 2019 5 Brittle stars
3 2021 100 Centipede
在pyarrow中可以这样做:
columns = ['column1', 'column2']
table.select(columns)