将长列表保存到pandas中的csv中

Question

我正试图将一个列表(有7000个值(保存到csv文件中。但当我打开csv时，列表会被截断，如下所示"[1 2 3…6999 7000]"，并且也会存储为字符串。有没有一种方法可以将长列表的列表存储在csv中，而不会截断值。

x = []
a = np.arange(0,7000,1)
x.append(a)
b  = np.arange(7001,14000,1)
x.append(b)
x
Out: [array([   0,    1,    2, ..., 6997, 6998, 6999]),
array([ 7001,  7002,  7003, ..., 13997, 13998, 13999])]

df = pd.DataFrame({"x":x})
df.to_csv("x.csv")
df = pd.read_csv("x.csv")
df["x"][0]
Out: '[   0    1    2 ... 6997 6998 6999]'
type(df["x"][0])
Out: str

Answer 1

如果您想将数据保存到csv，只需将数据类型转换为字符串str.

import pandas as pd
import numpy as np
alist = []
a = np.arange(0,7000,1)
alist.append(a)
b  = np.arange(7001,14000,1)
alist.append(b)
df = pd.DataFrame({"alist":alist})
# merge data as string
df['alist'] = df['alist'].map(lambda x: ','.join(map(str, x)))
df.to_csv("list.csv", index=False)

读取csv文件：

dfn = pd.read_csv("list.csv")
dfn['alist'] = dfn['alist'].str.split(',')
dfn['alist'] = dfn['alist'].map(lambda x: list(map(int, x)))
dfn['alist'][0]

或者考虑另一种方式：

# Examples
# For the simplest code, use the dump() and load() functions.
import pickle
# An arbitrary collection of objects supported by pickle.
data = {
'a': [1, 2.0, 3, 4+6j],
'b': ("character string", b"byte string"),
'c': {None, True, False}
}
with open('data.pickle', 'wb') as f:
# Pickle the 'data' dictionary using the highest protocol available.
pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)
# The following example reads the resulting pickled data.
with open('data.pickle', 'rb') as f:
# The protocol version used is detected automatically, so we do not
# have to specify it.
data = pickle.load(f)

Answer 2

因为numpy数组的字符串表示被截断。另一种方法是在将numpy数组保存到csv文件之前，将其转换为python列表。

import pandas as pd
import numpy as np
df = pd.DataFrame({
'long_list': [np.arange(0, 7000).tolist()]
})
df.to_csv('temp.csv')