数字大小在大型列表中失败



我有一长串熊猫数据帧,根据类型,"pandas.core.series.Series"。此列表中可能有一百万个条目。 我通过用numpy.array_split分割数据框来创建它。

每个数据帧此时应该只包含一个数字:

In[29]: df1[0:5]
Out[29]: 
[1    12149992.0
Name: 3121916261129, dtype: float64, 2    12149995.0
Name: 3121916261129, dtype: float64, 3    12149997.0
Name: 3121916261129, dtype: float64, 4    12149994.0
Name: 3121916261129, dtype: float64, 5    12149993.0
Name: 3121916261129, dtype: float64]

现在,当我想知道这个列表有多大时,我使用了numpy的大小函数,该函数在一段时间后失败:

In [31]: np.size(df1)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/home/sw/Dropbox (IQOQI_Vienna)/dataanalysis_results/allan/allanvariance_pandas1khz.py in <module>()
----> 1 np.size(df1)
/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.pyc in size(a, axis)
2693             return a.size
2694         except AttributeError:
-> 2695             return asarray(a).size
2696     else:
2697         try:
/usr/lib/python2.7/dist-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
529 
530     """
--> 531     return array(a, dtype, copy=False, order=order)
532 
533 
/usr/lib/python2.7/dist-packages/pandas/core/series.pyc in __getitem__(self, key)
601         key = com._apply_if_callable(key, self)
602         try:
--> 603             result = self.index.get_value(self, key)
604 
605             if not is_scalar(result):
/usr/lib/python2.7/dist-packages/pandas/indexes/base.pyc in get_value(self, series, key)
2167         try:
2168             return self._engine.get_value(s, k,
-> 2169                                           tz=getattr(series.dtype, 'tz', None))
2170         except KeyError as e1:
2171             if len(self) > 0 and self.inferred_type in ['integer', 'boolean']:
pandas/index.pyx in pandas.index.IndexEngine.get_value (pandas/index.c:3557)()
pandas/index.pyx in pandas.index.IndexEngine.get_value (pandas/index.c:3240)()
pandas/index.pyx in pandas.index.IndexEngine.get_loc (pandas/index.c:4279)()
pandas/src/hashtable_class_helper.pxi in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:8564)()
pandas/src/hashtable_class_helper.pxi in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:8508)()
KeyError: 0
In [32]: 

坦率地说,我不明白这个错误消息。另外,我正在寻找一种更好的方法来确定此列表的大小。有人可以帮我吗?

错误表明 NumPy 正在尝试将您的列表转换为数组,但失败了。相反,您可以通过以下方式将大小相加:

sum(series.size for series in df1)

最新更新