如何根据频率对文本列进行分类

Question

我有一个城市名称列表，我想根据它们的频率对其进行分类。我最初想使用binning，但由于这需要单调的间距，我放弃了。下一个，甚至更好的方法是使用pandas.qcut根据频率基于分位数创建类别。但有了分位数，我不知道如何根据分位数创建额外的列。示例：

import numpy as np
import pandas as pd
np.random.seed(0)
cities = np.random.choice(['Ontario', 'Ottawa', 'Vancouver','Edmonton',
'Winnipeg', 'Churchill'], 500)
# Create fake data and their frequencies
df = pd.DataFrame (cities, columns=['City'])
freq = df['City'].value_counts()
print (freq)
# Create quantiles
qc = pd.qcut (freq, 3)
print (qc)
# And now? I have the quantiles but how to assign a categorie to each City?
category_for_each_city = df['City'] in qc # does not work, but many other things neither

我尝试了很多方法，但都不起作用。我应该能够为此编写一个循环，但我无法想象这是Python的方式。我试着寻找一些sklearn变形金刚，但找不到任何具体的解决方案。如有任何帮助，我们将不胜感激。

此外，我有许多偏斜的分布，一个可以扩展到例如日志转换的解决方案将非常有帮助。

Answer 1

你几乎到了。。。

In [106]: category_for_each_city = df['City'].map(qc)
In [107]: category_for_each_city
Out[107]:
0      (77.333, 84.667]
1      (72.999, 77.333]
2       (84.667, 100.0]
3       (84.667, 100.0]
4       (84.667, 100.0]
5       (84.667, 100.0]
6      (77.333, 84.667]
...
493     (84.667, 100.0]
494    (72.999, 77.333]
495    (77.333, 84.667]
496     (84.667, 100.0]
497    (77.333, 84.667]
498    (77.333, 84.667]
499    (77.333, 84.667]
Name: City, Length: 500, dtype: category
Categories (3, interval[float64]): [(72.999, 77.333] < (77.333, 84.667] < (84.667, 100.0]]

更新：

In [114]: qc = pd.qcut (freq, 3, labels=[0,1,2])
In [115]: category_for_each_city = df['City'].map(qc)
In [116]: category_for_each_city
Out[116]:
0      1
1      0
2      2
3      2
4      2
5      2
6      1
..
493    2
494    0
495    1
496    2
497    1
498    1
499    1
Name: City, Length: 500, dtype: category
Categories (3, int64): [0 < 1 < 2]