。
假设我有以下数据
import pandas as pd
data = {
'Reference': [1, 2, 3, 4, 5],
'Brand': ['Volkswagen', 'Volvo', 'Volvo', 'Audi', 'Volkswagen'],
'Town': ['Berlin', 'Berlin', 'Stockholm', 'Munich', 'Berlin'],
'Mileage': [35000, 45000, 121000, 35000, 181000],
'Year': [2015, 2014, 2012, 2016, 2013]
}
df = pd.DataFrame(data)
我想对"品牌"和"城镇"两列进行单热编码,以训练分类器(例如使用Scikit-Learn)并预测年份。
训练分类器后,我将希望预测新传入数据的年份(不在训练中使用),我需要重新应用相同的热编码。例如:
new_data = {
'Reference': [6, 7],
'Brand': ['Volvo', 'Audi'],
'Town': ['Stockholm', 'Munich']
}
在这种情况下,对 Pandas 数据帧上的 2 列进行独热编码的最佳方法是什么,知道需要对多个列进行编码,并且以后需要能够对新数据应用相同的编码。
这是一个后续问题,即如何在SkLearn中重复使用LabelBinarizer进行输入预测
请考虑以下方法。
演示:
from sklearn.preprocessing import LabelBinarizer
from collections import defaultdict
d = defaultdict(LabelBinarizer)
In [7]: cols2bnrz = ['Brand','Town']
In [8]: df[cols2bnrz].apply(lambda x: d[x.name].fit(x))
Out[8]:
Brand LabelBinarizer(neg_label=0, pos_label=1, spars...
Town LabelBinarizer(neg_label=0, pos_label=1, spars...
dtype: object
In [10]: new = pd.DataFrame({
...: 'Reference': [6, 7],
...: 'Brand': ['Volvo', 'Audi'],
...: 'Town': ['Stockholm', 'Munich']
...: })
In [11]: new
Out[11]:
Brand Reference Town
0 Volvo 6 Stockholm
1 Audi 7 Munich
In [12]: pd.DataFrame(d['Brand'].transform(new['Brand']), columns=d['Brand'].classes_)
Out[12]:
Audi Volkswagen Volvo
0 0 0 1
1 1 0 0
In [13]: pd.DataFrame(d['Town'].transform(new['Town']), columns=d['Town'].classes_)
Out[13]:
Berlin Munich Stockholm
0 0 0 1
1 0 1 0
您可以使用
pandas 提供get_dummies函数并转换分类值。
像这样的东西..
import pandas as pd
data = {
'Reference': [1, 2, 3, 4, 5],
'Brand': ['Volkswagen', 'Volvo', 'Volvo', 'Audi', 'Volkswagen'],
'Town': ['Berlin', 'Berlin', 'Stockholm', 'Munich', 'Berlin'],
'Mileage': [35000, 45000, 121000, 35000, 181000],
'Year': [2015, 2014, 2012, 2016, 2013]
}
df = pd.DataFrame(data)
train = pd.concat([df.get(['Mileage','Reference','Year']),
pd.get_dummies(df['Brand'], prefix='Brand'),
pd.get_dummies(df['Town'], prefix='Town')],axis=1)
对于测试数据,您可以:
new_data = {
'Reference': [6, 7],
'Brand': ['Volvo', 'Audi'],
'Town': ['Stockholm', 'Munich']
}
test = pd.DataFrame(new_data)
test = pd.concat([test.get(['Reference']),
pd.get_dummies(test['Brand'], prefix='Brand'),
pd.get_dummies(test['Town'], prefix='Town')],axis=1)
# Get missing columns in the training test
missing_cols = set( train.columns ) - set( test.columns )
# Add a missing column in test set with default value equal to 0
for c in missing_cols:
test[c] = 0
# Ensure the order of column in the test set is in the same order than in train set
test = test[train.columns]