管道中的分类变量:维度不匹配

Question

我尝试用分类变量构建一个管道

import numpy as np
import pandas as pd
import sklearn
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn import linear_model
from sklearn.pipeline import Pipeline
df = pd.DataFrame({'a':range(6),  'c':['a', 'b', 'c']*2, 'd': ['m', 'f']*3        })
X = df[['c', 'd']]
y = df['a']
regressor = linear_model.SGDRegressor()

转换分类变量

class Cat(TransformerMixin):
    def transform(self, X, **transform_params):
        enc = DictVectorizer(sparse = False)
        enc_data = enc.fit_transform(X.T.to_dict().values())
        return enc_data
    def fit(self, X, y=None, **fit_params):
        return self

管道

 pipeline = Pipeline([ 
                ('categorical', Cat()),
    ('model_fitting', regressor),
])
pipeline.fit(X, y)

这是正确的。但我得到错误时，我试图拟合一个新的数据集。例如

contr = pd.DataFrame({'c':['a'], 'd': ['m']})
pred = pipeline.predict(contr)
pred
and
ValueError: shapes (1,2) and (5,) not aligned: 2 (dim 1) != 5 (dim 0)

我看到Cat(TransformerMixin)类中的问题。如何改进它?

Answer 1

我已经这样做了。它是

我已经改变了功能

class Cat(TransformerMixin):
def transform(self, X, y=None, **fit_params):
    enc = DictVectorizer(sparse = False)
    encc = enc.fit(df[['c', 'd']].T.to_dict().values())
    enc_data = encc.transform(X.T.to_dict().values())
    return enc_data
def fit_transform(self, X, y=None, **fit_params):
    self.fit(X, y, **fit_params)
    return self.transform(X)
def fit(self, X, y=None, **fit_params):
    return self

和一个新的数据集

control = pd.DataFrame({'c':['b'], 'd': ['f']})
pred = pipeline.predict(control)
pred