GridSearch在OneVsRestClassifier中查找估计器

Question

我想在SVC模型中执行GridSearchCV，但这使用了一对一策略。对于后一部分，我可以这样做：

model_to_set = OneVsRestClassifier(SVC(kernel="poly"))

我的问题是参数。假设我想尝试以下值：

parameters = {"C":[1,2,4,8], "kernel":["poly","rbf"],"degree":[1,2,3,4]}

为了执行GridSearchCV，我应该做一些类似的事情：

cv_generator = StratifiedKFold(y, k=10)
model_tunning = GridSearchCV(model_to_set, param_grid=parameters, score_func=f1_score, n_jobs=1, cv=cv_generator)

然而，然后我执行它，我得到：

Traceback (most recent call last):
File "/.../main.py", line 66, in <module>
argclass_sys.set_model_parameters(model_name="SVC", verbose=3, file_path=PATH_ROOT_MODELS)
File "/.../base.py", line 187, in set_model_parameters
model_tunning.fit(self.feature_encoder.transform(self.train_feats), self.label_encoder.transform(self.train_labels))
File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py", line 354, in fit
return self._fit(X, y)
File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py", line 392, in _fit
for clf_params in grid for train, test in cv)
File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py", line 473, in __call__
self.dispatch(function, args, kwargs)
File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py", line 296, in dispatch
job = ImmediateApply(func, args, kwargs)
File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py", line 124, in __init__
self.results = func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py", line 85, in fit_grid_point
clf.set_params(**clf_params)
File "/usr/local/lib/python2.7/dist-packages/sklearn/base.py", line 241, in set_params
% (key, self.__class__.__name__))
ValueError: Invalid parameter kernel for estimator OneVsRestClassifier

基本上，由于SVC位于OneVsRestClassifier中，并且这是我发送给GridSearchCV的估计器，因此无法访问SVC的参数。

为了实现我想要的，我看到了两种解决方案：

1. 在创建SVC时，以某种方式告诉它不要使用一对一策略，而是使用一对所有策略
2. 以某种方式向GridSearchCV指示参数对应于OneVsRestClassifier内部的估计器

我还没有找到一种方法来做任何提到的替代方案。你知道有没有办法做到这些吗？或者你可以建议另一种方法来获得同样的结果？

谢谢！

Answer 1

在网格搜索中使用嵌套估计量时，可以使用__作为分隔符来确定参数的范围。在这种情况下，SVC模型作为名为estimator的属性存储在OneVsRestClassifier模型中：

from sklearn.datasets import load_iris
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import SVC
from sklearn.grid_search import GridSearchCV
from sklearn.metrics import f1_score
iris = load_iris()
model_to_set = OneVsRestClassifier(SVC(kernel="poly"))
parameters = {
"estimator__C": [1,2,4,8],
"estimator__kernel": ["poly","rbf"],
"estimator__degree":[1, 2, 3, 4],
}
model_tunning = GridSearchCV(model_to_set, param_grid=parameters,
score_func=f1_score)
model_tunning.fit(iris.data, iris.target)
print model_tunning.best_score_
print model_tunning.best_params_

这就产生了：

0.973290762737
{'estimator__kernel': 'poly', 'estimator__C': 1, 'estimator__degree': 2}

Answer 2

对于Python 3，应该使用以下代码

from sklearn.datasets import load_iris
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import f1_score
iris = load_iris()
model_to_set = OneVsRestClassifier(SVC(kernel="poly"))
parameters = {
"estimator__C": [1,2,4,8],
"estimator__kernel": ["poly","rbf"],
"estimator__degree":[1, 2, 3, 4],
}
model_tunning = GridSearchCV(model_to_set, param_grid=parameters,
scoring='f1_weighted')
model_tunning.fit(iris.data, iris.target)
print(model_tunning.best_score_)
print(model_tunning.best_params_)

Answer 3

param_grid  = {"estimator__alpha": [10**-5, 10**-3, 10**-1, 10**1, 10**2]}
clf = OneVsRestClassifier(SGDClassifier(loss='log',penalty='l1'))
model = GridSearchCV(clf,param_grid, scoring = 'f1_micro', cv=2,n_jobs=-1)
model.fit(x_train_multilabel, y_train)