Scikit学习决策树模型评价

Question

这里是相关的代码和文档，想知道默认的cross_val_score没有明确指定score，输出数组意味着精度，AUC或其他一些指标?

使用Python 2.7和miniconda解释器

http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html

>>> from sklearn.datasets import load_iris
>>> from sklearn.cross_validation import cross_val_score
>>> from sklearn.tree import DecisionTreeClassifier
>>> clf = DecisionTreeClassifier(random_state=0)
>>> iris = load_iris()
>>> cross_val_score(clf, iris.data, iris.target, cv=10)
...                             
...
array([ 1.     ,  0.93...,  0.86...,  0.93...,  0.93...,
        0.93...,  0.93...,  1.     ,  0.93...,  1.      ])

问候,林

Answer 1

来自用户指南:

默认情况下，每次CV迭代计算的分数为该分数估计器的方法。可以通过使用评分参数:

摘自decisiontreecclassifier文档:

返回给定测试数据和标签的平均精度。在多标签分类，这是子集精度，是a苛刻的度量，因为您要求每个样本的每个标签集正确预测。

不要被"平均精度"弄糊涂了，这只是计算精度的常规方式。点击链接进入原文:

    from .metrics import accuracy_score
    return accuracy_score(y, self.predict(X), sample_weight=sample_weight)

现在metrics.accuracy_score的来源

def accuracy_score(y_true, y_pred, normalize=True, sample_weight=None):
    ...
    # Compute accuracy for each possible representation
    y_type, y_true, y_pred = _check_targets(y_true, y_pred)
    if y_type.startswith('multilabel'):
        differing_labels = count_nonzero(y_true - y_pred, axis=1)
        score = differing_labels == 0
    else:
        score = y_true == y_pred
    return _weighted_sum(score, sample_weight, normalize)

如果你仍然不相信:

def _weighted_sum(sample_score, sample_weight, normalize=False):
    if normalize:
        return np.average(sample_score, weights=sample_weight)
    elif sample_weight is not None:
        return np.dot(sample_score, sample_weight)
    else:
        return sample_score.sum()

注意:对于accuracy_score规范化参数默认为True，因此它只是返回布尔numpy数组的np.average，因此它只是正确预测的平均数量。

Answer 2

如果没有给出评分参数，cross_val_score将默认使用您正在使用的估计器的.score方法。对于DecisionTreeClassifier，它的平均精度(如下面的docstring所示):

In [11]: DecisionTreeClassifier.score?
Signature: DecisionTreeClassifier.score(self, X, y, sample_weight=None)
Docstring:
Returns the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset accuracy
which is a harsh metric since you require for each sample that
each label set be correctly predicted.
Parameters
----------
X : array-like, shape = (n_samples, n_features)
    Test samples.
y : array-like, shape = (n_samples) or (n_samples, n_outputs)
    True labels for X.
sample_weight : array-like, shape = [n_samples], optional
    Sample weights.
Returns
-------
score : float
    Mean accuracy of self.predict(X) wrt. y.