为什么cross_val_score的ROC_AUC比手动使用带有指标的StratfiedKFold要高得多?roc_a

Question

方法1 - StratifiedKFold交叉验证

skf = StratifiedKFold(n_splits=5, shuffle=False) 
roc_aucs_temp = []
for i, (train_index, test_index) in enumerate(skf.split(X_train_xgb, y_train_xgb)):   
X_train_fold, X_test_fold = X_train_xgb.iloc[train_index], X_train_xgb.iloc[test_index]
y_train_fold, y_test_fold = y_train_xgb[train_index], y_train_xgb[test_index]
xgb_temp.fit(X_train_fold, y_train_fold)
y_pred=model.predict(X_test_fold)
roc_aucs_temp.append(metrics.roc_auc_score(y_test_fold, y_pred))
print(roc_aucs_temp)
[0.8622474747474748, 0.8497474747474747, 0.9045918367346939, 0.8670918367346939, 0.879591836734694]

方法二CrossValScore

# this uses the same CV object as method 1 
print(cross_val_score(xgb, X_train_xgb, y_train_xgb, cv=skf, scoring='roc_auc')) 
[0.9614899  0.94861111 0.96045918 0.97270408 0.96977041]

我可能误解了cross_val_score的功能，但从我的理解来看，它创建了K折叠的训练和测试数据。然后在K-1次折叠上训练模型，并在1次折叠上重复测试。它应该与使用StratifiedKFold手动创建K折叠的精度相同。为什么不呢?

https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html

https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html

Answer 1

roc_auc_score的文档指出它的第二个参数是标签分数而不是预测的标签。就像他们在例子中展示的，你可能想要model.predict_proba(X_test_fold)[:, 1]而不是model.predict(X_test_fold)。cross_val_score和roc_auc是这样评估的，这就是为什么你看到了差异。