使用GridSearchCV进行Logistic回归时的精度计算警告

Question

我正在尝试使用LogisticRegression估计器运行GridSearchCV，并记录模型的准确性，精度，召回率，f1指标。

但是，我在精度度量上得到以下错误:

Precision is ill-defined and being set to 0.0 due to no predicted samples. 
Use `zero_division` parameter to control this behavior

我明白为什么我得到错误，因为在Kfold分割中没有输出值等于1的预测。然而，我不明白如何具体设置"零分割"。在GridSearchCV (logistic_reg变量)中作为1 .

原始代码

logistic_reg = GridSearchCV(estimator=LogisticRegression(penalty="l1", random_state=42, max_iter=10000), param_grid={
"C": [1e-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, 1e-1, 5e-1, 1, 5, 10, 20],
"solver": ["liblinear", "saga"]
}, scoring=["accuracy", "precision", "recall", "f1"], cv=StratifiedKFold(n_splits=10), refit="accuracy")

logistic_reg_X_train = self.X_train.copy()
logistic_reg_X_train.drop(self.columns_removed, axis=1, inplace=True)

logistic_reg.fit(logistic_reg_X_train, self.y_train)
logistic_reg_results = pd.DataFrame(logistic_reg.cv_results_)

我试过改变"精度"。到precision_score(zero_division=1)，但这给了我另一个错误(missing 2 required positional arguments: 'y_true' and 'y_pred')。我再次理解这一点，但是在应用fit方法之前没有定义两个缺失的参数。

如何为精度评分指标指定1zero_division参数?

编辑我不明白的是，我在我的train_testrongplit方法中分层了y数据，并在GridSearchCV中使用了StratifedKFold。我对此的理解是，训练/测试数据将具有相同的y值分割比例，并且在交叉验证期间应该发生相同的情况。这意味着在gridsearchcv样本中，数据的y值应该为0和1，因此精度不能等于0(模型将能够计算TP和FP，因为样本测试数据包含y等于1的样本)。我不确定从这里到哪里。

Answer 1

从进一步阅读这个问题，我的理解是，错误正在发生，因为不是我的y_test中的所有标签都出现在我的y_pred中。我的数据不是这样的。

我使用了G.Anderson的注释来删除警告(但它没有回答我的问题)

• 创建新的custom_scorer对象

• 创建customer_scoring字典

• 更新GridSearchCV评分和改装参数

  from sklearn.metrics import precision_score, make_scorer
  precision_scorer = make_scorer(precision_score, zero_division=0)
  custom_scoring = {"accuracy": "accuracy", "precision": precision_scorer, "recall": "recall", "f1": "f1"}
  logistic_reg = GridSearchCV(estimator=LogisticRegression(penalty="l1", random_state=42, max_iter=10000), param_grid={
  "C": [1e-4, 5e-4, 1e-3, 5e-3, 1e-2, 5e-2, 1e-1, 5e-1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20],
  "solver": ["liblinear", "saga"]
  }, scoring=custom_scoring, cv=StratifiedKFold(n_splits=10), refit="accuracy")
  

编辑-上述问题的答案

我使用GridSearchCV来找到模型的最佳超参数。为了查看每个分割的模型指标，我创建了一个具有最佳超参数的StratifedKFold估计器，然后自己进行交叉验证。这没有给我精确的警告信息。我不知道为什么GridSearchCV给我一个警告，但至少这种方式是有效的!!

注意:我从下面的方法和上面的问题中的GridSearchCV得到相同的结果。

skf = StratifiedKFold(n_splits=10)
logistic_reg_class_skf = LogisticRegression(penalty="l1", max_iter=10000, random_state=42, C=5, solver="liblinear")

logistic_reg_class_score = []

for train, test in skf.split(logistic_reg_class_X_train, self.y_train):

logistic_reg_class_skf_X_train = logistic_reg_class_X_train.iloc[train]
logistic_reg_class_skf_X_test = logistic_reg_class_X_train.iloc[test]
logistic_reg_class_skf_y_train = self.y_train.iloc[train]
logistic_reg_class_skf_y_test = self.y_train.iloc[test]

logistic_reg_class_skf.fit(logistic_reg_class_skf_X_train, logistic_reg_class_skf_y_train)
logistic_reg_skf_y_pred = logistic_reg_class_skf.predict(logistic_reg_class_skf_X_test)

skf_accuracy_score = metrics.accuracy_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
skf_precision_score = metrics.precision_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
skf_recall_score = metrics.recall_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
skf_f1_score = metrics.f1_score(logistic_reg_class_skf_y_test, logistic_reg_skf_y_pred)
logistic_reg_class_score.append([skf_accuracy_score, skf_precision_score, skf_recall_score, skf_f1_score])
classification_results = pd.DataFrame({"Algorithm": ["Logistic Reg Train"], "Accuracy": [0.0], "Precision": [0.0],
"Recall": [0.0], "F1 Score": [0.0]})

for i in range (0, 10):
classification_results.loc[i] = ["Logistic Reg Train", logistic_reg_class_score[i][0], logistic_reg_class_score[i][1],
logistic_reg_class_score[2][0], logistic_reg_class_score[3][0]]