从h2o模型中使用iml解决兼容性问题

Question

我无法重现我能找到的将h2o与iml一起使用的唯一示例(https://www.r-bloggers.com/2018/08/iml-and-h2o-machine-learning-model-interpretability-and-feature-explanation/)如本文所述(使用FeatureImp$new和H2O提取变量重要性时出错(。有人能指出一个变通方法或其他将iml与h2o一起使用的例子吗？

---

再现性示例：

library(rsample)   # data splitting
library(ggplot2)   # allows extension of visualizations
library(dplyr)     # basic data transformation
library(h2o)       # machine learning modeling
library(iml)       # ML interprtation
library(modeldata) #attrition data 

# initialize h2o session
h2o.no_progress()
h2o.init()
# classification data
data("attrition", package = "modeldata")
df <- rsample::attrition %>% 
mutate_if(is.ordered, factor, ordered = FALSE) %>%
mutate(Attrition = recode(Attrition, "Yes" = "1", "No" = "0") %>% factor(levels = c("1", "0")))
# convert to h2o object
df.h2o <- as.h2o(df)
# create train, validation, and test splits
set.seed(123)
splits <- h2o.splitFrame(df.h2o, ratios = c(.7, .15), destination_frames = 
c("train","valid","test"))
names(splits) <- c("train","valid","test")
# variable names for resonse & features
y <- "Attrition"
x <- setdiff(names(df), y) 
# elastic net model 
glm <- h2o.glm(
x = x, 
y = y, 
training_frame = splits$train,
validation_frame = splits$valid,
family = "binomial",
seed = 123
)
# 1. create a data frame with just the features
features <- as.data.frame(splits$valid) %>% select(-Attrition)
# 2. Create a vector with the actual responses
response <- as.numeric(as.vector(splits$valid$Attrition))
# 3. Create custom predict function that returns the predicted values as a
#    vector (probability of purchasing in our example)
pred <- function(model, newdata)  {
results <- as.data.frame(h2o.predict(model, as.h2o(newdata)))
return(results[[3L]])
}
# create predictor object to pass to explainer functions
predictor.glm <- Predictor$new(
model = glm, 
data = features, 
y = response, 
predict.fun = pred,
class = "classification"
)
imp.glm <- FeatureImp$new(predictor.glm, loss = "mse")

获得错误：

Error in `[.data.frame`(prediction, , self$class, drop = FALSE): undefined columns 
selected
traceback()
1. FeatureImp$new(predictor.glm, loss = "mse")
2. .subset2(public_bind_env, "initialize")(...)
3. private$run.prediction(private$sampler$X)
4. self$predictor$predict(data.frame(dataDesign))
5. prediction[, self$class, drop = FALSE]
6. `[.data.frame`(prediction, , self$class, drop = FALSE)
7. stop("undefined columns selected")

Answer 1

在iml包文档中，它说class参数是"要返回的类列&"；。当您设置class = "classification"时，它正在查找一个名为"的列；分类"；未找到。至少在GitHub上，自那篇博客文章以来，iml包似乎已经经过了相当多的开发，所以我想有些功能可能不再向后兼容了。

在阅读了软件包文档后，我认为您可能想尝试以下内容：

predictor.glm <- Predictor$new(
model = glm, 
data = features, 
y = "Attrition",
predict.function = pred,
type = "prob"
)
# check ability to predict first
check <- predictor.glm$predict(features)
print(check)

更好的可能是利用H2O在机器学习可解释性方面的广泛功能。

h2o.varimp(glm)将为用户提供每个特征的可变重要性

h2o.varimp_plot(glm, 10)将呈现一个图形，显示每个特征的相对重要性。

h2o.explain(glm, as.h2o(features))是可解释性接口的包装器，默认情况下将提供混淆矩阵(在这种情况下(以及每个特征的变量重要性和部分依赖关系图。

对于某些算法(例如，基于树的方法(，h2o.shap_explain_row_plot()和h2o.shap_summary_plot()将提供形状贡献。

h2o-3文档可能有助于探索更多