r语言 - 如何使用 MLR 包预处理新数据以进行预测

Question

如果我想使用 mlr 包对新数据进行预测，我该如何预处理新数据，以便使用原始数据预处理所需的信息。 例如，如果我合并小因子水平，并且新数据集中的频率与第一个数据集不同， 生成的因子水平可能不同，并且无法进行预测。注意：我在这里假设在训练模型时新数据尚不可用，这不是关于测试数据，而是关于预测新数据。那么，在 MLR 中应该如何完成新数据的预处理呢？下面是一个示例，其中我创建了一个新任务来预处理导致错误的新数据集：

library(mlr)
a <- data.frame(y=factor(c(1,1,1,1,1,1,1,1,0,0,1,0)), 
x1=rep(c("a","b", "c"), times=c(10,1,1)))
# most frequent x1 factor is "a"
aTask <- makeClassifTask(data = a, target = "y", positive="1")
aTask <- mergeSmallFactorLevels(aTask, cols=c("x1"), min.perc=0.1)
# combines "b" and "c" into factor ".merged"
getTaskData(aTask)
aLearner <- makeLearner("classif.rpart", predict.type = "prob")
model <- train(aLearner, aTask)
b <- data.frame(y=factor(c(1,0,1,1,1,1,1,1,0,0,1,0)), 
x1=rep(c("a","b", "c"), times=c(1,10,1)))
# most frequent x1 factor is "b"
# target would be made up, because at this stage there would be now target
# variable availabel
newdataTask <- makeClassifTask(data = b, target = "y", positive="1")
newdataTask <- mergeSmallFactorLevels(newdataTask, cols="x1", 
min.perc = 0.1)
# combines "a" and "c" into factor ".merged"
getTaskData(newdataTask)
pred <- predict(model, newdataTask)
#Error in model.frame.default(Terms, newdata, na.action = na.action, 
#                              xlev = attr(object,  : 
#Faktor 'x1' hat neue Stufen b (= factor 'x1' has new level b)

我的解决方案的另一个问题是，新任务似乎需要一个目标变量，而该变量不适用于新数据集。

Answer 1

>mlr不提供任何自动执行此操作的内容，但您可以轻松检查哪些因子水平已被替换并在新数据中相应地重命名：

library(plyr)
to.replace = setdiff(levels(b$x1), levels(getTaskData(aTask)$x1))
b$x1 = mapvalues(b$x1, from = to.replace, to = rep(".merged", times = length(to.replace)))

完整示例：

library(mlr)
a = data.frame(y=factor(c(1,1,1,1,1,1,1,1,0,0,1,0)), 
x1=rep(c("a","b", "c"), times=c(10,1,1)))
aTask = makeClassifTask(data = a, target = "y", positive="1")
aTask = mergeSmallFactorLevels(aTask, cols=c("x1"), min.perc=0.1)
aLearner = makeLearner("classif.rpart", predict.type = "prob")
model = train(aLearner, aTask)
b = data.frame(y=factor(c(1,0,1,1,1,1,1,1,0,0,1,0)), 
x1=rep(c("a","b", "c"), times=c(1,10,1)))
library(plyr)
to.replace = setdiff(levels(b$x1), levels(getTaskData(aTask)$x1))
b$x1 = mapvalues(b$x1, from = to.replace, to = rep(".merged", times = length(to.replace)))
newdataTask = makeClassifTask(data = b, target = "y", positive="1")
pred = predict(model, newdataTask)

对于这样的事情，通常最好将学习器与预处理融合在一起，以便在训练和预测时透明且自动地发生这种情况。在这种情况下，一个完整的示例如下所示：

lrn = makeLearner("classif.rpart", predict.type = "prob")
trainfun = function(data, target, args) {
task = makeClassifTask(data = data, target = target, positive = "1")
new.task = mergeSmallFactorLevels(task, cols = c("x1"), min.perc = 0.1)
return(list(data = getTaskData(new.task), control = list(levels(getTaskData(new.task)$x1))))
}
predictfun = function(data, target, args, control) {
library(plyr)
to.replace = setdiff(levels(data$x1), control[[1]])
data$x1 = mapvalues(data$x1, from = to.replace, to = rep(".merged", times = length(to.replace)))
return(data)
}
lrn = makePreprocWrapper(lrn, train = trainfun, predict = predictfun)
a = data.frame(y=factor(c(1,1,1,1,1,1,1,1,0,0,1,0)), 
x1=rep(c("a","b", "c"), times=c(10,1,1)))
aTask = makeClassifTask(data = a, target = "y", positive="1")
model = train(lrn, aTask)
b = data.frame(y=factor(c(1,0,1,1,1,1,1,1,0,0,1,0)), 
x1=rep(c("a","b", "c"), times=c(1,10,1)))
newdataTask = makeClassifTask(data = b, target = "y", positive = "1")
pred = predict(model, newdataTask)

这只是一个概念证明 - 您可能希望有参数来指定应处理哪些特征以及阈值应该是多少，并调整predictfun代码以处理任意数量的已处理特征。