具有来自分类器的预测值的数据帧

Question

我想知道如何使用具有预测值的新电子邮件创建新的panda数据帧。我使用了以下模型：

import string
from nltk.corpus import stopwords
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report,confusion_matrix, accuracy_score
from sklearn.naive_bayes import MultinomialNB
df = pd.DataFrame(data={'Email': [
"Hi, I am Andrew and I want too buy VIAGRA",
"Dear subscriber, your account will be closed",
"Please click below to verify and access email restore",
"Hi Anne, I miss you so much! Can’t wait to see you",
"Dear Professor Johnson, I was unable to attend class today",
"I am pleased to inform you that you have won our grand prize.",
"I can’t help you with that cuz it’s too hard.",
"I’m sorry to tell you but im sick and will not be able to come to class.",
"Can I see an example before all are shipped or will that cost extra?",
"I appreciate your assistance and look forward to hearing back from you.",], 
'Spam': [1, 1, 1, 0, 0, 1, 0, 0, 0, 0]})
def fun(text):    
# Removing Punctuations
remove_punc = [c for c in text if c not in string.punctuation]
remove_punc = ''.join(remove_punc)
# Removing StopWords
cleaned = [w for w in remove_punc.split() if w.lower() not in stopwords.words('english')]
return cleaned
vectorizer = CountVectorizer(analyzer=fun)
X = vectorizer.fit_transform(df['Email'])
X_train, X_test, y_train, y_test = train_test_split(X, df['Spam'], test_size = 0.25, random_state = 0)
classifier = MultinomialNB()
classifier.fit(X_train, y_train)
pred = classifier.predict(X_test)

给出一个新电子邮件列表：

new_email = ["Hi, my name is Christopher", "Buy this movie at a great price!!!", "Should we meet?"]

我想创建一个新的数据帧，其中包含这些电子邮件和预测值：

Email                                 Spam
Hi, my name is Christopher             1
Buy this movie at a great price!!!     1
University of London: meeting request  0

我为一封电子邮件所做的如下：

X_new = vectorizer.transform(fun(new_email))
and predicted as follows:
classifier.predict(X_new)

然而，我需要一个像上面那样的数据帧。你能帮我一下吗？

Answer 1

在X_new分配中删除fun()可以解决问题。您不需要这样做，因为fun已经在vectorizer的构造函数中指定为analyzer。

代码

new_email = [
"Hi, my name is Christopher", 
"Buy this movie at a great price!!!",
"Should we meet?"
]
X_new = vectorizer.transform(new_email)
pred_new = classifier.predict(X_new)
pd.DataFrame(
data={
"Email": new_email,
"Spam": pred_new
}
)

输出

Email  Spam
0          Hi, my name is Christopher     0
1  Buy this movie at a great price!!!     0
2                     Should we meet?     0

N.B.1.预测的垃圾邮件结果与您的帖子略有不同(第0行，第1行(，但整个过程应该是完整的。用先前训练过的样本替换句子表明该模型工作正常。我不确定这是否是训练过程中固有的随机性造成的。