在Scikit-Learn(Python 3)中使用持久的SVM模型来探讨持续的SVM模型

Question

我正在使用Scikit-Learn在Python中进行机器学习程序，该程序将根据其内容对问题类型进行对电子邮件进行排序。例如：有人给我发电子邮件说"此程序没有启动"，该机器将其归类为"崩溃问题"。

我正在使用一种SVM算法，该算法读取电子邮件内容及其各自的类别标签，来自2个CSV文件。我写了两个程序：

1. 第一个程序训练机器并使用Joblib.dump（）导出训练的模型，以便第二个程序可以使用训练的模型
2. 第二个程序通过导入训练的模型来做出预测。我希望第二个程序能够通过重新安装分类器的新数据来更新受过训练的模型。但是我不确定如何完成此操作。预测程序要求用户在其中键入电子邮件，然后将进行预测。然后，它将询问用户其预测是否正确。在这两种情况下，我都希望机器从结果中学习。

培训程序：

import numpy as np
import pandas as pd
from pandas import DataFrame
import os
from sklearn import svm
from sklearn import preprocessing
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.externals import joblib

###### Extract and Vectorize the features from each email in the Training Data ######
features_file = "features.csv" #The CSV file that contains the descriptions of each email. Features will be extracted from this text data
features_df = pd.read_csv(features_file, encoding='ISO-8859-1') 
vectorizer = TfidfVectorizer()
features = vectorizer.fit_transform(features_df['Description'].values.astype('U')) #The sole column in the CSV file is labeled "Description", so we specify that here

###### Encode the class Labels of the Training Data ######
labels_file = "labels.csv" #The CSV file that contains the classification labels for each email
labels_df = pd.read_csv(labels_file, encoding='ISO-8859-1')
lab_enc = preprocessing.LabelEncoder()
labels = lab_enc.fit_transform(labels_df)

###### Create a classifier and fit it to our Training Data ######
clf = svm.SVC(gamma=0.01, C=100)
clf.fit(features, labels)

###### Output persistent model files ######
joblib.dump(clf, 'brain.pkl')
joblib.dump(vectorizer, 'vectorizer.pkl')
joblib.dump(lab_enc, 'lab_enc.pkl')
print("Training completed.")

预测程序：

import numpy as np
import os
from sklearn import svm
from sklearn import preprocessing
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.externals import joblib

###### Load our model from our training program ######
clf = joblib.load('brain.pkl')
vectorizer = joblib.load('vectorizer.pkl')
lab_enc = joblib.load('lab_enc.pkl')

###### Prompt user for input, then make a prediction ######
print("Type an email's contents here and I will predict its category")
newData = [input(">> ")]
newDataFeatures = vectorizer.transform(newData)
print("I predict the category is: ", lab_enc.inverse_transform(clf.predict(newDataFeatures)))

###### Feedback loop - Tell the machine whether or not it was correct, and have it learn from the response ######
print("Was my prediction correct? y/n")
feedback = input(">> ")
inputValid = False
while inputValid == False: 
    if feedback == "y" or feedback == "n":
        inputValid = True
    else:
        print("Response not understood. Was my prediction correct? y/n")
        feedback = input(">> ")
if feedback == "y":
    print("I was correct. I'll incorporate this new data into my persistent model to aid in future predictions.")
    #refit the classifier using the new features and label
elif feedback == "n":
    print("I was incorrect. What was the correct category?")
    correctAnswer = input(">> ")
    print("Got it. I'll incorporate this new data into my persistent model to aid in future predictions.")
    #refit the classifier using the new features and label

从我完成的阅读中，我收集到SVM并不真正支持增量学习，因此我认为我需要将新数据纳入旧的培训数据中，并每次都从头开始重新审阅整个模型添加的新数据。很好，但是我不太确定如何实际实施它。我是否需要预测程序来更新两个CSV文件以包括新数据，以便培训可以重新开始？

Answer 1

我最终弄清楚我问题的概念答案是我需要更新我最初使用的CSV文件来训练机器。收到反馈后，我简单地将新功能和标签写入了它们各自的CSV文件，然后可以用培训数据集中的新信息重新训练机器。