如何测试Weka文本分类(FilteredClassifier)

Question

为此看了很多例子，但到目前为止运气不佳。我想对自由文本进行分类。

1. 配置文本分类器。(使用StringToWordVector和LibSVM的FilteredClassifier)
2. 训练分类器(添加大量文档，对过滤后的文本进行训练)
3. 将FilteredClassifier序列化到磁盘，退出应用程序

然后

1. 加载序列化的FilteredClassifier
2. 把东西分类

当我尝试从磁盘中读取并对事物进行分类时，它就可以了。所有的文档和示例都显示了同时构建的训练列表和测试列表，在我的情况下，我试图在事后构建一个测试列表。

仅使用FilteredClassifier不足以创建与原始训练集具有相同"字典"的测试实例，那么我如何保存以后需要分类的所有内容

http://weka.wikispaces.com/Use+WEKA+in+your+Java+代码只是说"从某个地方加载的实例"，而没有说使用类似的字典。

ClassifierFramework cf = new WekaSVM();
if (!cf.isTrained()) {
train(cf); // Train, save to disk
cf = new WekaSVM(); // reloads from file
}
cf.test("this is a test");

最终抛出

java.lang.ArrayIndexOutOfBoundsException: 2
at weka.core.DenseInstance.value(DenseInstance.java:332)
at weka.filters.unsupervised.attribute.StringToWordVector.convertInstancewoDocNorm(StringToWordVector.java:1587)
at weka.filters.unsupervised.attribute.StringToWordVector.input(StringToWordVector.java:688)
at weka.classifiers.meta.FilteredClassifier.filterInstance(FilteredClassifier.java:465)
at weka.classifiers.meta.FilteredClassifier.distributionForInstance(FilteredClassifier.java:495)
at weka.classifiers.AbstractClassifier.classifyInstance(AbstractClassifier.java:70)
at ratchetclassify.lab.WekaSVM.test(WekaSVM.java:125)

Answer 1

序列化包含训练数据定义的Instances-类似字典？-在序列化分类器时：

Instances trainInstances = ... //
Instances trainHeader = new Instances(trainInstances, 0);
trainHeader.setClassIndex(trainInstances .classIndex());
OutputStream os = new FileOutputStream(fileName);
ObjectOutputStream objectOutputStream = new ObjectOutputStream(os);
objectOutputStream.writeObject(classifier);
if (trainHeader != null)
objectOutputStream.writeObject(trainHeader);
objectOutputStream.flush();
objectOutputStream.close();

具体化：

Classifier classifier = null;
Instances trainHeader = null;
InputStream is = new BufferedInputStream(new FileInputStream(fileName));
ObjectInputStream objectInputStream = new ObjectInputStream(is);
classifier = (Classifier) objectInputStream.readObject();
try { // see if we can load the header
trainHeader = (Instances) objectInputStream.readObject();
} catch (Exception e) {
} 
objectInputStream.close();

使用trainHeader创建新的Instance:

int numAttributes = trainHeader.numAttributes();
double[] vals = new double[numAttributes];
for (int i = 0; i < numAttributes - 1; i++) {
Attribute attribute = trainHeader.attribute(i);
//If your attribute is nominal or string:       
double value = attribute.indexOfValue(myStrVal); //get myStrVal from your source
//If your attribute is numeric
double value = myNumericVal; //get myNumericVal from your source
vals[i] = value;
}
vals[numAttributes] = Instance.missingValue();
Instance instance = new Instance(1.0, vals);
instance.setDataset(trainHeader);
return instance;