NLTK和Scikit中两组不同大小的关键词的计算精度和召回率

Question

我正在尝试计算两组关键字的精度和召回率。gold_standard有823个项，test有1497个项。

使用nltk.metrics的precision和recall版本，我能够很好地提供这两个集合。但对Scikit做同样的事情会给我带来一个错误：

ValueError：发现样本数量不一致的数组：[823 1497]

如何解决此问题？

#!/usr/bin/python3
from nltk.metrics import precision, recall
from sklearn.metrics import precision_score
from sys import argv
from time import time
import numpy
import csv
def readCSVFile(filename):
termList = set()
with open(filename, 'rt', encoding='utf-8') as f:
reader = csv.reader(f)
for row in reader:
termList.update(row)
return termList
def readDocuments(gs_file, fileToProcess):
print("Reading CSV files...")
gold_standard = readCSVFile(gs_file)
test = readCSVFile(fileToProcess)
print("All files successfully read!")
return gold_standard, test
def calcPrecisionScipy(gs, test):
gs = numpy.array(list(gs))
test = numpy.array(list(test))
print("Precision Scipy: ",precision_score(gs, test, average=None))
def process(datasest):
print("Processing input...")
gs, test = dataset
print("Precision: ", precision(gs, test))
calcPrecisionScipy(gs, test)
def usage():
print("Usage: python3 generate_stats.py gold_standard.csv termlist_to_process.csv")
if __name__ == '__main__':
if len(argv) != 3:
usage()
exit(-1)
t0 = time()
process(readDocuments(argv[1], argv[2]))
print("Total runtime: %0.3fs" % (time() - t0))

我参考了以下页面进行编码：

• http://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html
• http://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html#sklearn.metrics.precision_score

=========================================更新==========================

好吧，所以我试图将"非敏感"数据添加到列表中，使其长度相等：

def calcPrecisionScipy(gs, test):
if len(gs) < len(test):
gs.update(list(range(len(test)-len(gs))))
gs = numpy.array(list(gs))
test = numpy.array(list(test))
print("Precision Scipy: ",precision_score(gs, test, average=None))

现在我有另一个错误：

UndefinedMetricWarning:精度定义错误，在没有预测样本的标签中设置为0.0。

Answer 1

在科学上似乎不可能计算两组不同长度的精度或召回率。我想nltk必须做的是将集合截断为相同的长度，您可以在脚本中执行同样的操作。

import numpy as np
import sklearn.metrics
set1 = [True,True]
set2 = [True,False,False]
length = np.amin([len(set1),len(set2)])
set1 = set1[:length]
set2 = set2[:length]
print sklearn.metrics.precision_score(set1,set2))