属性错误："列表"对象没有属性"较低"：聚类

Question

我正在尝试进行集群。我和熊猫和sklearn在一起。

import pandas
import pprint
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score
from sklearn.feature_extraction.text import TfidfVectorizer
dataset = pandas.read_csv('text.csv', encoding='utf-8')
dataset_list = dataset.values.tolist()

vectors = TfidfVectorizer()
X = vectors.fit_transform(dataset_list)
clusters_number = 20
model = KMeans(n_clusters = clusters_number, init = 'k-means++', max_iter = 300, n_init = 1)
model.fit(X)
centers = model.cluster_centers_
labels = model.labels_
clusters = {}
for comment, label in zip(dataset_list, labels):
print ('Comment:', comment)
print ('Label:', label)
try:
clusters[str(label)].append(comment)
except:
clusters[str(label)] = [comment]
pprint.pprint(clusters)

但是我有以下错误，尽管我从未使用过lower((：

File "clustering.py", line 19, in <module>
X = vetorizer.fit_transform(dataset_list)
File "/usr/lib/python3/dist-packages/sklearn/feature_extraction/text.py", line 1381, in fit_transform
X = super(TfidfVectorizer, self).fit_transform(raw_documents)
File "/usr/lib/python3/dist-packages/sklearn/feature_extraction/text.py", line 869, in fit_transform
self.fixed_vocabulary_)
File "/usr/lib/python3/dist-packages/sklearn/feature_extraction/text.py", line 792, in _count_vocab
for feature in analyze(doc):
File "/usr/lib/python3/dist-packages/sklearn/feature_extraction/text.py", line 266, in <lambda>
tokenize(preprocess(self.decode(doc))), stop_words)
File "/usr/lib/python3/dist-packages/sklearn/feature_extraction/text.py", line 232, in <lambda>
return lambda x: strip_accents(x.lower())
AttributeError: 'list' object has no attribute 'lower'

我不明白，我的文本(text.csv(已经是小写了。我从来没有打电话给lower((

数据：

你好，希望取消订单，谢谢确认

你好，我想取消今天的订单商店世界

尺寸床不兼容想知道如何通过取消退款今天诚挚地发送

你好，可能真诚地取消订单

你好想取消订单要求退款

你好，希望取消此订单，可以诚挚地指示流程

你好，看到日期交货想取消订单谢谢

你好，想取消匹配的订单，交货期为111111

嗨，我想取消这个订单

你好订单产品商店取消行为doublon提前衷心感谢

你好希望取消订单谢谢退款问候

你好，可能会取消订单，请提前感谢，诚挚地转发

Answer 1

错误在这一行：

dataset_list = dataset.values.tolist()

您可以看到，dataset是熊猫数据帧，所以当您执行dataset.values时，它将转换为形状为(n_rows，1(的二维数据集(即使列数为1(。然后在此调用tolist()将生成一个列表列表，类似于以下内容：

print(dataset_list)
[[hello wish to cancel order thank you confirmation],
[hello would like to cancel order made today store house world],
[dimensions bed not compatible would like to know how to pass cancellation refund send today cordially]
...
...
...]]

如你所见，这里有两个方括号。

现在TfidfVectorizer只需要一个句子列表，而不需要列表列表，因此会出现错误(因为TfidfVectorizer假设内部数据是句子，但这里它是一个列表(。

所以你只需要这样做：

# Use ravel to convert 2-d to 1-d array
dataset_list = dataset.values.ravel().tolist()

或

# Replace `column_name` with your actual column header, 
# which converts DataFrame to Series
dataset_list = dataset['column_name'].values).tolist()