如何修复错误ValueError:无法在python的NLP项目中将字符串转换为浮点数?

Question

我正在使用jupyter notebook编写python代码训练和测试数据集以返回正确的情感。

当我试图预测短语的情绪时，系统崩溃并显示以下错误的问题:

ValueError: could not convert string to float: 'this book was so .插话让我不开心。

注意我有一个不平衡的数据集，所以我使用SMOTE为了对数据集进行over_sampling

<标题>代码:

import pandas as pd
import numpy as np
from imblearn.over_sampling import SMOTE# for inbalance dataset
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import TfidfTransformer,TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score,classification_report,confusion_matrix
from sklearn.pipeline import Pipeline
df = pd.read_csv("data/Apple-Twitter-Sentiment-DFE.csv",encoding="ISO-8859-1")
df
# data is cleaned using preprocessing functions
# Solving inbalanced dataset using SMOTE 
vectorizer = TfidfVectorizer()
vect_df =vectorizer.fit_transform(df["clean_text"])
oversample = SMOTE(random_state = 42)
x_smote,y_smote = oversample.fit_resample(vect_df, df["sentiment"])
print("shape x before SMOTE: {}".format(vect_df.shape))
print("shape x after SMOTE: {}".format(x_smote.shape))
print("balance of targets feild %")
y_smote.value_counts(normalize = True)*100

# split the dataset into train and test 
x_train,x_test,y_train,y_test = train_test_split(x_smote,y_smote,test_size = 0.2,random_state =42)

logreg = Pipeline([
('tfidf', TfidfTransformer()),
('clf', LogisticRegression(n_jobs=1, C=1e5)),
])
logreg.fit(x_train, y_train)
y_pred = logreg.predict(x_test)
print('accuracy %s' % accuracy_score(y_pred, y_test))
print(classification_report(y_test, y_pred))
# Make prediction 
exl = "this book was so interstening it made me not happy"
logreg.predict(exl)

Answer 1

您应该这样定义变量exl:

exl = vectorizer.transform(["this book was so interstening it made me not happy"])

，然后做预测。

首先，将测试数据放在一个列表中，然后使用vectorizer使用从训练数据中提取的特征进行预测。