对于字符串/句子不起作用的最接近的匹配算法



我写了一个程序,该程序从用户那里提出问题。然后,它将该问题与预定义的问题列表匹配并返回答案。它应该是准确的,并且仅与接近(模糊匹配)或用户输入的问题相匹配。

我的sssce:

http://ideone.com/jtcf73

代码:

#include <iostream>
#include <cstdint>
#include <algorithm>
#include <numeric>
#include <functional>
int min_index(const std::vector<int>& list)
{
    int index = 0;
    int smallest = list[0];
    for (size_t i = 0; i < list.size(); ++i) {
        if (list[i] < smallest) {
            smallest = list[i];
            index = i;
        }
    }
    return index;
}
std::uint32_t LevenshteinDistance(const std::string &First, const std::string &Second)
{
    const std::size_t FirstLength = First.size();
    const std::size_t SecondLength = Second.size();
    std::vector<std::uint32_t> Current(SecondLength + 1);
    std::vector<std::uint32_t> Previous(SecondLength + 1);
    std::size_t I = 0;
    std::generate(Previous.begin(), Previous.end(), [&] {return I++; });
    for (I = 0; I < FirstLength; ++I)
    {
        Current[0] = I + 1;
        for (std::size_t J = 0; J < SecondLength; ++J)
        {
            auto Cost = First[I] == Second[J] ? 0 : 1;
            Current[J + 1] = std::min(std::min(Current[J] + 1, Previous[J + 1] + 1), Previous[J] + Cost);
        }
        Current.swap(Previous);
    }
    return Previous[SecondLength];
}
std::vector<std::string> questions = 
{
    "What is the most popular program at GBC?",
    "How much is the tuition at GBC?",
    "Do I have to pay my fees before I can register?",
    "What are my fee payment options?",
    "How do I know when I'm allowed to register?",
    "How do I add and/or drop courses from my timetable?",
    "What do I do if I can't find my PASSWORD?",
    "How do I withdraw from a program?",
    "What are the college policies?",
    "How much math do I need to know?",
    "What is the program code for computer programming?",
    "What is stu-view?",
    "What is the college best known for?",
    "How easy is it to find work after program completion?",
    "What if I plan to continue my education after?"
};
std::vector<std::string> answers =
{
    "Fashion",
    "3000 a semester",
    "Yes you have to pay the fees before registering",
    "You may pay online on your student account through the student portal",
    "You may register two weeks or more before the start of the program",
    "You may drop courses from online through the student portal",
    "You can call ... and an agent will assist you",
    "You may withdraw using the student portal online",
    "They are located at the following link...",
    "It depends on the program you are entering",
    "T127 is the code for computer science",
    "Stu-View is a student portal to manage student account and view marks.",
    "The cafeteria food",
    "Depends on the field of work and timing",
    "You may do so within three years after program completion"
};
int main()
{
    std::string user_question = "program";
    std::vector<int> distances = std::vector<int>(questions.size(), 0);
    for (size_t I = 0; I < questions.size(); ++I)
    {
        int dist = LevenshteinDistance(user_question, questions[I]);
        distances[I] = dist;
    }
    std::cout<<"Distance:      "<<distances[min_index(distances)]<<"n";
    std::cout<<"User-Question: "<<user_question<<"n";
    std::cout<<"Question-Key:  "<<questions[min_index(distances)]<<"n";
    std::cout<<"Answer-Value:  "<<answers[min_index(distances)]<<"n";
    return 0;
}

因此,在上面,用户输入"program" ..应该从问题列表中找到最接近的匹配并返回相应的答案。

但是,它打印:

Distance:      17
User-Question: program
Question-Key:  What is stu-view?
Answer-Value:  Stu-View is a student portal to manage student account and view marks.

它应该具有更好的结果或准确性,但似乎并不在乎句子是否具有用户输入的关键字:S对小案例有效,但对于大型数据库或上面有更多的数据库超过5个句子,它很难..尤其是很少的关键字;

我在做什么错?有什么想法我如何修复它并使其更准确?我尝试了hammingdistance,类似的结果。

与其他字符串相比,"program""What is stu-view?"都很短。尽管"program"一词很常见,但将"program"转换为"What is stu-view?"的CC_4比"program"更容易。

我在做什么错?

我不认为您做错了什么。如果您对结果不满意,这意味着您当前的形式主义(最小化Levenshtein距离)不是您想要的。

您可以选择更多本地解决方案:例如引导字符串,计算单词之间的成对利文斯素距离,然后合并结果(平均,sup inf ...)

更好的解决方案需要进行一些书目(可能是无监督的机器学习主题)

最新更新