Java标记化字符串中每个令牌的字符数

Question

我想知道我是否可以计算每个令牌的字符数并显示这些信息，例如：

天被标记化，我的输出将是："；一天有3个字符"并对每个令牌继续这样做。

我打印出每个令牌中字符数的最后一个循环从未打印：

public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
ArrayList<String> tokenizedInput = new ArrayList<>();
String sentenceRetrieved;
// getting the sentence from the user
System.out.println("Please type a sentence containing at least 4 words, with a maximum of 8 words: ");
sentenceRetrieved = sc.nextLine();
StringTokenizer strTokenizer = new StringTokenizer(sentenceRetrieved);
// checking to ensure the string has 4-8 words
while (strTokenizer.hasMoreTokens()) {
if (strTokenizer.countTokens() > 8) {
System.out.println("Please re-enter a sentence with at least 4 words, and a maximum of 8");
break;
} else {
while (strTokenizer.hasMoreTokens()) {
tokenizedInput.add(strTokenizer.nextToken());
}
System.out.println("Thank you.");
break;
}
}
// printing out the sentence
System.out.println("You entered: ");
System.out.println(sentenceRetrieved);
// print out each word given
System.out.println("Each word in your sentence is: " + tokenizedInput);
// count the characters in each word
// doesn't seem to run
int totalLength = 0;
while (strTokenizer.hasMoreTokens()) {
String token;
token = sentenceRetrieved;
token = strTokenizer.nextToken();
totalLength += token.length();
System.out.println("Word: " + token + " Length:" + token.length());
}
}
}

控制台示例：

请键入一个包含至少4个单词的句子，最多8个单词：

你好，这是一个测试

谢谢。

您输入的：

你好，这是一个测试

你句子中的每个单词都是：[你好，有，这个，是，a，测试]

Answer 1

首先，我添加了必要的导入，并围绕这个主方法构建了一个类。这应该编译。

import java.util.ArrayList;
import java.util.Scanner;
import java.util.StringTokenizer;
public class SOQ_20200913_1
{
public static void main(String[] args) {

Scanner sc = new Scanner(System.in);

ArrayList<String> tokenizedInput = new ArrayList<>();
String sentenceRetrieved;

// getting the sentence from the user
System.out.println("Please type a sentence containing at least 4 words, with a maximum of 8 words: ");
sentenceRetrieved = sc.nextLine();
StringTokenizer strTokenizer = new StringTokenizer(sentenceRetrieved);

// checking to ensure the string has 4-8 words
while (strTokenizer.hasMoreTokens()) {
if (strTokenizer.countTokens() > 8) {
System.out.println("Please re-enter a sentence with at least 4 words, and a maximum of 8");
break;

} else {
while (strTokenizer.hasMoreTokens()) {
tokenizedInput.add(strTokenizer.nextToken());
}

System.out.println("Thank you.");
break;
}
}

// printing out the sentence
System.out.println("You entered: ");
System.out.println(sentenceRetrieved);

// print out each word given
System.out.println("Each word in your sentence is: " + tokenizedInput);

// count the characters in each word
// doesn't seem to run

int totalLength = 0;
while (strTokenizer.hasMoreTokens()) {
String token;
token = sentenceRetrieved;
token = strTokenizer.nextToken();
totalLength += token.length();
System.out.println("Word: " + token + " Length:" + token.length());
}

}
}

接下来，让我们看看这个工作示例。直到最后一个while循环(计算字符长度的循环(为止，似乎一切都很好。但如果您注意到，在最后一个循环之前的while循环将继续循环，直到它没有更多的令牌可获取为止。因此，在它完成收集所有令牌并且没有更多令牌可收集之后，您可以尝试创建最后的while循环，要求它收集更多令牌。直到收集的代币用完，它才到达while循环！

最后，为了解决这个问题，您可以简单地浏览您在倒数第二个while循环中添加的列表，并在最后一个循环中循环浏览它！

例如：

int totalLength = 0;
for (String each : tokenizedInput) {
totalLength += each.length();
System.out.println("Word: " + each + " Length:" + each.length());
}