将lucene FST文件从5.1.0迁移到8.9.0

Question

我有用lucene 5.1.0创建的FST文件。

升级到lucene 8.9.0后，当我试图从文件中读取FST时，我得到了异常:

org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource org.apache.lucene.store.InputStreamDataInput@34ce8af7): 4 (needs to be between 6 and 7). This version of Lucene only supports indexes created with release 6.0 and later.

是否有办法将旧的FST文件升级为新格式?

Answer 1

我是这样解决的。

将FST中的所有内容写入文本文件:

public static <T> void writeToTextFile(FST<T> fst, Path filePath) throws IOException {
try (BufferedWriter writer = Files.newBufferedWriter(filePath)) {
BytesRefFSTEnum<T> fstEnum = new BytesRefFSTEnum<>(fst);
while (fstEnum.next() != null) {
BytesRefFSTEnum.InputOutput<T> inputOutput = fstEnum.current();
writer.write(inputOutput.input.utf8ToString() + "t" + inputOutput.output.toString() + "n");
}
}
}

将lucene版本更改为new并从file中读取内容:

public static <T> FST<T> readFromTextFile(Path filePath, Outputs<T> outputs, Function<String, T> fromString) throws IOException {
Builder<T> builder = new Builder<>(FST.INPUT_TYPE.BYTE1, outputs);
IntsRefBuilder scratchInts = new IntsRefBuilder();
try (BufferedReader reader = Files.newBufferedReader(filePath)) {
String[] split = reader.readLine().split("t");
BytesRef scratchBytes = new BytesRef(split[0]);
builder.add(Util.toIntsRef(scratchBytes, scratchInts), fromString.apply(split[1]));
}
return builder.finish();
}