如何通过Java中的PrintWriter将数据分块写入csv

Question

我有一些数据正在处理，然后将其保存在字符串生成器中，最后写入csv。

目前，我正在使用处理某个idData数组后获得的println一次编写整个字符串生成器。但我想批量写入，所以假设id的数组数据长度为200，我想一次写入50个id的数据。

我该怎么做？

Existing code of writing the entire string builder result

class A {
private StringBuilder sb;
public void appendBody(String[] idData) {
for(String id: idData) {
Data data = getData(id);
processData(data);

}
writeToCsv();
} 
public void processData() {
.. processing that involves populating private property string builder
// adding some sample data here
this.sb.append("Name,Phone,Emailn");
this.sb.append("Glen,1234,[e-mail]");
this.sb.append("John,5678,[e-mail]");
// end populating string builder
}
public String writeToCsv() throws DataNotFoundException {
String filename = "outputFileNamePath";  // sample filename, modified to be pasted here
PrintWriter writer;
try {
writer = new PrintWriter(new File(filename));
} catch (FileNotFoundException e) {
writeLog("ERROR", e.getMessage);
throw new Exception("Error while creating output file");
}
writer.println(this.sb.toString());
writer.close();
return filename;
}
}

一些建议和提示会有所帮助。

p.S：请不要对日志消息或重新抛出的异常、变量的名称或任何与我的问题无关的细节发表评论，因为它们已经被修改为添加到这里，并提前原谅我的任何错误！

Answer 1

以下是您的问题的解决方案。

基本思想是控制对writeToCsv((的调用(这将等于所需的块大小(，并始终以附加模式打开文件

在这里，我使用mod操作来控制对writeToCsv((的调用。您可以添加任何符合您卡盘尺寸要求的逻辑。文本将在5次迭代后被推送到文件中。

此外，要在附加模式下打开文件，请使用

FileWriter fileWriter = new FileWriter(filename, true); // Set true for append mode
writer = new PrintWriter(fileWriter);

这是完整的程序。

public class WriteToFileInChunk {
private static StringBuilder sb = new StringBuilder();
public static void main(String[] args) throws IOException {
WriteToFileInChunk writeToFileInChunk = new WriteToFileInChunk();
writeToFileInChunk.writeInchuncks();
}
private void writeInchuncks() throws IOException {
String[] idData = { "1,", "2,", "3,", "4,", "5,", "6,", "7,", "8,", "9,", "10,", "11,", "12,", "13,", "14,",
"15,", "16,", "17,", "18,", "19,", "20" };
for (int i = 0; i < idData.length; i++) {
StringBuilder data = new StringBuilder(idData[i]);
sb.append(processData(data));
if (i % 5 == 0) {
writeToCsv(sb);
sb = new StringBuilder();
}
}
}
private StringBuilder processData(StringBuilder stringBuilder) {
return stringBuilder.append(System.currentTimeMillis() + ", Name, Phone,Emailn");
}
private String writeToCsv(StringBuilder sb) throws IOException {
String filename = "outputFileNamePath"; // sample filename, modified to be pasted here
PrintWriter writer;
try {
FileWriter fileWriter = new FileWriter(filename, true); // Set true for append mode
writer = new PrintWriter(fileWriter);
writer.println(sb.toString());
writer.close();
} catch (FileNotFoundException e) {
System.out.printf("ERROR : %s", e.getMessage());
new Exception("Error while creating output file", e);
}
return filename;
}
}

Answer 2

您需要将idData拆分为多批数组。然后每批处理完writeToCsv。(注意您必须清除sb(

public void appendBody(String[] idData) {
for (String[] batch : split(idData, BATCH_SIZE)) {
sb = new StringBuilder();
for (String id : batch) {
Data data = getData(id);
processData(data);
}
writeToCsv();
}
}

您可以使用以下代码来拆分数组：

static List<String[]> split(String[] idData, int batchSize) {
List<String[]> list = new ArrayList<>();
for (int i = 0; i < idData.length; i += batchSize) {
int realBatchSize = Math.min(batchSize, idData.length - i);
String[] batch = new String[realBatchSize];
for (int j = 0; j < realBatchSize; j++)
batch[j] = idData[j + i];
list.add(batch);
}
return list;
}

打开文件时，请记住使用APPEND模式。