将下载的缓冲区转换为节点中的可读文本



我正试图从网站下载一系列csv文件,并将它们连接到一个大文件中。每个文件的输出以开头

Number of records: x
Header row
record 1
record 2
....

我对缓冲区和流的概念还很陌生,但我决定将csv作为流下载并加载到缓冲区中进行操作。我有:

async function stream2buffer(stream) {
return new Promise((resolve, reject) => {
const _buf = [];
stream.on("data", (chunk) => _buf.push(chunk));
stream.on("end", () => resolve(Buffer.concat(_buf)));
stream.on("error", (err) => reject(err));
});
}
(async () => {
const finishedDownload = promisify(stream.finished);
const writer = fs.createWriteStream('./file1.csv');
const response = await axios({
method: 'GET',
url: URL,
responseType: 'stream',
});
const buf = await stream2buffer(response.data);
console.log('buffer?');
console.log(buf);
// response.data.pipe(writer);
// await finishedDownload(writer);
})();

没有错误,但输出不是文本:

buffer?
<Buffer 54 6f 74 61 6c 20 52 65 73 75 6c 74 73 3a 20 31 34 0d 0a 41 50 4e 2c 4f 77 6e 65 72 2c 41 64 64 72 65 73 73 2c 43 69 74 79 2c 5a 69 70 2c 53 75 62 64 ... 1731 more bytes>

如何将缓冲区信息转换为可以使用的文本?

我不知道这是否回答了你的问题:

console.log('buffer?');
// console.log(buf);
console.log(buf.toString());

看起来它是以十六进制输出的,这回答了你的问题吗?:

function hex_txt(hex) {
var str = '';
hex = hex.replace(/ /g, "")//.replace(/<Buffer/g, "").replace(/>/g, ""); // uncomment this if it doesn't work
for (let i = 0; i < hex.length; i += 2) {
str += String.fromCharCode(parseInt(hex.substr(i, 2), 16));
}
return str;
}
console.log(hex_txt(buf.toString())); //instead of console.log(buf), put this

没有太多细节,所以我不知道这是否正确。

最新更新