在Node.js中，从S3对象或web URL创建读流，并通过多个管道传递读流

Question

基于x的Lambda函数，我正在尝试

1. 从S3下载对象
2. 如果S3中对象不存在，则从web下载文件
3. 通过下载的数据，不管它是如何到达多个写流的

I tried following

var request = require('request');
const readStream = ({ Bucket, Key }) => {
s3.getObjectMetadata({ Bucket, Key })
.promise().then(() => {
return s3.getObject().createReadStream();
})
.catch(error => {
if (error.statusCode === 404) {
return request.get('http://example.com/' + Key);
}
});
};
readStream({ ... })
.pipe(sharp().resize(w, h).toFormat('png'))
.pipe(writeStream);

如果对象在s3中可用，则上面的

有效，但是catch块不起作用。

我需要await还是request.get的承诺?

我也试着跟随，没有运气

http.get('http://example.com/' + Key, function(response) { 
return response;
});

Answer 1

首先我们需要创建一个promise来检查对象是否存在于bucket

const isObjectExists = (params) => {
return s3.headObject(params)
.promise()
.then(
() => true,
(error) => {
if (err.statusCode === 404) {
return false;
}
throw error;
}
)}
const awsParams = { Bucket: ..., Key: s3ObjectKey };
const exists = await isObjectExists(awsParams);

接下来，我们需要从s3或web URL创建一个读流。

使用已弃用的request包，因为我无法使用http让事情工作。

let readStream = null;
if (exists) {
readStream = s3.getObject(awsParams).createReadStream();
} else {
readStream = request.get('http://example.com/' + s3ObjectKey);
}

将刚刚创建的readStream传递给进程，最后传到上传流

const pass = new PassThrough();
readStream.
.pipe(sharp().resize(w, h).toFormat('png'))
.pipe(s3.upload({
Body: pass,
Bucket,
ContentType: 'image/png',
s3ObjectKey + '-resized'
}).promise());
const uploadedData = await pass;

添加一些先决条件

import awS3 from 'aws-sdk/clients/s3';
import { PassThrough } from 'stream'
import * as request from 'request';
const s3 = new awS3({signatureVersion: 'v4'});