使用 ETAG 的 S3 流式传输(节点)



我们使用 aws-sdk 中的createReadStream()在节点中流式传输来自 S3 的内容。 我们想添加 etag 支持。 如果我们从客户端添加"If-None-Match"标头,s3 会将 NotModified 作为我似乎无法处理的错误抛出。

retrieveFile = function(req, res) {
var s3 = new AWS.S3(); 
var params = {
Bucket: bucket,
key: key
};
if (req.get('If-None-Match')) {
params.IfNoneMatch = req.get('If-None-Match');
}
return s3.getObject(params).on('httpHeaders', function(statusCode, headers) {
if (headers.etag) {
res.set('etag', headers.etag);
}
if (headers['content-length']) {
return res.set('content-length', headers['content-length']);
}
}).createReadStream().pipe(res);
};

我尝试收听流中的事件,并对从 getObject 返回的请求使用回调。 我可以通过这种方式获取错误的消息,但是 aws-sdk 中的其他内容似乎正在扼杀我的进程。

/Projects/my-app/node_modules/aws-sdk/lib/request.js:31
throw err;
^
NotModified: null
at Request.extractError (/Projects/my-app/node_modules/aws-sdk/lib/services/s3.js:519:35)
at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/request.js:673:14)
at Request.transition (/Projects/my-app/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:675:12)
at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/request.js:673:14)
at Request.transition (/Projects/my-app/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:675:12)
at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
at callNextListener (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
at IncomingMessage.onEnd (/Projects/my-app/node_modules/aws-sdk/lib/event_listeners.js:244:11)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)

似乎错误是在流中抛出的,如果您正在 AWS 上收听。请求您将收到错误,但仍未捕获流错误。

考虑一下:

s3.getObject({
Bucket: 'foo',
Key: 'bar'
}).on('error', function (err) {
console.log('Error event!');
}).createReadStream();

这将显示"错误事件!",但它也会抛出错误并退出进程。这是因为由createReadStream创建的流也收到错误。

考虑一下:

s3.getObject({
Bucket: 'lalaland',
Key: 'blabaliets'
}).on('error', function (err) {
console.log('Error event!');
}).createReadStream().on('error', function (err) {
console.log('Error event on stream!');
});

它将在 AWS 上侦听。请求对象错误,但也在流对象上。这将首先打印"错误事件!",然后打印"流中的错误事件!",并且进程不会退出。

请注意,这:

s3.getObject({
Bucket: 'lalaland',
Key: 'blabaliets'
}).createReadStream().on('error', function (err) {
console.log('Error event on stream!');
});

只会打印"流中的错误事件!"而不退出。最后一个选项可能是您想要的:仅侦听流上的错误,而不侦听请求对象上的错误。

回到你的初衷;此时(在流上的打开错误中),你想检查它是否确实是一个 NotModified 错误,并执行类似res.status(304).end()的操作。

最新更新