小贝子编程

416从CloudFront s3 Origin Video/MP4资源替换为更大的文件时



为什么在以下情况下，云方面响应416状态，即使我相信应该满足吗？/p>

原始S3对象

场景：

  1. 配置具有S3 Origin的CloudFront Distribution，对象缓存设置为"自定义"，min/max/default ttl全部设置为0
  2. 将视频/MP4文件上传到S3，没有相关的标头设置
  3. Chrome中的播放视频，通过CloudFront

在这一点上，一切都很好。我看到了3个网络请求到资源 - 为什么Chrome请求在2和3中重叠范围，但我不明白，但是此时，视频播放不错。

覆盖S3对象

现在，不久之后：

  1. 将较大（〜2x）文件上传到同一S3位置
  2. Chrome中的播放视频，通过CloudFront

这次发生了错误。

似乎可以的1个还可以（即内容长度看起来正确） - 尽管不确定为什么它是 Miss from cloudfront-我希望在TTL设置

的情况下，我会期望它是RefreshHit

为什么第二页的" req 2"是416，即使请求的范围位于resp 1？

的内容长度之内，即使所请求的范围在

注意：

  • 直接进入S3，没有问题
  • 将较大的文件替换回较小的文件，视频再次播放
  • 将文件上传到S3上使用" CACHE-CORTROL：NO-CACHE"，这没问题，但是我总是会得到"从云范围内错过"。我的理解是，CloudFront应该始终与S3进行检查，以查看此配置是否已更改
  • 等待一段时间（例如30年代） - 问题解决了
  • 以前的尝试复制这种情况的尝试看到了覆盖物后从S3到CloudFront的添加304响应，但是我无法复制这种确切的行为（现在只有一个216）

logs

原始S3对象

chrome 

[
  {
    "url": "https://dvayusv1lektq.cloudfront.net/video.mp4?u=1481498631683",
    "response_status": 206,
    "request_headers": [
      { "name": ":path", "value": "/video.mp4?u=1481498631683" },
      { "name": "pragma", "value": "no-cache" },
      { "name": "accept-encoding", "value": "identity;q=1, *;q=0" },
      { "name": "accept-language", "value": "en-US,en;q=0.8,fr;q=0.6,id;q=0.4" },
      { "name": "user-agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" },
      { "name": "accept", "value": "*/*" },
      { "name": "cache-control", "value": "no-cache" },
      { "name": ":authority", "value": "dvayusv1lektq.cloudfront.net" },
      { "name": ":scheme", "value": "https" },
      { "name": "range", "value": "bytes=0-" },
      { "name": ":method", "value": "GET" }
    ],
    "response_headers": [
      { "name": "date", "value": "Sun, 11 Dec 2016 23:23:54 GMT" },
      { "name": "via", "value": "1.1 0ea9662a9e73b2ca5836ede6924f81b0.cloudfront.net (CloudFront)" },
      { "name": "last-modified", "value": "Sun, 11 Dec 2016 23:23:45 GMT" },
      { "name": "server", "value": "AmazonS3" },
      { "name": "etag", "value": ""4ab3cf8dcd7747d45c1723eb19c0c7fa"" },
      { "name": "status", "value": "206" },
      { "name": "x-cache", "value": "Miss from cloudfront" },
      { "name": "content-type", "value": "video/mp4" },
      { "name": "content-range", "value": "bytes 0-535350/535351" },
      { "name": "accept-ranges", "value": "bytes" },
      { "name": "content-length", "value": "535351" },
      { "name": "x-amz-cf-id", "value": "-6zzzNwipKKtO_L-vU3o4dbH30cBHV2zu-28rZXwVrZm5uI8oKADYw==" }
    ]
  },
  {
    "url": "https://dvayusv1lektq.cloudfront.net/video.mp4?u=1481498631683",
    "response_status": 206,
    "request_headers": [
      { "name": ":path", "value": "/video.mp4?u=1481498631683" },
      { "name": "pragma", "value": "no-cache" },
      { "name": "accept-encoding", "value": "identity;q=1, *;q=0" },
      { "name": "accept-language", "value": "en-US,en;q=0.8,fr;q=0.6,id;q=0.4" },
      { "name": "user-agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" },
      { "name": "accept", "value": "*/*" },
      { "name": "cache-control", "value": "no-cache" },
      { "name": ":authority", "value": "dvayusv1lektq.cloudfront.net" },
      { "name": ":scheme", "value": "https" },
      { "name": "if-match", "value": ""4ab3cf8dcd7747d45c1723eb19c0c7fa"" },
      { "name": "range", "value": "bytes=524288-" },
      { "name": ":method", "value": "GET" }
    ],
    "response_headers": [
      { "name": "date", "value": "Sun, 11 Dec 2016 23:23:54 GMT" },
      { "name": "via", "value": "1.1 0ea9662a9e73b2ca5836ede6924f81b0.cloudfront.net (CloudFront)" },
      { "name": "last-modified", "value": "Sun, 11 Dec 2016 23:23:45 GMT" },
      { "name": "server", "value": "AmazonS3" },
      { "name": "etag", "value": ""4ab3cf8dcd7747d45c1723eb19c0c7fa"" },
      { "name": "status", "value": "206" },
      { "name": "x-cache", "value": "RefreshHit from cloudfront" },
      { "name": "content-type", "value": "video/mp4" },
      { "name": "content-range", "value": "bytes 524288-535350/535351" },
      { "name": "accept-ranges", "value": "bytes" },
      { "name": "content-length", "value": "11063" },
      { "name": "x-amz-cf-id", "value": "_z3F_A7pVXHz5PBulj8-4OeRolEzWdgT9R4-JdvgUpTLq463MZ-C_A==" }
    ]
  },
  {
    "url": "https://dvayusv1lektq.cloudfront.net/video.mp4?u=1481498631683",
    "response_status": 206,
    "request_headers": [
      { "name": ":path", "value": "/video.mp4?u=1481498631683" },
      { "name": "pragma", "value": "no-cache" },
      { "name": "accept-encoding", "value": "identity;q=1, *;q=0" },
      { "name": "accept-language", "value": "en-US,en;q=0.8,fr;q=0.6,id;q=0.4" },
      { "name": "user-agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" },
      { "name": "accept", "value": "*/*" },
      { "name": "cache-control", "value": "no-cache" },
      { "name": ":authority", "value": "dvayusv1lektq.cloudfront.net" },
      { "name": ":scheme", "value": "https" },
      { "name": "if-match", "value": ""4ab3cf8dcd7747d45c1723eb19c0c7fa"" },
      { "name": "range", "value": "bytes=32768-" },
      { "name": ":method", "value": "GET" }
    ],
    "response_headers": [
      { "name": "date", "value": "Sun, 11 Dec 2016 23:23:54 GMT" },
      { "name": "via", "value": "1.1 0ea9662a9e73b2ca5836ede6924f81b0.cloudfront.net (CloudFront)" },
      { "name": "last-modified", "value": "Sun, 11 Dec 2016 23:23:45 GMT" },
      { "name": "server", "value": "AmazonS3" },
      { "name": "etag", "value": ""4ab3cf8dcd7747d45c1723eb19c0c7fa"" },
      { "name": "status", "value": "206" },
      { "name": "x-cache", "value": "RefreshHit from cloudfront" },
      { "name": "content-type", "value": "video/mp4" },
      { "name": "content-range", "value": "bytes 32768-535350/535351" },
      { "name": "accept-ranges", "value": "bytes" },
      { "name": "content-length", "value": "502583" },
      { "name": "x-amz-cf-id", "value": "8MGICqcddKwl5HZ2sNN6fpTSwO1I8qkvvurVfbBftlikXKdi-FQhdQ==" }
    ]
  }
]

云范围

#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version
2016-12-11      23:23:53        MEL50   34771   150.101.108.33  GET     dvayusv1lektq.cloudfront.net    /video.mp4      206     -       Mozilla/5.0%2520(Macintosh;%2520Intel%2520Mac%2520OS%2520X%252010_12_1)%2520AppleWebKit/537.36%2520(KHTML,%2520like%2520Gecko)%2520Chrome/55.0.2883.87%2520Safari/537.36        u=1481498631683 -       Error   -6zzzNwipKKtO_L-vU3o4dbH30cBHV2zu-28rZXwVrZm5uI8oKADYw==        dvayusv1lektq.cloudfront.net    https   46      1.613   -       TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256     Miss    HTTP/2.0
2016-12-11      23:23:56        MEL50   11408   150.101.108.33  GET     dvayusv1lektq.cloudfront.net    /video.mp4      206     -       Mozilla/5.0%2520(Macintosh;%2520Intel%2520Mac%2520OS%2520X%252010_12_1)%2520AppleWebKit/537.36%2520(KHTML,%2520like%2520Gecko)%2520Chrome/55.0.2883.87%2520Safari/537.36        u=1481498631683 -       RefreshHit      _z3F_A7pVXHz5PBulj8-4OeRolEzWdgT9R4-JdvgUpTLq463MZ-C_A==        dvayusv1lektq.cloudfront.net    https   47      2.194   -       TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256     RefreshHit      HTTP/2.0
2016-12-11      23:23:56        MEL50   503468  150.101.108.33  GET     dvayusv1lektq.cloudfront.net    /video.mp4      206     -       Mozilla/5.0%2520(Macintosh;%2520Intel%2520Mac%2520OS%2520X%252010_12_1)%2520AppleWebKit/537.36%2520(KHTML,%2520like%2520Gecko)%2520Chrome/55.0.2883.87%2520Safari/537.36        u=1481498631683 -       RefreshHit      8MGICqcddKwl5HZ2sNN6fpTSwO1I8qkvvurVfbBftlikXKdi-FQhdQ==        dvayusv1lektq.cloudfront.net    https   47      0.259   -       TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256     RefreshHit      HTTP/2.0

S3 

8cd9b72de42431df1df4dadadab73aabf29ea0b34b5d821565fe4a16a3080509 bad-video-test [11/Dec/2016:23:23:44 +0000] 150.101.108.33 arn:aws:iam::<IAM ID>:user/<username> FB18FDABAA5DF6CA REST.PUT.OBJECT video.mp4 "PUT /video.mp4 HTTP/1.1" 200 - - 535351 6448 118 "-" "aws-cli/1.9.11 Python/2.7.10 Darwin/16.1.0 botocore/1.4.26" -
8cd9b72de42431df1df4dadadab73aabf29ea0b34b5d821565fe4a16a3080509 bad-video-test [11/Dec/2016:23:23:53 +0000] 54.239.202.78 - 934DB75AC20953C3 REST.GET.OBJECT video.mp4 "GET /video.mp4 HTTP/1.1" 206 - 535351 535351 101 98 "-" "Amazon CloudFront" -
8cd9b72de42431df1df4dadadab73aabf29ea0b34b5d821565fe4a16a3080509 bad-video-test [11/Dec/2016:23:23:54 +0000] 54.239.202.78 - 56EF7496F985D4B3 REST.GET.OBJECT video.mp4 "GET /video.mp4 HTTP/1.1" 304 - - 535351 11 - "-" "Amazon CloudFront" -
8cd9b72de42431df1df4dadadab73aabf29ea0b34b5d821565fe4a16a3080509 bad-video-test [11/Dec/2016:23:23:56 +0000] 54.239.202.78 - 7F0087B4769D0FD3 REST.GET.OBJECT video.mp4 "GET /video.mp4 HTTP/1.1" 304 - - 535351 4 - "-" "Amazon CloudFront" -

覆盖S3对象

chrome 

[
  {
    "url": "https://dvayusv1lektq.cloudfront.net/video.mp4?u=1481498656967",
    "response_status": 206,
    "request_headers": [
      { "name": ":path", "value": "/video.mp4?u=1481498656967" },
      { "name": "pragma", "value": "no-cache" },
      { "name": "accept-encoding", "value": "identity;q=1, *;q=0" },
      { "name": "accept-language", "value": "en-US,en;q=0.8,fr;q=0.6,id;q=0.4" },
      { "name": "user-agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" },
      { "name": "accept", "value": "*/*" },
      { "name": "cache-control", "value": "no-cache" },
      { "name": ":authority", "value": "dvayusv1lektq.cloudfront.net" },
      { "name": ":scheme", "value": "https" },
      { "name": "range", "value": "bytes=0-" },
      { "name": ":method", "value": "GET" }
    ],
    "response_headers": [
      { "name": "date", "value": "Sun, 11 Dec 2016 23:24:19 GMT" },
      { "name": "via", "value": "1.1 0ea9662a9e73b2ca5836ede6924f81b0.cloudfront.net (CloudFront)" },
      { "name": "last-modified", "value": "Sun, 11 Dec 2016 23:24:07 GMT" },
      { "name": "server", "value": "AmazonS3" },
      { "name": "etag", "value": ""d4d5776a96931962b41476857f34ab6d"" },
      { "name": "status", "value": "206" },
      { "name": "x-cache", "value": "Miss from cloudfront" },
      { "name": "content-type", "value": "video/mp4" },
      { "name": "content-range", "value": "bytes 0-956851/956852" },
      { "name": "accept-ranges", "value": "bytes" },
      { "name": "content-length", "value": "956852" },
      { "name": "x-amz-cf-id", "value": "CjmlHAFcyEWCiV68Q0G3gltuQSV7maR5bUoX0CfngDgDBp5fDvI38A==" }
    ]
  },
  {
    "url": "https://dvayusv1lektq.cloudfront.net/video.mp4?u=1481498656967",
    "response_status": 416,
    "request_headers": [
      { "name": ":path", "value": "/video.mp4?u=1481498656967" },
      { "name": "pragma", "value": "no-cache" },
      { "name": "accept-encoding", "value": "identity;q=1, *;q=0" },
      { "name": "accept-language", "value": "en-US,en;q=0.8,fr;q=0.6,id;q=0.4" },
      { "name": "user-agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" },
      { "name": "accept", "value": "*/*" },
      { "name": "cache-control", "value": "no-cache" },
      { "name": ":authority", "value": "dvayusv1lektq.cloudfront.net" },
      { "name": ":scheme", "value": "https" },
      { "name": "if-match", "value": ""d4d5776a96931962b41476857f34ab6d"" },
      { "name": "range", "value": "bytes=917504-" },
      { "name": ":method", "value": "GET" }
    ],
    "response_headers": [
      { "name": "date", "value": "Sun, 11 Dec 2016 23:24:19 GMT" },
      { "name": "via", "value": "1.1 0ea9662a9e73b2ca5836ede6924f81b0.cloudfront.net (CloudFront)" },
      { "name": "server", "value": "CloudFront" },
      { "name": "x-cache", "value": "Error from cloudfront" },
      { "name": "content-type", "value": "text/html" },
      { "name": "status", "value": "416" },
      { "name": "content-length", "value": "49" },
      { "name": "x-amz-cf-id", "value": "WIb50z_8rXTdqaC4CzUSYSL0kuIE9CWlCnKNgzps7AoCSRoJplBBbA==" },
      { "name": "expires", "value": "Sun, 11 Dec 2016 23:24:19 GMT" }
    ]
  },
  {
    "url": "https://dvayusv1lektq.cloudfront.net/video.mp4?u=1481498656967",
    "response_status": 0,
    "request_headers": [
      { "name": "User-Agent", "value": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" },
      { "name": "Range", "value": "bytes=32768-" },
      { "name": "Accept-Encoding", "value": "identity;q=1, *;q=0" }
    ],
    "response_headers": []
  }
]

云范围

#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version
2016-12-11      23:24:19        MEL50   52198   150.101.108.33  GET     dvayusv1lektq.cloudfront.net    /video.mp4      206     -       Mozilla/5.0%2520(Macintosh;%2520Intel%2520Mac%2520OS%2520X%252010_12_1)%2520AppleWebKit/537.36%2520(KHTML,%2520like%2520Gecko)%2520Chrome/55.0.2883.87%2520Safari/537.36        u=1481498656967 -       Error   CjmlHAFcyEWCiV68Q0G3gltuQSV7maR5bUoX0CfngDgDBp5fDvI38A==        dvayusv1lektq.cloudfront.net    https   46      1.551   -       TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256     Miss    HTTP/2.0
2016-12-11      23:24:19        MEL50   302     150.101.108.33  GET     dvayusv1lektq.cloudfront.net    /video.mp4      416     -       Mozilla/5.0%2520(Macintosh;%2520Intel%2520Mac%2520OS%2520X%252010_12_1)%2520AppleWebKit/537.36%2520(KHTML,%2520like%2520Gecko)%2520Chrome/55.0.2883.87%2520Safari/537.36        u=1481498656967 -       Error   WIb50z_8rXTdqaC4CzUSYSL0kuIE9CWlCnKNgzps7AoCSRoJplBBbA==        dvayusv1lektq.cloudfront.net    https   47      0.001   -       TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256     Error   HTTP/2.0

S3 

8cd9b72de42431df1df4dadadab73aabf29ea0b34b5d821565fe4a16a3080509 bad-video-test [11/Dec/2016:23:24:06 +0000] 150.101.108.33 arn:aws:iam::<IAM ID>:user/<username> E58826B8A66DBA1B REST.PUT.OBJECT video.mp4 "PUT /video.mp4 HTTP/1.1" 200 - - 956852 7367 53 "-" "aws-cli/1.9.11 Python/2.7.10 Darwin/16.1.0 botocore/1.4.26" -
8cd9b72de42431df1df4dadadab73aabf29ea0b34b5d821565fe4a16a3080509 bad-video-test [11/Dec/2016:23:24:18 +0000] 54.239.202.45 - BA1C06FEA7DAC83F REST.GET.OBJECT video.mp4 "GET /video.mp4 HTTP/1.1" 206 - 956852 956852 40 35 "-" "Amazon CloudFront" -

您看不到的是哪个对象 - 旧一个或新对象 - s3在覆盖之后的第一个请求中cloudfront服务于云。

当您在S3中覆盖现有对象时，覆盖始终是一个原子操作，从某种意义上说，每个请求都会满足完整的旧对象或完整的新对象...但是不能保证时机，因为S3具有最终覆盖现有对象的一致性模型。

Amazon S3提供最终的一致性，以用于覆盖所有区域的覆盖和删除。

单键更新是原子。例如，如果您输入现有密钥，则随后的读取可能会返回旧数据或更新的数据，但永远不会写入损坏或部分数据。

http://docs.aws.amazon.com/amazons3/latest/dev/introduction.html#consistencymodel

即使存储桶未启用版本启用，当两个对象都存在S3中时，时间窗口也很短。鉴于上述语句是正确的。

，这是其他任何方法。

同样，可以从S3请求不存在的对象，然后上传对象，然后再次请求它，然后在很短的时间内继续在下载成功之前，继续在短时间内获得404（或403，取决于存储桶配置）的响应。

相反，如果从来没有尝试下载不存在的对象，则在上传后，您总是会立即找到它可立即下载。这是一个实质上是大规模设计必要性的权衡。

存储桶和分发日志应该有助于揭示场景后面发生的事情，尤其是如果在上传后的任何下载尝试中从S3下载CloudFront下载的字节计数与旧对象一致。

。

在进一步反思后... 有第二个可能的解释在这里发生了什么，并且很难决定是否可以将其视为云层中的错误，如果确实如此实际问题。

在对象被替换为S3之后，让我们假设一会儿的一致性为不是一个问题 - 让我们假设上传后所有后续请求实际上会导致S3提供当前版本云范围的对象。

这里的问题可能是，该对象被替换后的第一次下载 - 应该导致CloudFront驱逐其对象的旧缓存版本 - 正在由浏览器取消。（我看过Chrome这样做，尽管我不知道为什么会这样做。）

取消下载时，来自原点的响应是不是缓存。

取消请求

如果对象不在边缘缓存中，并且如果cloudfront从您的来源获取对象，但在它可以传递请求的对象之前，查看器终止了会话（例如，关闭浏览器）边缘位置的对象。

http://docs.aws.amazon.com/amazoncloudfront/latest/developerguide/requestandresponseponsebehaviors3origin.html#response-s3-conse-s3-canceled-requests

授予，这说"如果对象不在边缘缓存中"，您可以说它实际上是...但是它可能是语义的问题：从s3请求的对象cloudfront是高速缓存中的不是 - 缓存中的内容是一个不同的对象（在同一URI处）。如果这种逻辑成立，那么"云额不会缓存对象"的断言大概仍然存在。

so ...请求2要求在 old 对象范围之外的范围，这是Cloudfront知道的唯一一件事。CloudFront尽职尽责地报告了一个事实，即 - 据此，该请求确实超出了范围，所请求的范围不满意。

可以说，CloudFront应该验证该对象是新鲜的，首先...但可以说，浏览器不应向未成功下载的对象提出后续范围请求。

也许我们可以使用相关的日志文件对此进行一些启示。

相关内容

最新更新


热门标签：

javascript python java c# php android html jquery c++ css ios sql mysql arrays asp.net json python-3.x ruby-on-rails .net sql-server django objective-c excel regex ruby linux ajax iphone xml vba spring asp.net-mvc database wordpress string postgresql wpf windows xcode bash git oracle list vb.net multithreading eclipse algorithm macos powershell visual-studio image forms numpy scala function api selenium