慢速 S3 -> EC2 传输。代码错误？

Question

我正在将此脚本与 PHP5-CLI 一起使用，将文件从 Amazon S3 下载到 EC2 实例，但它相当慢（160 MB 需要 1 分钟，大约 2 MB/s）。据说至少 10 MB/s 应该是可能的。我在代码中做错了什么吗？

require 'aws-sdk/sdk.class.php';
$s3 = new AmazonS3(array(
        'key' => '********',
        'secret' => '******************'
));
$s3->get_object('mahbucket', 'filename.tar.gz', array('fileDownload' => 'downloaded.tar.gz'));

存储桶处于eu-west-1，实例处于eu-west-1a

（交叉发布到 AWS 论坛）

Answer 1

您的脚本看起来不错，但是我不久前遇到了类似的问题，我无法解决...

我的新方法是使用 s3cmd 在 S3 存储桶和 EC2 实例之间进行文件传输。你只需要配置它一次，然后你就可以通过PHP触发它。它也更安全，因为您不必将 AWS 凭证存储在脚本中。

在我看来速度还可以：

user@mothership:~/s3# s3cmd put test.bin s3://data.example.com/test.bin
test.bin -> s3://data.example.com/test.bin  [part 1 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.39 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 2 of 13, 15MB]
 15728640 of 15728640   100% in    0s    15.55 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 3 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.18 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 4 of 13, 15MB]
 15728640 of 15728640   100% in    0s    17.32 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 5 of 13, 15MB]
 15728640 of 15728640   100% in    0s    18.87 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 6 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.58 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 7 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.29 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 8 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.95 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 9 of 13, 15MB]
 15728640 of 15728640   100% in    0s    15.33 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 10 of 13, 15MB]
 15728640 of 15728640   100% in    0s    17.01 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 11 of 13, 15MB]
 15728640 of 15728640   100% in    0s    15.36 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 12 of 13, 15MB]
 15728640 of 15728640   100% in    0s    16.56 MB/s  done
test.bin -> s3://data.example.com/test.bin  [part 13 of 13, 10MB]
 11256320 of 11256320   100% in    0s    14.15 MB/s  done
user@mothership:~/s3# s3cmd get s3://data.example.com/test.bin test.bin.new
s3://data.example.com/test.bin -> test.bin.new  [1 of 1]
 200000000 of 200000000   100% in   11s    16.20 MB/s  done

也许这篇文章也很有趣：Amazon EC2 和 Amazon S3 中的网络性能