Django将pdf复制到s3bucket

Question

我在将本地文件保存到s3bucket时遇到问题。

我在Django项目中有一个玉米作业，一段时间后，它会生成一个pdf文件。我想把文件保存在s3bucket中。

目前，Django s3bucket运行得很好，就像把我上传的文件保存到s3bucket一样，还有很多事情在运行。

但我不知道如何复制本地文件并保存在s3bucket中。

目前我正在保存在我的本地机器像这样：

shutil.copyfile('/var/www/local.pdf' ,'media/newfileins3bucket.pdf')

但它不会以这种方式工作，我想直接将其保存到s3bucket中。

在这种情况下有人能帮我吗？

我正在使用这个，直接将pdf保存到s3bucket没有意义：https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html

Answer 1

from copy import deepcopy
s3 = boto3.client('s3',
region_name=""#put region here,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key)

files = request.FILES.getlist('file')  # get all files
for file in files:
deep_file = deepcopy(file)
status, aws_file_path = upload_to_aws(deep_file)
if status == api_status.HTTP_200_OK:
reference_id = [aws_file_path]
logger.debug("AWS_STORAGE file path {}".format(reference_id))
message = "Uploaded Successfully"
else:
message = "COULD NOT CONNECT AWS"
status_api = status
return HttpResponse({}, status=status_api)

def upload_to_aws(file):
global s3
try:
is_bucket = check_is_bucket_present()
except botocore.exceptions.NoCredentialsError:
logger.debug("Unable to locate credentials for AWS")
return api_status.HTTP_500_INTERNAL_SERVER_ERROR, {}
if not is_bucket:
try:
bucket = s3.create_bucket(Bucket=settings.AWS_BUCKET)
except botocore.exceptions.ClientError as e:
logger.debug("AWS_ Error while Creating Bucket {} : ".format(str(e)))
return api_status.HTTP_500_INTERNAL_SERVER_ERROR, {}
file_name_uuid = uuid.uuid4().hex[:20]
folder_name = ''.join(file_name_uuid)
try:
# this does not return anything so try except
file_path = str(folder_name + "/resume/" + str(file.name))
# todo need to look into uploading via client
# GB = 1024 ** 3
# config = TransferConfig(multipart_threshold=5 * GB)
# s3.upload_file('result1.csv', bucket_name, 'folder_name/result1.csv', Config=config)
# was working with path but not with inmemoryobject
s3 = boto3.resource('s3',
region_name="us-------"#put region here,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key
)
s3.Bucket(settings.AWS_BUCKET).put_object(Key=file_path, Body=file)
file_path_bucket = settings.AWS_BUCKET + "/" + file_path
return api_status.HTTP_200_OK, file_path_bucket
except botocore.exceptions.ClientError as e:
logger.debug("AWS_STORAGE Error {}".format(str(e)))
return api_status.HTTP_500_INTERNAL_SERVER_ERROR, {}
except Exception as e:
logger.debug("AWS_STORAGE Error {}".format(str(e)))
return api_status.HTTP_500_INTERNAL_SERVER_ERROR, {}

Answer 2

有几种方法可以做到这一点，但我认为以下其中一种方法应该适用。

注意：我假设(正如您所提到的(您有Django Storages，其中S3后端在您的设置中设置为默认存储

在模型上使用FileField上载

如果您设置了一个模型来保存对生成的报告的引用，您可以执行以下操作：

from django.db import models 
from django.core.files import File
class Report(models.Model):
# this links to the S3 bucket if you use the correct Django Storages backend
report_file = models.FileField()
# in your cron script, when your report is generated at '/var/www/local.pdf'
local_file = open('/var/www/local.pdf', 'rb')
report = Report()
# this uploads the context of file to s3, also saves to database
report.report_file.save('media/newfileins3bucket.pdf', File(local_file))

请注意，您必须将本地文件包装在DjangoFile对象中。

在文件字段上调用save()也会自动将模型保存在数据库中，除非将save=False添加到调用中。有关更多信息，请参阅FileField.save((上的文档

无模型直接上传

如果你只想上传文件到S3而不保存在模型中，你可以这样做：

from django.core.files.storage import default_storage
local_file = open('/var/www/local.pdf', 'rb')
# default_storage will be the S3 storage if you set use Django Storage with S3 backend in your settings
with default_storage.open('media/newfileins3bucket.pdf', 'wb') as target:
target.write(local_file.read())

免责声明：我使用了类似的东西，但没有测试上面的确切代码。不过，它应该为你指明正确的方向