使用base64的大型文件出现MemoryError

Question

我面临着一个问题，我希望能深入了解这个问题。我在我的网络应用程序中对视频进行了加密，它将视频解密到BytesIO内存对象中，然后进行base64编码，允许用户通过HTML5视频播放器播放视频。

视频本身永远不应该在文件系统上真正解密（比如说）。然而，如果我尝试加载大于50MB的视频文件，它会失败，并在web2py中显示"MemoryError"消息。我如何解决这个问题？

（我认为它在base64.b64编码方法中失败了）

我正在使用Python27和Web2py 进行开发

import io
import base64
cry_upload = request.args(0)
cry = db(db.video_community.id == int(cry_upload)).select(db.video_community.ALL).first()
if cry:
    if auth.user and auth.user.id not in (cry.community.members):    
        return redirect(URL('default', 'cpanel'), client_side = True)
    filepath = os.path.join(request.folder, 'uploads/', '%s' % cry.file_upload)
    form = FORM(LABEL('The Associated Key'), INPUT(_name='key', value="", _type='text', _class='string'), INPUT(_type='submit', _class='btn btn-primary'), _class='form-horizontal')
    if form.accepts(request, session):
        outfile = io.BytesIO()
        with open(filepath, 'rb') as infile:
            key = str(form.vars.key).rstrip()
            try:
                decrypt(infile, outfile, key)
            except:
                session.flash = 'Decryption error. Please verify your key'
                return redirect(URL('default', 'cpanel'), client_side = True)
        outfile.seek(0)
        msg = base64.b64encode(outfile.getvalue().encode('utf-8'))
        outfile.close()
        response.flash = 'Success! Your video is now playable'
        return dict(form=form, cry=cry, msg=msg)

如有任何建议，不胜感激。

解密功能

提供如何使用Python/PyCrypto以OpenSSL兼容的方式对文件进行AES加密/解密？

from hashlib import md5
from Crypto.Cipher import AES
from Crypto import Random
from Crypto.Random import random
def derive_key_and_iv(password, salt, key_length, iv_length):
    d = d_i = ''
    while len(d) < key_length + iv_length:
        d_i = md5(d_i + password + salt).digest()
        d += d_i
    return d[:key_length], d[key_length:key_length+iv_length]

def decrypt(in_file, out_file, password, key_length=32):
    bs = AES.block_size
    salt = in_file.read(bs)[len('Salted__'):]
    key, iv = derive_key_and_iv(password, salt, key_length, bs)
    cipher = AES.new(key, AES.MODE_CBC, iv)
    next_chunk = ''
    finished = False
    while not finished:
        chunk, next_chunk = next_chunk, cipher.decrypt(in_file.read(1024 * bs))
        if len(next_chunk) == 0:
            padding_length = ord(chunk[-1])
            if padding_length < 1 or padding_length > bs:
               raise ValueError("bad decrypt pad (%d)" % padding_length)
            # all the pad-bytes must be the same
            if chunk[-padding_length:] != (padding_length * chr(padding_length)):
               # this is similar to the bad decrypt:evp_enc.c from openssl program
               raise ValueError("bad decrypt")
            chunk = chunk[:-padding_length]
            finished = True
        out_file.write(chunk)

这是视图中的视频播放器

{{if msg != None:}}
    <div class="row" style="margin-top:20px;">
        <div class="video-js-box">
            <video class="video-js vjs-default-skin" data-setup='{"controls": true, "autoplay": false, "preload": "none", "poster": "https://www.cryaboutcrypt.ninja/static/crying.png", "width": 720, "height": 406}'>
                    <source type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"' src="data:video/mp4;base64,{{=msg}}" />
            </video>
       </div>
    </div>
{{pass}}

如前所述，我将base64编码的BytesIO数据传递给播放器@安东尼-我不能百分之百肯定你的建议。

这是发生的MemoryError

File "/usr/lib/python2.7/base64.py", line 53, in b64encode
    encoded = binascii.b2a_base64(s)[:-1]
MemoryError

Answer 1

与其在页面的HTML中嵌入base64编码的视频文件，不如提供一个URL作为视频播放器源，并从该URL流式传输文件：

        <source type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'
         src="{{=URL('default', 'video', args=request.args(0))}}" />

然后，您的大部分代码将转移到default.py控制器中的video函数。不使用基64对视频进行编码，只需通过response.stream()返回outfile即可。