在我之前的问题(在c#中处理文件后RAM没有被释放)中，我询问了一种清除RAM的方法。有人建议使用流而不是读入变量。我发现加密/解密使用流的大文件(. net)，但它不使用AesGcm。问题是我找不到如何使用AesGcm与流。AesGcm.decrypt只接受密文字段中的Byte[]，和AesManaged没有CihperMode.GCM.
目前，解密800MB的文件需要4GB的内存。我如何解密一个文件与AesGcm而不填充RAM?
谢谢。
我会说。net中的AesGcm(可能还有AesCcm)不支持"流";模式，似乎共识(https://crypto.stackexchange.com/questions/51537/delayed-tag-checks-in-aes-gcm-for-streaming-data)是，你不应该创建一个流模式AesGcm。我将为此添加另一个参考https://github.com/dotnet/runtime/issues/27348。我不是密码学专家，所以我不清楚流加密文档和仅在最后检查其身份验证标签的问题是什么。
如果可能的话，你应该改变算法。否则，可以找到其他解决方案。Bouncycastle库支持AesGcm。
我发布了一个非流回答，因为我有一个相当不错的低分配的AesGcm实现，可以满足您的需求。您可以将ArraySegment<byte>直接放入流中，并使用FileStream写入磁盘。内存分配不应超过文件本身的两倍(显然是x2，因为您将文件加载到内存中，并且必须存储加密的字节)。它的性能也很好，但这不是我的功劳，显然只是Net5.0的增强。
如果需要使用另一种解密机制，字节结构是直接的。
12 bytes 16 bytes n bytes up to int.IntMax - 28 bytes.
[ Nonce / IV ][ Tag / MAC ][ Ciphertext ]
链接到我的Github Repo.
用法的例子)
// HashKey or PassKey or Passphrase in 16/24/32 byte format.
var encryptionProvider = new AesGcmEncryptionProvider(hashKey, "ARGON2ID");
// This is an ArraySegment<byte>, this allows a defer allocation of byte[]
var encryptedData = encryptionProvider.Encrypt(_data);
// When you are ready for a byte[]
encryptedData.ToArray()
// You can also use
encryptedData.Array
// but this is a buffer and often exceeds the actual size of your bytes.
// Use conscientiously but does prevent a copy / allocation of the bytes.
// To Decrypt - same return type in case you need to serialize / decompress etc.
var decryptedData = encryptionProvider.Decrypt(encryptedData);
decrypted.ToArray() // the proper decrypted bytes of data.
decrypted.Array // the buffer used.
// Convert to a string
Encoding.UTF8.GetString(_encryptedData.ToArray())
如果你看到任何问题让我知道，很高兴做出改变/修复-或者更好的是在Github上提交一个问题/PR，这样我就可以保持真正的代码是最新的。
基准// * Summary *
BenchmarkDotNet=v0.13.0, OS=Windows 10.0.18363.1556 (1909/November2019Update/19H2)
Intel Core i7-9850H CPU 2.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK=5.0.203
[Host] : .NET 5.0.6 (5.0.621.22011), X64 RyuJIT
Job-ADZLQM : .NET 5.0.6 (5.0.621.22011), X64 RyuJIT
.NET 5.0 : .NET 5.0.6 (5.0.621.22011), X64 RyuJIT
Runtime=.NET 5.0
| Method | Job | IterationCount | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------------ |----------- |--------------- |--------------:|--------------:|--------------:|--------------:|------:|--------:|----------:|---------:|---------:|----------:|
| Encrypt1KBytes | .NET 5.0 | Default | 1.512 us | 0.0298 us | 0.0398 us | 1.504 us | 1.00 | 0.00 | 0.1926 | - | - | 1 KB |
| Encrypt2KBytes | .NET 5.0 | Default | 1.965 us | 0.0382 us | 0.0408 us | 1.951 us | 1.30 | 0.04 | 0.3548 | - | - | 2 KB |
| Encrypt4kBytes | .NET 5.0 | Default | 2.946 us | 0.0583 us | 0.0942 us | 2.948 us | 1.96 | 0.07 | 0.6828 | - | - | 4 KB |
| Encrypt8KBytes | .NET 5.0 | Default | 4.630 us | 0.0826 us | 0.0733 us | 4.631 us | 3.09 | 0.08 | 1.3351 | - | - | 8 KB |
| Decrypt1KBytes | .NET 5.0 | Default | 1.234 us | 0.0247 us | 0.0338 us | 1.216 us | 0.82 | 0.03 | 0.1869 | - | - | 1 KB |
| Decrypt2KBytes | .NET 5.0 | Default | 1.644 us | 0.0328 us | 0.0378 us | 1.630 us | 1.09 | 0.04 | 0.3510 | - | - | 2 KB |
| Decrypt4kBytes | .NET 5.0 | Default | 2.462 us | 0.0274 us | 0.0214 us | 2.460 us | 1.64 | 0.04 | 0.6752 | - | - | 4 KB |
| Decrypt8KBytes | .NET 5.0 | Default | 4.167 us | 0.0828 us | 0.1016 us | 4.179 us | 2.76 | 0.12 | 1.3275 | - | - | 8 KB |
及时的代码快照。
public class AesGcmEncryptionProvider : IEncryptionProvider
{
/// <summary>
/// Safer way of generating random bytes.
/// https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.rngcryptoserviceprovider?redirectedfrom=MSDN&view=net-5.0
/// </summary>
private readonly RNGCryptoServiceProvider _rng = new RNGCryptoServiceProvider();
private readonly ArrayPool<byte> _pool = ArrayPool<byte>.Create();
private readonly byte[] _key;
public string Type { get; private set; }
public AesGcmEncryptionProvider(byte[] key, string hashType)
{
if (!Constants.Aes.ValidKeySizes.Contains(key.Length)) throw new ArgumentException("Keysize is an invalid length.");
_key = key;
switch (_key.Length)
{
case 16: Type = "AES128"; break;
case 24: Type = "AES192"; break;
case 32: Type = "AES256"; break;
}
if (!string.IsNullOrWhiteSpace(hashType)) { Type = $"{hashType}-{Type}"; }
}
public ArraySegment<byte> Encrypt(ReadOnlyMemory<byte> data)
{
using var aes = new AesGcm(_key);
// Slicing Version
// Rented arrays sizes are minimums, not guarantees.
// Need to perform extra work managing slices to keep the byte sizes correct but the memory allocations are lower by 200%
var encryptedBytes = _pool.Rent(data.Length);
var tag = _pool.Rent(AesGcm.TagByteSizes.MaxSize); // MaxSize = 16
var nonce = _pool.Rent(AesGcm.NonceByteSizes.MaxSize); // MaxSize = 12
_rng.GetBytes(nonce, 0, AesGcm.NonceByteSizes.MaxSize);
aes.Encrypt(
nonce.AsSpan().Slice(0, AesGcm.NonceByteSizes.MaxSize),
data.Span,
encryptedBytes.AsSpan().Slice(0, data.Length),
tag.AsSpan().Slice(0, AesGcm.TagByteSizes.MaxSize));
// Prefix ciphertext with nonce and tag, since they are fixed length and it will simplify decryption.
// Our pattern: Nonce Tag Cipher
// Other patterns people use: Nonce Cipher Tag // couldn't find a solid source.
var encryptedData = new byte[AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize + data.Length];
Buffer.BlockCopy(nonce, 0, encryptedData, 0, AesGcm.NonceByteSizes.MaxSize);
Buffer.BlockCopy(tag, 0, encryptedData, AesGcm.NonceByteSizes.MaxSize, AesGcm.TagByteSizes.MaxSize);
Buffer.BlockCopy(encryptedBytes, 0, encryptedData, AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize, data.Length);
_pool.Return(encryptedBytes);
_pool.Return(tag);
_pool.Return(nonce);
return encryptedData;
}
public async Task<MemoryStream> EncryptAsync(Stream data)
{
using var aes = new AesGcm(_key);
var buffer = _pool.Rent((int)data.Length);
var bytesRead = await data
.ReadAsync(buffer.AsMemory(0, (int)data.Length))
.ConfigureAwait(false);
if (bytesRead == 0) throw new InvalidDataException();
// Slicing Version
// Rented arrays sizes are minimums, not guarantees.
// Need to perform extra work managing slices to keep the byte sizes correct but the memory allocations are lower by 200%
var encryptedBytes = _pool.Rent((int)data.Length);
var tag = _pool.Rent(AesGcm.TagByteSizes.MaxSize); // MaxSize = 16
var nonce = _pool.Rent(AesGcm.NonceByteSizes.MaxSize); // MaxSize = 12
_rng.GetBytes(nonce, 0, AesGcm.NonceByteSizes.MaxSize);
aes.Encrypt(
nonce.AsSpan().Slice(0, AesGcm.NonceByteSizes.MaxSize),
buffer.AsSpan().Slice(0, (int)data.Length),
encryptedBytes.AsSpan().Slice(0, (int)data.Length),
tag.AsSpan().Slice(0, AesGcm.TagByteSizes.MaxSize));
// Prefix ciphertext with nonce and tag, since they are fixed length and it will simplify decryption.
// Our pattern: Nonce Tag Cipher
// Other patterns people use: Nonce Cipher Tag // couldn't find a solid source.
var encryptedStream = new MemoryStream(new byte[AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize + (int)data.Length]);
using (var binaryWriter = new BinaryWriter(encryptedStream, Encoding.UTF8, true))
{
binaryWriter.Write(nonce, 0, AesGcm.NonceByteSizes.MaxSize);
binaryWriter.Write(tag, 0, AesGcm.TagByteSizes.MaxSize);
binaryWriter.Write(encryptedBytes, 0, (int)data.Length);
}
_pool.Return(buffer);
_pool.Return(encryptedBytes);
_pool.Return(tag);
_pool.Return(nonce);
encryptedStream.Seek(0, SeekOrigin.Begin);
return encryptedStream;
}
public MemoryStream EncryptToStream(ReadOnlyMemory<byte> data)
{
return new MemoryStream(Encrypt(data).ToArray());
}
public ArraySegment<byte> Decrypt(ReadOnlyMemory<byte> encryptedData)
{
using var aes = new AesGcm(_key);
// Slicing Version
var nonce = encryptedData
.Slice(0, AesGcm.NonceByteSizes.MaxSize)
.Span;
var tag = encryptedData
.Slice(AesGcm.NonceByteSizes.MaxSize, AesGcm.TagByteSizes.MaxSize)
.Span;
var encryptedBytes = encryptedData
.Slice(AesGcm.NonceByteSizes.MaxSize + AesGcm.TagByteSizes.MaxSize)
.Span;
var decryptedBytes = new byte[encryptedBytes.Length];
aes.Decrypt(nonce, encryptedBytes, tag, decryptedBytes);
return decryptedBytes;
}
public MemoryStream Decrypt(Stream stream)
{
using var aes = new AesGcm(_key);
using var binaryReader = new BinaryReader(stream);
var nonce = binaryReader.ReadBytes(AesGcm.NonceByteSizes.MaxSize);
var tag = binaryReader.ReadBytes(AesGcm.TagByteSizes.MaxSize);
var encryptedBytes = binaryReader.ReadBytes((int)binaryReader.BaseStream.Length - AesGcm.NonceByteSizes.MaxSize - AesGcm.TagByteSizes.MaxSize);
var decryptedBytes = new byte[encryptedBytes.Length];
aes.Decrypt(nonce, encryptedBytes, tag, decryptedBytes);
return new MemoryStream(decryptedBytes);
}
public MemoryStream DecryptToStream(ReadOnlyMemory<byte> data)
{
return new MemoryStream(Decrypt(data).ToArray());
}
}
是这样的:
using var aes = new AesGcm(_key);
using FileStream fs = new(<path to file>, FileMode.Open);
int bytesRead;
while ((bytesRead = fs.Read(buffer)) > 0)
{
aes.Encrypt(nonce, buffer[..bytesRead], buffer[..bytesRead], tag);
using var encfs = new FileStream($@"{path to output file}.enc", FileMode.Append);
encfs.Write(_salt);
encfs.Write(nonce);
encfs.Write(buffer[..bytesRead]);
encfs.Write(tag);
}
打开指向文件的流，根据缓冲区大小通过缓冲区将文件流式传输，并将结果密码写入同一文件。这个阶段的内存应该是缓冲区的大小加上一些很小的东西，这些东西来自于程序中当时还活着的各种对象。
我有问题加载，例如一个3gb的文件，加密它，并有内存很好，清晰，上面的代码提供了没有任何问题。