我想通过boost的zlib压缩将多个缓冲区(在我的例子中是来自不同来源的视频帧)压缩到一个新的缓冲区中，然后将所有内容写入磁盘上的一个文件中。我需要这两个步骤，因为我想在包含压缩缓冲区的最终大小的文件中添加一个头(稍后将作为解析器的偏移量)。我想用boost的iostreams库实现这一点。
出现了以下相关问题:
a)我是否需要使用filtering_streambuf中的filtering_stream?我希望后者已经具有某种缓冲行为。
b)我如何关闭filtering_stream(buf)并将其写入缓冲区?
c)如何读取压缩数据的最终大小?.tellg()不实现这些filtering_streams(如在SO的其他地方提到的)
d)你可以有多个源，即我的三个缓冲区或我需要组合它们?(我的方法见下文)。
class Frame {
private:
/* other things */
public:
float buf1[3];
float buf2[3];
float buf3[4];
/* more things */
};
int main() {
Frame frame;
using boost::iostreams bio;
bio::filtering_streambuf<bio::input> in;
in.push(bio::gzip_compressor());
/* Could you also add the buffers indiviually? */
in.push(bio::array_source(reinterpret_cast<const char*>(frame.buf1), 3 + 7 + 12 + (sizeof(float) * 3)));
const char *compressed = /* How to close in and write the contents to this buffer? */
int compressedSize = /* How to get this from in? in.tellg() does not work */
std::stringstream headerInformation;
headerInformation << "START";
headerInformation << "END " << compressedSize;
std::ofstream ofs("ouput.data", std::ofstream::out | std::ofstream::binary | std::ofstream::app);
bio::filtering_ostream out;
out.push(ofs);
out.write(headerInformation.str(), headerInformation.str().length());
out.write(compressed, compressedSize);
boost::iostreams::close(out);
boost::iostreams::close(in);
return 0;
}
a)我需要使用filtering_streambuf的filtering_stream吗?我将期望后者已经具有某种缓冲区行为。
都可以。流添加了文本和区域设置特性，就像在标准库中一样。
b)我如何关闭filtering_stream(但是)并将其写入缓冲区?
您可以使用array_sink,back_inserter_device，内存映射等。参见https://www.boost.org/doc/libs/1_72_0/libs/iostreams/doc/("Models")。
c)如何读取压缩数据的最终大小?.tellg ()没有为这些filtering_streams实现(如前所述)
从您的底层输出设备/流检测它。在此之前别忘了冲洗/关闭过滤层。
d)你可以有多个源，即我的三个缓冲区或我需要把它们结合起来?(我的方法见下文)。
你可以做你想做的。
Show Me The Code…
我会反过来，让过滤器压缩，并将写入输出缓冲区:
using RawBuffer = std::vector<char>;
using Device = bio::back_insert_device<RawBuffer>;
RawBuffer compressed_buffer; // optionally reserve some size
{
bio::filtering_ostream filter;
filter.push(bio::gzip_compressor());
filter.push(Device{ compressed_buffer });
filter.write(reinterpret_cast<char const*>(&frame.buf1),
sizeof(frame) - offsetof(Frame, buf1));
}
使用过滤流:
{
bio::filtering_ostreambuf filter;
filter.push(bio::gzip_compressor());
filter.push(Device{ compressed_buffer });
std::copy_n(reinterpret_cast<char const*>(&frame.buf1),
sizeof(frame) - offsetof(Frame, buf1),
std::ostreambuf_iterator<char>(&filter));
}
现在你的问题的答案很明显了:
const char *compressed = compressed_buffer.data();
int compressedSize = compressed_buffer.size();
{
std::ofstream ofs("ouput.data", std::ios::binary | std::ios::app);
ofs << "START";
ofs << "END " << compressed_buffer.size();
ofs.write(compressed_buffer.data(), compressed_buffer.size());
}
考虑不要为每一帧重新打开输出流:)
现场演示
Live On Coliru
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/device/back_inserter.hpp>
#include <iterator>
#include <fstream>
#include <vector>
namespace bio = boost::iostreams;
class Frame {
private:
/* other things */
public:
float buf1[3];
float buf2[3];
float buf3[4];
/* more things */
};
int main() {
Frame const frames[]{
{
{ 1, 2, 3 },
{ 4, 5, 6 },
{ 7, 8, 9, 10 },
},
{
{ 11, 12, 13 },
{ 14, 15, 16 },
{ 17, 18, 19, 20 },
},
{
{ 21, 22, 23 },
{ 24, 25, 26 },
{ 27, 28, 29, 30 },
},
};
// avoiding UB:
static_assert(std::is_trivial_v<Frame> &&
std::is_standard_layout_v<Frame>);
using RawBuffer = std::vector<char>;
using Device = bio::back_insert_device<RawBuffer>;
std::remove("output.data");
std::ofstream ofs("output.data", std::ios::binary | std::ios::app);
RawBuffer compressed_buffer; // optionally reserve some size
for (Frame const& frame : frames) {
compressed_buffer.clear(); // do not shrink_to_fit optimizing allocation
{
bio::filtering_ostreambuf filter;
filter.push(bio::gzip_compressor());
filter.push(Device{ compressed_buffer });
std::copy_n(reinterpret_cast<char const*>(&frame.buf1),
sizeof(frame) - offsetof(Frame, buf1),
std::ostreambuf_iterator<char>(&filter));
}
ofs << "START";
ofs << "END " << compressed_buffer.size();
ofs.write(compressed_buffer.data(), compressed_buffer.size());
}
}
确定性地生成输出。data:
00000000: 5354 4152 5445 4e44 2035 301f 8b08 0000 STARTEND 50.....
00000010: 0000 0000 ff63 6068 b067 6060 7000 2220 .....c`h.g``p."
00000020: 6e00 e205 407c 0088 1f00 3183 2303 8300 n...@|....1.#...
00000030: 102b 3802 0058 a049 af28 0000 0053 5441 .+8..X.I.(...STA
00000040: 5254 454e 4420 3438 1f8b 0800 0000 0000 RTEND 48........
00000050: 00ff 6360 3070 6460 7000 e200 204e 00e2 ..c`0pd`p... N..
00000060: 0220 6e00 e20e 209e 00c4 3380 7881 2300 . n... ...3.x.#.
00000070: 763b 7371 2800 0000 5354 4152 5445 4e44 v;sq(...STARTEND
00000080: 2034 391f 8b08 0000 0000 0000 ff63 6058 49..........c`X
00000090: e1c8 c0b0 0188 7700 f101 203e 01c4 1780 ......w... >....
000000a0: f806 103f 00e2 1740 fcc1 1100 dfb4 6cde ...?...@......l.
000000b0: 2800 0000 (...