我正在尝试以视频格式对10位图像进行无损编码,最好使用HEVC编码。图像存储为16位png文件(但只使用10位),我一直在使用ffmpeg创建和读取视频文件。
到目前为止,我最好的尝试是基于https://stackoverflow.com/a/66180140/17261462,但正如上面提到的,我得到了一些像素强度差异,这可能是由于在10和16位表示之间转换时的舍入。我尝试了几种不同的方法(位移位,左位复制,基于浮点的缩放),但还没有弄清楚如何获得真正的无损重建。
下面是一小段代码来复制我的问题。我可能做错了什么,所以反馈将是感激的。
import subprocess
import numpy as np
import matplotlib.pyplot as plt
import tempfile
import imageio
# Create simple image
bitdepth = 10
hbd = int(bitdepth/2)
im0 = np.zeros((1<<hbd,1<<hbd),dtype=np.uint16)
im0[:] = np.arange(0,1<<bitdepth).reshape(im0.shape)
print('im0',np.min(im0),np.max(im0),im0.shape,im0.dtype)
# tile it to be at least 64 pix
im0 = np.tile(im0, (2, 2))
print('im0',np.min(im0),np.max(im0),im0.shape,im0.dtype)
im0ref = im0
# bitshift it or rescale intensities
#im0 = (im0<<6)
#im0 = (im0<<6) + (im0>>4)
im0 = np.uint16(np.round(im0 * np.float64((1<<16)-1)/np.float64((1<<10)-1)))
print('im0',np.min(im0),np.max(im0),im0.shape,im0.dtype)
# Save it as png
tmp0 = tempfile.NamedTemporaryFile(suffix='.png', delete=False)
print(f'Using tmp file: {tmp0.name}')
imageio.imwrite(tmp0.name,im0)
# Encode with ffmpeg
tmp1 = tempfile.NamedTemporaryFile(suffix='.mkv', delete=False)
# note that adding the following doesn't seem to impact the results
# + ' -bsf:v hevc_metadata=video_full_range_flag=1'
mycmd = f'ffmpeg -y -i {tmp0.name}'
+ ' -c:v libx265 -x265-params lossless=1'
+ ' -pix_fmt gray10be'
+ f' {tmp1.name}'
print(mycmd)
p = subprocess.run(mycmd.split(), capture_output=True)
print( 'stdout:', p.stdout.decode() )
print( 'stderr:', p.stderr.decode() )
tmp2 = tempfile.NamedTemporaryFile(suffix='.png', delete=False)
mycmd = f'ffmpeg -y -i {tmp1.name}'
+ ' -pix_fmt gray16be'
+ f' {tmp2.name}'
print(mycmd)
p = subprocess.run(mycmd.split(), capture_output=True)
print( 'stdout:', p.stdout.decode() )
print( 'stderr:', p.stderr.decode() )
# Read back with ffmpeg
im1 = imageio.imread(tmp2.name)
print('im1',np.min(im1),np.max(im1),im1.shape,im1.dtype)
# Bitshift or scale back
im1pre = im1
#im1 = (im1>>6)
im1 = np.uint16(np.round(im1 * np.float64((1<<10)-1)/np.float64((1<<16)-1)))
# check the result
plt.figure()
plt.imshow(im0ref)
plt.colorbar()
plt.figure()
plt.imshow(im1)
plt.colorbar()
plt.figure()
plt.imshow(np.int32(im1)-np.int32(im0ref))
plt.colorbar()
print('err: ',np.linalg.norm((np.float32(im1)-np.float32(im0ref)).ravel()))
plt.show()
编辑:我现在也在ffmpeg用户列表上发布了我的问题:http://ffmpeg.org/pipermail/ffmpeg-user/2021-November/053761.html
同样为了方便,下面提供了一个简单的脚本来生成使用16位和10位数据的不同变体:
import numpy as np
import imageio
# Create simple image with gradient from
# 0 to (2^bitdepth - 1)
bitdepth = 10
unusedbitdepth = 16-bitdepth
hbd = int(bitdepth/2)
im0 = np.zeros((1<<hbd,1<<hbd),dtype=np.uint16)
im0[:] = np.arange(0,1<<bitdepth).reshape(im0.shape)
# Tile it to be at least 64 pix as ffmpeg encoder may only work
# with image of size 64 and up
im0 = np.tile(im0, (2, 2))
print('im0',np.min(im0),np.max(im0),im0.shape,im0.dtype)
# Save it
imageio.imwrite('gradient10bit-lsb.png',im0)
# Bitshift the values to use most significant bits
im1 = (im0<<unusedbitdepth)
print('im1',np.min(im1),np.max(im1),im1.shape,im1.dtype)
imageio.imwrite('gradient10bit-msb.png',im1)
# Scale the values use all 16 bits
im2 = np.uint16(np.round(im0 * np.float64((1<<16)-1)/np.float64((1<<bitdepth)-1)))
print('im2',np.min(im2),np.max(im2),im2.shape,im2.dtype)
imageio.imwrite('gradient10bit-scaledto16bits.png',im2)
# Left bit replication as a cost-effective approximation of scaling
# See http://www.libpng.org/pub/png/spec/1.1/PNG-Encoders.html
im3 = (im0<<unusedbitdepth) + (im0>>(bitdepth-unusedbitdepth))
print('im3',np.min(im3),np.max(im3),im3.shape,im3.dtype)
imageio.imwrite('gradient10bit-leftbitreplication.png',im3)
以及原始的ffmpeg/image magic命令。
编码:
ffmpeg -y -i gradient10bit-scaledto16bits.png -c:v libx265 -x265-params lossless=1 -pix_fmt gray10be gradient10bit-scaledto16bits.mkv
解码回png:
ffmpeg -y -i gradient10bit-scaledto16bits.mkv -pix_fmt gray16be recons-gradient10bit-scaledto16bits.png
比较:
magick compare -verbose -metric mae gradient10bit-scaledto16bits.png recons-gradient10bit-scaledto16bits.png diff-scaledto16bits.png
许多谢谢,
汤姆
如果,如您所说,您想要将10位图像无损编码为视频,那么您肯定会更好地使用能够存储此类内容的无损格式-例如ffv1
-然后您可以存储完整的16位而无需移动/缩放或做任何事情。
#!/bin/bash
# Generate 16-bit greyscale PNG
magick -size 1920x1080 xc:gray +noise random 1.png
magick 1.png -format "File: %f Unique colours: %k, Min: %[min], Max: %[max]n" info:
# Encode to video
ffmpeg -v warning -y -i 1.png -c:v ffv1 -pix_fmt gray16le video.mkv
# Decode back to PNG
ffmpeg -v warning -y -i video.mkv 2.png
magick 2.png -format "File: %f Unique colours: %k, Min: %[min], Max: %[max]n" info:
# Compare
magick compare -verbose -metric ae {1,2}.png null:
File: 1.png Unique colours: 65536, Min: 0, Max: 65535
File: 2.png Unique colours: 65536, Min: 0, Max: 65535
1.png PNG 1920x1080 1920x1080+0+0 16-bit Gray 3.96256MiB 0.030u 0:00.029
2.png PNG 1920x1080 1920x1080+0+0 16-bit Gray 4161790B 0.020u 0:00.017
Image: 1.png
Channel distortion: AE
gray: 0
all: 0
1.png=> PNG 1920x1080 16-bit Gray 3.96256MiB 0.760u 0:00.064
感谢Paul B Mahol在ffmpeg-user邮件列表上,我已经能够在使用临时rawvideo文件时解决这个问题。尽管如此,一个没有临时的解决方案将是更好的。
# convert png to rawvideo in 16 bits
ffmpeg -y -i gradient10bit-lsb.png -f rawvideo -pix_fmt gray16le gradient10bit-lsb.raw
# convert rawvideo to hevc-mkv in 10 bits by tricking the rawvideo demuxer
# into thinking the input is a 10 bit video
ffmpeg -y -f rawvideo -pixel_format gray10le -video_size 64x64 -i gradient10bit-lsb.raw -c:v libx265 -x265-params lossless=1 -pix_fmt gray10le gradient10bit-lsb.mkv
# delete tmp file
rm -f gradient10bit-lsb.raw
# convert hevc-mkv to rawvideo 10 bit
ffmpeg -y -i gradient10bit-lsb.mkv -f rawvideo -pix_fmt gray10le gradient10bit-lsb-postmkv.raw
# convert rawvideo back to png 16bits by tricking the rawvideo demuxer
# into thinking the input is 16 bits
ffmpeg -y -f rawvideo -pixel_format gray16le -video_size 64x64 -i gradient10bit-lsb-postmkv.raw -pix_fmt gray16be recons-gradient10bit-lsb.png
# delete tmp file
rm -f gradient10bit-lsb-postmkv.raw
# compare
magick compare -verbose -metric mae gradient10bit-lsb.png recons-gradient10bit-lsb.png diff-lsb.png