如何通过live555从通过rtsp协议发送的h264比特流中计算或读取帧率



问题:

在我的例子中,比特流来自h264编码器,它是由现场摄像机提供的。比特流通过live555库使用RTSP协议发送。RTSP数据包由RTSP客户端接收,如vlc播放器(最重要的)、图腾播放器、mplayer。我需要知道如何设置较低的帧速率,因为我的相机产生9帧/秒的帧速,但大多数RTSP客户端都期望默认为25帧/秒,这会导致视频流无法正常播放(缓冲、丢弃延迟数据包等)。

问题:

我想知道RTSP客户端(播放软件)是如何从比特流中读取(或评估)帧率的。这些信息存储在哪里,如何发送到播放软件?

更多信息:

我使用的是live555中的testOnDemandRTSPServer示例代码。我已经从live555lib中编写了DeviceSource类的实现。我已经正确设置了演示时间:

gettimeofday(&fPresentationTime, NULL);
fDurationInMicroseconds = 1100000;

序列参数集NAL还包含适当的定时信息:

timing_info_present_flag : 1 
       num_units_in_tick : 1001 
       time_scale : 9000 
       fixed_frame_rate_flag : 1

RTSP标头包含帧速率:

a=framerate:9.0

不幸的是,这无济于事。我认为视频播放器将平静地等待新的帧,并且帧速率将和接收另一个RTP分组(另一个NAL)的频率相关。

视频播放器的行为:

VLC播放器-视频是不稳定的,因为VLC预计下一帧会更早(25帧而不是9帧)。结果,vlc将缓冲时间增加到例如10s,在此期间视频被冻结。

图腾玩家-SPS Nal的计时信息由图腾解析。帧速率显示正常,但视频仍不稳定(延迟约500ms)。它似乎在等待更多的帧(每秒25帧)。

Mplayer识别出错误的帧速率值,显示每秒25帧,但视频非常流畅,质量良好:

mplayer ffmpeg://rtsp://192.168.1.82:8554/testStream
.
.
.
VIDEO:  [H264]  320x240  0bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)

Live555 RTSP客户端测试应用程序./live555/testProgs/testRTSPClientrtsp://192.168.1.82:8554/testStream

Opening connection to 192.168.1.82, port 8554...
...remote connection opened
Sending request: DESCRIBE rtsp://192.168.1.82:8554/testStream RTSP/1.0
CSeq: 2
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Accept: application/sdp

Received 675 new bytes of response data.
Received a complete DESCRIBE response:
RTSP/1.0 200 OK
CSeq: 2
Date: Fri, Jan 01 1988 00:00:23 GMT
Content-Base: rtsp://192.168.1.82:8554/testStream/
Content-Type: application/sdp
Content-Length: 506
v=0
o=- 567993603275971 1 IN IP4 192.168.1.82
s=Session streamed by "testOnDemandRTSPServer"
i=testStream
t=0 0
a=tool:LIVE555 Streaming Media v2016.02.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "testOnDemandRTSPServer"
a=x-qt-text-inf:testStream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42801E;sprop-parameter-sets=Z0KAHpWgUHxA,aM44gA==
a=framerate:9.0
a=control:track1
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Got a SDP description:
v=0
o=- 567993603275971 1 IN IP4 192.168.1.82
s=Session streamed by "testOnDemandRTSPServer"
i=testStream
t=0 0
a=tool:LIVE555 Streaming Media v2016.02.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "testOnDemandRTSPServer"
a=x-qt-text-inf:testStream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42801E;sprop-parameter-sets=Z0KAHpWgUHxA,aM44gA==
a=framerate:9.0
a=control:track1
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Initiated the "video/H264" subsession (client ports 57660-57661)
Sending request: SETUP rtsp://192.168.1.82:8554/testStream/track1 RTSP/1.0
CSeq: 3
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Transport: RTP/AVP;unicast;client_port=57660-57661

Received 215 new bytes of response data.
Received a complete SETUP response:
RTSP/1.0 200 OK
CSeq: 3
Date: Fri, Jan 01 1988 00:00:23 GMT
Transport: RTP/AVP;unicast;destination=192.168.1.109;source=192.168.1.82;client_port=57660-57661;server_port=6970-6971
Session: 06B1DFA4;timeout=65

[URL:"rtsp://192.168.1.82:8554/testStream/"]: Set up the "video/H264" subsession (client ports 57660-57661)
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Created a data sink for the "video/H264" subsession
Sending request: PLAY rtsp://192.168.1.82:8554/testStream/ RTSP/1.0
CSeq: 4
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Session: 06B1DFA4
Range: npt=0.000-

Received 96 new bytes of response data.
Received 94 new bytes of response data.
Received a complete PLAY response:
RTSP/1.0 200 OK
CSeq: 4
Date: Fri, Jan 01 1988 00:00:23 GMT
Range: npt=0.000-
Session: 06B1DFA4
RTP-Info: url=rtsp://192.168.1.82:8554/testStream/track1;seq=52833;rtptime=3161001994

[URL:"rtsp://192.168.1.82:8554/testStream/"]: Started playing session...
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 10874 bytes.   Presentation time: 567993623.442288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12054 bytes.   Presentation time: 567993623.482288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12227 bytes.   Presentation time: 567993623.522288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12281 bytes.   Presentation time: 567993623.562288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12384 bytes.   Presentation time: 567993623.602288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12129 bytes.   Presentation time: 567993623.642288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12141 bytes.   Presentation time: 567993623.682288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12287 bytes.   Presentation time: 567993623.722288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 16415 bytes.   Presentation time: 567993623.762288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12360 bytes.   Presentation time: 567993623.802288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264:      Received 12167 bytes.   Presentation time: 567993623.842288

我通过工具h264_analyze解析了我的NAL,下面是输出(只有几个第一个NAL):

!! Found NAL at offset 4 (0x0004), size 18 (0x0012) 
XX 00 00 00 01 67 42 80 1E 95 A0 50 7C 84 00 00 0F 
==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 3 
 nal_unit_type : 7 ( Sequence parameter set ) 
======= SPS =======
 profile_idc : 66 
 constraint_set0_flag : 1 
 constraint_set1_flag : 0 
 constraint_set2_flag : 0 
 constraint_set3_flag : 0 
 constraint_set4_flag : 0 
 constraint_set5_flag : 0 
 reserved_zero_2bits : 0 
 level_idc : 30 
 seq_parameter_set_id : 0 
 chroma_format_idc : 1 
 residual_colour_transform_flag : 0 
 bit_depth_luma_minus8 : 0 
 bit_depth_chroma_minus8 : 0 
 qpprime_y_zero_transform_bypass_flag : 0 
 seq_scaling_matrix_present_flag : 0 
 log2_max_frame_num_minus4 : 4 
 pic_order_cnt_type : 2 
   log2_max_pic_order_cnt_lsb_minus4 : 0 
   delta_pic_order_always_zero_flag : 0 
   offset_for_non_ref_pic : 0 
   offset_for_top_to_bottom_field : 0 
   num_ref_frames_in_pic_order_cnt_cycle : 0 
 num_ref_frames : 1 
 gaps_in_frame_num_value_allowed_flag : 0 
 pic_width_in_mbs_minus1 : 19 
 pic_height_in_map_units_minus1 : 14 
 frame_mbs_only_flag : 1 
 mb_adaptive_frame_field_flag : 0 
 direct_8x8_inference_flag : 0 
 frame_cropping_flag : 0 
   frame_crop_left_offset : 0 
   frame_crop_right_offset : 0 
   frame_crop_top_offset : 0 
   frame_crop_bottom_offset : 0 
 vui_parameters_present_flag : 1 
=== VUI ===
 aspect_ratio_info_present_flag : 0 
   aspect_ratio_idc : 0 
     sar_width : 0 
     sar_height : 0 
 overscan_info_present_flag : 0 
   overscan_appropriate_flag : 0 
 video_signal_type_present_flag : 0 
   video_format : 0 
   video_full_range_flag : 0 
   colour_description_present_flag : 0 
     colour_primaries : 0 
   transfer_characteristics : 0 
   matrix_coefficients : 0 
 chroma_loc_info_present_flag : 0 
   chroma_sample_loc_type_top_field : 0 
   chroma_sample_loc_type_bottom_field : 0 
 timing_info_present_flag : 1 
   num_units_in_tick : 1001 
   time_scale : 9000 
   fixed_frame_rate_flag : 1
 nal_hrd_parameters_present_flag : 0 
 vcl_hrd_parameters_present_flag : 0 
   low_delay_hrd_flag : 0 
 pic_struct_present_flag : 0 
 bitstream_restriction_flag : 0 
   motion_vectors_over_pic_boundaries_flag : 0 
   max_bytes_per_pic_denom : 0 
   max_bits_per_mb_denom : 0 
   log2_max_mv_length_horizontal : 0 
   log2_max_mv_length_vertical : 0 
   num_reorder_frames : 0 
   max_dec_frame_buffering : 0 
=== HRD ===
 cpb_cnt_minus1 : 0 
 bit_rate_scale : 0 
 cpb_size_scale : 0 
   bit_rate_value_minus1[0] : 0 
   cpb_size_value_minus1[0] : 0 
   cbr_flag[0] : 0 
 initial_cpb_removal_delay_length_minus1 : 0 
 cpb_removal_delay_length_minus1 : 0 
 dpb_output_delay_length_minus1 : 0 
 time_offset_length : 0 
!! Found NAL at offset 26 (0x001A), size 4 (0x0004) 
XX 00 00 00 01 68 CE 38 80 
==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 3 
 nal_unit_type : 8 ( Picture parameter set ) 
======= PPS =======
 pic_parameter_set_id : 0 
 seq_parameter_set_id : 0 
 entropy_coding_mode_flag : 0 
 pic_order_present_flag : 0 
 num_slice_groups_minus1 : 0 
 slice_group_map_type : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 weighted_pred_flag : 0 
 weighted_bipred_idc : 0 
 pic_init_qp_minus26 : 0 
 pic_init_qs_minus26 : 0 
 chroma_qp_index_offset : 0 
 deblocking_filter_control_present_flag : 0 
 constrained_intra_pred_flag : 0 
 redundant_pic_cnt_present_flag : 0 
 transform_8x8_mode_flag : 0 
 pic_scaling_matrix_present_flag : 0 
 second_chroma_qp_index_offset : 0 
!! Found NAL at offset 33 (0x0021), size 339 (0x0153) 
XX 80 00 00 01 65 B8 04 04 3F FF F8 7A 28 03 EF BE 
==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 3 
 nal_unit_type : 5 ( Coded slice of an IDR picture ) 
======= Slice Header =======
 first_mb_in_slice : 0 
 slice_type : 2 ( I slice ) 
 pic_parameter_set_id : 0 
 frame_num : 0 
 field_pic_flag : 0 
 bottom_field_flag : 0 
 idr_pic_id : 0 
 pic_order_cnt_lsb : 0 
 delta_pic_order_cnt_bottom : 0 
 redundant_pic_cnt : 0 
 direct_spatial_mv_pred_flag : 0 
 num_ref_idx_active_override_flag : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 cabac_init_idc : 0 
 slice_qp_delta : -16 
 sp_for_switch_flag : 0 
 slice_qs_delta : 0 
 disable_deblocking_filter_idc : 0 
 slice_alpha_c0_offset_div2 : 0 
 slice_beta_offset_div2 : 0 
 slice_group_change_cycle : 0 
=== Prediction Weight Table ===
 luma_log2_weight_denom : 0 
 chroma_log2_weight_denom : 0 
=== Ref Pic List Reordering ===
 ref_pic_list_reordering_flag_l0 : 0 
 ref_pic_list_reordering_flag_l1 : 0 
=== Decoded Ref Pic Marking ===
 no_output_of_prior_pics_flag : 0 
 long_term_reference_flag : 0 
 adaptive_ref_pic_marking_mode_flag : 0 
!! Found NAL at offset 376 (0x0178), size 1897 (0x0769) 
XX 00 00 00 01 41 E0 20 3F 36 5F FF 87 A2 80 03 7E 
==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 2 
 nal_unit_type : 1 ( Coded slice of a non-IDR picture ) 
======= Slice Header =======
 first_mb_in_slice : 0 
 slice_type : 0 ( P slice ) 
 pic_parameter_set_id : 0 
 frame_num : 1 
 field_pic_flag : 0 
 bottom_field_flag : 0 
 idr_pic_id : 0 
 pic_order_cnt_lsb : 0 
 delta_pic_order_cnt_bottom : 0 
 redundant_pic_cnt : 0 
 direct_spatial_mv_pred_flag : 0 
 num_ref_idx_active_override_flag : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 cabac_init_idc : 0 
 slice_qp_delta : -15 
 sp_for_switch_flag : 0 
 slice_qs_delta : 0 
 disable_deblocking_filter_idc : 0 
 slice_alpha_c0_offset_div2 : 0 
 slice_beta_offset_div2 : 0 
 slice_group_change_cycle : 0 
=== Prediction Weight Table ===
 luma_log2_weight_denom : 0 
 chroma_log2_weight_denom : 0 
=== Ref Pic List Reordering ===
 ref_pic_list_reordering_flag_l0 : 0 
 ref_pic_list_reordering_flag_l1 : 0 
=== Decoded Ref Pic Marking ===
 no_output_of_prior_pics_flag : 0 
 long_term_reference_flag : 0 
 adaptive_ref_pic_marking_mode_flag : 0 
!! Found NAL at offset 2277 (0x08E5), size 2989 (0x0BAD) 
XX 00 00 00 01 41 E0 40 3F 37 0F 0C 3E D8 A0 00 4E 
==================== NAL ====================
 forbidden_zero_bit : 0 
 nal_ref_idc : 2 
 nal_unit_type : 1 ( Coded slice of a non-IDR picture ) 
======= Slice Header =======
 first_mb_in_slice : 0 
 slice_type : 0 ( P slice ) 
 pic_parameter_set_id : 0 
 frame_num : 2 
 field_pic_flag : 0 
 bottom_field_flag : 0 
 idr_pic_id : 0 
 pic_order_cnt_lsb : 0 
 delta_pic_order_cnt_bottom : 0 
 redundant_pic_cnt : 0 
 direct_spatial_mv_pred_flag : 0 
 num_ref_idx_active_override_flag : 0 
 num_ref_idx_l0_active_minus1 : 0 
 num_ref_idx_l1_active_minus1 : 0 
 cabac_init_idc : 0 
 slice_qp_delta : -15 
 sp_for_switch_flag : 0 
 slice_qs_delta : 0 
 disable_deblocking_filter_idc : 0 
 slice_alpha_c0_offset_div2 : 0 
 slice_beta_offset_div2 : 0 
 slice_group_change_cycle : 0 
=== Prediction Weight Table ===
 luma_log2_weight_denom : 0 
 chroma_log2_weight_denom : 0 
=== Ref Pic List Reordering ===
 ref_pic_list_reordering_flag_l0 : 0 
 ref_pic_list_reordering_flag_l1 : 0 
=== Decoded Ref Pic Marking ===
 no_output_of_prior_pics_flag : 0 
 long_term_reference_flag : 0 
 adaptive_ref_pic_marking_mode_flag : 0 
!! Found NAL at offset 5270 (0x1496), size 4074 (0x0FEA) 
XX 00 00 00 01 41 E0 60 10 CD B7 C1 BF B8 36 00 6A 

我想知道RTSP客户端(播放软件)是如何从比特流中读取(或评估)帧率的。这些信息存储在哪里,如何发送到播放软件?

客户端不需要知道播放接收到的视频的帧速率。播放是根据RTP和NTP时间戳计算的呈现时间进行的。

流";rtsp://192.168.1.82:8554/testStream/";video/H264:已接收10874字节。展示时间:567993623.442288

流";rtsp://192.168.1.82:8554/testStream/";video/H264:接收12054字节。展示时间:567993623.482288

您似乎没有正确设置演示时间。这可能是你每秒25帧的速度。如果设置正确,testRTSPClient将每秒打印9个样本,它们之间的差异大约为1/9秒。你的差异大约是40毫秒,给你25帧/秒。

a=帧速率:9.0

我已经开发了使用各种帧速率内容的RTSP服务器,并且从未需要设置SDP属性。我不确定是什么客户端软件真正利用了它

VLC播放器-视频是不稳定的,因为VLC预计下一帧会更早(25帧而不是9帧)。结果,vlc将缓冲时间增加到例如10s,在此期间视频被冻结。

根据我过去几年的经验,VLC不擅长回放现场内容。我测试过各种RTSP服务器,VLC经常遇到问题,即使在使用更大的网络抖动缓冲区时也是如此。

你可能想看看其他一些东西:

fDurationInMicroseconds=1100000;

如注释中所述,对于实时源,fDurationInMicroseconds应设置为0。

b=AS:500

比特率似乎设置错误:您的流绝对不是500kbps。

问题已解决。我在流媒体上使用了错误的类。此处描述:http://lists.live555.com/pipermail/live-devel/2016-April/020013.html

根据Ross的提示,我将H264VideoStreamFramer替换为H264VideoStreamDiscreteFramer,并从NAL单元中删除了起始代码。现在一切都很顺利。

最新更新