From dde1cb6212d3abf26b35abfc737ca5e90f5518d8 Mon Sep 17 00:00:00 2001 From: Philipp Hancke Date: Wed, 11 Oct 2023 12:03:56 +0300 Subject: [PATCH] Add note about two-byte extension to VLA docs since the extension can be too large to fit the 16 bytes available to one-byte extensions https://www.rfc-editor.org/rfc/rfc8285#section-4.2 when including the width and height fields. Also document when those fields are sent. BUG=webrtc:12000 Change-Id: If17f57d40c0bde9b060f223c548e407d6c124b82 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/321200 Reviewed-by: Harald Alvestrand Reviewed-by: Per Kjellander Commit-Queue: Philipp Hancke Cr-Commit-Position: refs/heads/main@{#40910} --- .../rtp-hdrext/video-layers-allocation00/README.md | 12 +++++++++--- modules/rtp_rtcp/source/rtp_sender_video.cc | 6 +++--- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/native-code/rtp-hdrext/video-layers-allocation00/README.md b/docs/native-code/rtp-hdrext/video-layers-allocation00/README.md index c4454d8ee1..cfa8c4cb0d 100644 --- a/docs/native-code/rtp-hdrext/video-layers-allocation00/README.md +++ b/docs/native-code/rtp-hdrext/video-layers-allocation00/README.md @@ -22,6 +22,10 @@ rtp stream (SVC), or independent spatial layers sent on multiple rtp streams ## RTP header extension format +Note: when including the optional width, height and maximum framerate +fields, the total data length of the extension can exceed 16 bytes +and is sent as a two-byte header extension [1] + ### Data layout ``` @@ -68,7 +72,7 @@ alignment. layers. Values are stored in ascending order of spatial id. Zero-padded to byte alignment. -Target bitrate in kbps. Values are stored using leb128 encoding [1]. One value per +Target bitrate in kbps. Values are stored using leb128 encoding [2]. One value per temporal layer. Values are stored in (RTP stream id, spatial id, temporal id) ascending order. All bitrates are total required bitrate to receive the corresponding layer, i.e. in simulcast mode they include only corresponding @@ -78,9 +82,11 @@ temporal layers are also included. Resolution and framerate. Optional. Presence is inferred from the rtp header extension size. Encoded (width - 1), 16-bit, (height - 1), 16-bit, max frame rate 8-bit per spatial layer per RTP stream. Values are stored in (RTP stream -id, spatial id) ascending order. +id, spatial id) ascending order. Only sent when the resolution differs from the +last values, the framerate changed by more than 5fps and on key frames. An empty layer allocation (i.e nothing sent on ssrc) is encoded as special case with a single 0 byte. -[1] https://aomediacodec.github.io/av1-spec/#leb128 +[1] https://www.rfc-editor.org/rfc/rfc8285#section-4.3 +[2] https://aomediacodec.github.io/av1-spec/#leb128 diff --git a/modules/rtp_rtcp/source/rtp_sender_video.cc b/modules/rtp_rtcp/source/rtp_sender_video.cc index 4d819a80a9..9f70a78f48 100644 --- a/modules/rtp_rtcp/source/rtp_sender_video.cc +++ b/modules/rtp_rtcp/source/rtp_sender_video.cc @@ -118,7 +118,7 @@ absl::optional LoadVideoPlayoutDelayOverride( // Some packets can be skipped and the stream can still be decoded. Those // packets are less likely to be retransmitted if they are lost. -bool PacketWillLikelyBeRequestedForRestransmitionIfLost( +bool PacketWillLikelyBeRequestedForRestransmissionIfLost( const RTPVideoHeader& video_header) { return IsBaseLayer(video_header) && !(video_header.generic.has_value() @@ -442,7 +442,7 @@ void RTPSenderVideo::AddRtpHeaderExtensions(const RTPVideoHeader& video_header, first_packet && send_allocation_ != SendVideoLayersAllocation::kDontSend && (video_header.frame_type == VideoFrameType::kVideoFrameKey || - PacketWillLikelyBeRequestedForRestransmitionIfLost(video_header))) { + PacketWillLikelyBeRequestedForRestransmissionIfLost(video_header))) { VideoLayersAllocation allocation = allocation_.value(); allocation.resolution_and_frame_rate_is_valid = send_allocation_ == SendVideoLayersAllocation::kSendWithResolution; @@ -733,7 +733,7 @@ bool RTPSenderVideo::SendVideo(int payload_type, } if (video_header.frame_type == VideoFrameType::kVideoFrameKey || - PacketWillLikelyBeRequestedForRestransmitionIfLost(video_header)) { + PacketWillLikelyBeRequestedForRestransmissionIfLost(video_header)) { // This frame will likely be delivered, no need to populate playout // delay extensions until it changes again. playout_delay_pending_ = false;