This reverts commit d427e83a15ad2950095ce1d352cc7e11eaf6cad3.
Reason for revert: Flaky test fixed.
Refactor FrameCandenceAdapter to keep track of input frame rate. This fixes an issue where frame rate is calculated too low if congestion window drop a frame.
Also a field trial WebRTC-FrameCadenceAdapter-UseVideoFrameTimestamp is added to control if VideoFrame timestamp should be used or local clock when calculating frame rate.
Uma is recorded to tell if input frame timestamp is monotonically increasing.
Bug: webrtc:10481, webrtc:15887, webrtc:15893
Change-Id: I76268aa0991dbc99c1b881fb251a76aa54ff2673
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/344561
Reviewed-by: Erik Språng <sprang@google.com>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41972}
This reverts commit 784af1f42e89735587c442855fa01fc90475c449.
Reason for revert: Seems like test test_support_unittests
ResolutionAdaptsToAvailableBandwidth is flaky with this cl.
Original change's description:
> FrameCadenceAdapter keep track of Input framerate
>
> Refactor FrameCandenceAdapter to keep track of input frame rate.
>
> Also a field trial WebRTC-FrameCadenceAdapter-UseVideoFrameTimestamp is added to control if VideoFrame timestamp should be used or local clock when calculating frame rate.
> Uma is recorded to tell if input frame timestamp is monotonically increasing.
>
> Bug: webrtc:10481, webrtc:15887
> Change-Id: I6d698e9f9dcfe8c023d2d35371435c47f70102b0
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/342760
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#41967}
Bug: webrtc:10481, webrtc:15887
Change-Id: Id9672764768f2f40f8e711e990ad8ac18c28efcc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/344560
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41969}
Refactor FrameCandenceAdapter to keep track of input frame rate.
Also a field trial WebRTC-FrameCadenceAdapter-UseVideoFrameTimestamp is added to control if VideoFrame timestamp should be used or local clock when calculating frame rate.
Uma is recorded to tell if input frame timestamp is monotonically increasing.
Bug: webrtc:10481, webrtc:15887
Change-Id: I6d698e9f9dcfe8c023d2d35371435c47f70102b0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/342760
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41967}
This is a reland of commit b39c2a8464c48306a495f14beccf431b91e51efd
Original change's description:
> FrameCadenceAdapter: align video encoding to metronome
>
> This CL aligns the video encoding tasks to metronome tick which
> similar with the metronome decoding.
>
> Design doc: https://docs.google.com/document/d/18PvEgS-DehClK6twCSCATOlX-j9acmXd-3vjb0tR9-Y
>
> Bug: b/304158952
> Change-Id: I262bd4a5097fdaeed559b9d7391a059ae86e2d63
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327460
> Reviewed-by: Markus Handell <handellm@webrtc.org>
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Commit-Queue: Zhaoliang Ma <zhaoliang.ma@intel.com>
> Cr-Commit-Position: refs/heads/main@{#41469}
Bug: b/304158952
Change-Id: Icf4e1ad91f5c98f3c32a88ffe4d6277e907353e6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/333464
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41479}
This CL aligns the video encoding tasks to metronome tick which
similar with the metronome decoding.
Design doc: https://docs.google.com/document/d/18PvEgS-DehClK6twCSCATOlX-j9acmXd-3vjb0tR9-Y
Bug: b/304158952
Change-Id: I262bd4a5097fdaeed559b9d7391a059ae86e2d63
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327460
Reviewed-by: Markus Handell <handellm@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Zhaoliang Ma <zhaoliang.ma@intel.com>
Cr-Commit-Position: refs/heads/main@{#41469}
This change ensures that the FCA now is informed about a new max fps
when VideoStreamEncoder::OnVideoSourceRestrictionsUpdated is called.
The latest restricted frame rate which is provided to the FCA will
only affect the cadence of repeated non-idle (quality has not
converged) frames and the main goal is to ensure that the FCA reduces
its repeat rate in situations where the video source is constrained.
UpdateVideoSourceRestrictions is added to the FrameCadenceAdapter API
and it is called from the VideoStreamEncoder when its source
parameters (resolution and/or frame rate) are restricted.
This modification has no effect on the flow driven by
ProcessOnDelayedCadence (non repeated frames).
Bug: webrtc:15539
Change-Id: I26dee6480e5137f82c5ccf57091b737cad82dbf6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/328300
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41308}
The FrameCadenceAdapterInterface::Callback::OnFrame method in
VideoStreamEncoder only changed frame handling on
frames_scheduled_for_processing being 1. This CL changes
the parameter to more explicitly signal queue overload via
boolean parameter.
Bug: None
Change-Id: I1eb46b34fc4d748b7e2f1921642497c939adf197
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327761
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41226}
The frame cadence adapter would previously crash on encountering
unconfigured layer updates. Fix this to silently ignore such updates.
Fixed: webrtc:14417
Change-Id: I524aa196f6aed10ffb8cd4ddcb4b053c71dfabba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/273325
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37980}
This CL accomplishes three things:
1) It enables feeding frame drop indications into the
AdaptedVideoTrackSource for the benefit of downstream projects.
2) Under zero hertz source delivery, a discarded frame ending a
sequence of frames which happened to contain important information
can be seen as a capture freeze. Avoid this by starting requesting
refresh frames after a grace period.
3) It changes the duration until first refresh frame requests on new
streams to three frame periods.
Bug: chromium:1324120, chromium:1336952
Change-Id: I0214852f1a26540588f6c193dd88a65c34ec0d99
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265871
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37238}
convert almost all of video/ (and the collateral)
Bug: webrtc:10335
Change-Id: Ic94e05937f54d11ee8a635b6b66fd146962d9f11
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254601
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36192}
The frame cadence adapter ignores key frame processing that happens
before the point where zero-hertz mode is activated, which leads to
no refresh frame requests if the key frame request comes too early.
Fix this to register a pending refresh frame request that gets
serviced when zero-hertz mode is activated. The CL changes the
FrameCadenceAdapterInterface::ProcessKeyFrameRequest from returning
whether to request a refresh frame into the frame cadence adapter
actively doing so itself via a new Callback::RequestRefreshFrame
API.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I53c2dbf6468e883eb2a2e81498e7134b1b35c336
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242963
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35598}
Under zero-hertz mode, provided that a frame arrived to the
VideoStreamEncoder, the receiver may experience up to a second
between incoming frames. This results in key frame requests getting
serviced with that delay, which is undesired.
What's worse is also the fact that if no frame ever arrived to the
VideoStreamEncoder, it will not service the keyframe requests at all
until the first frame comes.
This change introduces VideoSourceInterface::RequestRefreshFrame
which results in a refresh frame being sent from complying sources.
The method is used under zero-hertz mode from the VideoStreamEncoder
when frames didn't arrive to it yet (with changes to the zero-hertz
adapter).
With this change, when the frame adapter has received at least one
frame, it will conditionally repeat the last frame in response to the
key frame request.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I6f97813b3a938747357d45e5dda54f759129b44d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242361
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35562}
The frame cadence adapter previously resulted in unconditional
frame repeating at max FPS. Change this to slow down to an idle
rate (1 Hz) when quality convergence in all configured spatial
layers has been achieved.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: Ifa593dbf8a61aa29da20ac250da332734ae82791
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241421
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35547}
The encoders wrapped in VideoStreamEncoder grossly over-estimates
available bitrate when capture FPS falls close to zero, and frames
re-commence highly frequent delivery. Avoid this by moving the input
RateStatistics inside VSE into the frame cadence adapter, and changing
the reported framerate under zero-hertz encoding mode to always return
the configured max FPS.
Bug: chromium:1255737
Change-Id: Iaa71ef51c0755b12e24e435d86d9562122ed494e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239126
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35431}
This change switches the sequence used by the FrameCadenceAdapter
to be the encoder_queue, enabling VideoStreamEncoder::OnFrame to be
invoked directly on the encoder_queue and eliminates the contained
PostTasks.
Bug: chromium:1255737
Change-Id: Ib86fc96ad2be9a38585fef2535855e3f9cc7e57c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238171
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35380}
VideoStreamEncoder receives frames on an undefined threading
context with the only requirement being that frames are serially
arriving. This CL changes this to post all frames arriving at the
FrameCadenceAdapter to the worker thread before further
processing, transitively leading to frame entry into the
VideoStreamEncoder on the worker thread.
Bug: chromium:1255737
Change-Id: I04d69cb4a5048d671d2dcd3bd6d669fbcda52b3f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237142
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35320}
This change introduces a new FrameCadenceAdapter class which takes the
role of being a VideoFrameSinkInterface<> instead of VideoStreamEncoder.
The FrameCadenceAdapter will see its functionality grow in future CLs
and eventually enable screenshare capture sources to have zero hertz as
the minimum capture frequency.
This CL moves logic related to UMA collection and constraints into the
adapter.
The adapter has two major modes. Future functionality is planned to be
added under the WebRTC-ZeroHertzScreenshare field trial. Unit tests are
added that verify passthrough operation when WebRTC-ZeroHertzScreenshare
isn't specified or disabled.
Just specifying the WebRTC-ZeroHertzScreenshare field trial isn't
enough to activate the feature, but the caller has to additionally
configure screen content type, minimum FPS 0, and maximum FPS > 0 for
the new mode.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I1799110ed40843152786ad80df10acfb83a608b1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236682
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35315}
