38 Commits

Author SHA1 Message Date
Florent Castelli
8037fc6ffa Migrate absl::optional to std::optional
Bug: webrtc:342905193
No-Try: True
Change-Id: Icc968be43b8830038ea9a1f5f604307220457807
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361021
Auto-Submit: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42911}
2024-09-02 12:16:47 +00:00
Markus Handell
e864dec2e9 VSyncEncodeAdapterMode: avoid UAF.
This CL fixes a problem where VSEAM's `queue_` was dereferenced
post destruction. A sequence triggering it is:

0. FrameCadenceAdapter (FCA) is configured with Metronome with >= 34 ms tick delay.
1. A frame is queued for processing on worker queue.
2. The FCA is destroyed. The contained VSyncEncodeAdapterMode instance is scheduled for deletion on worker queue.
3. Encode queue is destroyed.
4. Worker queue is executed, which runs a task that dereferences `queue_`.

Bug: chromium:356423094
Change-Id: Iae8dc070304ef5ec0cfb0b4f27bbb7fe86e7def7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/358660
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42745}
2024-08-08 11:02:50 +00:00
Markus Handell
a57229bf36 Hard-code WebRTC-ZeroHertzScreenshare default-on.
The field trial has been default on for ages. This CL removes it.

Bug: b/40200151
Change-Id: I171f663a3e725b856238b14b26d083f6684586e4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/347621
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42080}
2024-04-16 09:29:39 +00:00
Per K
e975b44a45 Reland "FrameCadenceAdapter keep track of Input framerate"
This reverts commit d427e83a15ad2950095ce1d352cc7e11eaf6cad3.

Reason for revert: Flaky test fixed.

Refactor FrameCandenceAdapter to keep track of input frame rate. This fixes an issue where frame rate is calculated too low if congestion window drop a frame.

Also a field trial WebRTC-FrameCadenceAdapter-UseVideoFrameTimestamp is added to control if VideoFrame timestamp should be used or local clock when calculating frame rate.
Uma is recorded to tell if input frame timestamp is monotonically increasing.

Bug: webrtc:10481, webrtc:15887, webrtc:15893
Change-Id: I76268aa0991dbc99c1b881fb251a76aa54ff2673
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/344561
Reviewed-by: Erik Språng <sprang@google.com>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41972}
2024-03-27 12:58:03 +00:00
Per Kjellander
d427e83a15 Revert "FrameCadenceAdapter keep track of Input framerate"
This reverts commit 784af1f42e89735587c442855fa01fc90475c449.

Reason for revert: Seems like test test_support_unittests 
 ResolutionAdaptsToAvailableBandwidth is flaky with this cl.

Original change's description:
> FrameCadenceAdapter keep track of Input framerate
>
> Refactor FrameCandenceAdapter to keep track of input frame rate.
>
> Also a field trial WebRTC-FrameCadenceAdapter-UseVideoFrameTimestamp is added to control if VideoFrame timestamp should be used or local clock when calculating frame rate.
> Uma is recorded to tell if input frame timestamp is monotonically increasing.
>
> Bug: webrtc:10481, webrtc:15887
> Change-Id: I6d698e9f9dcfe8c023d2d35371435c47f70102b0
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/342760
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#41967}

Bug: webrtc:10481, webrtc:15887
Change-Id: Id9672764768f2f40f8e711e990ad8ac18c28efcc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/344560
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41969}
2024-03-26 15:56:15 +00:00
Per K
784af1f42e FrameCadenceAdapter keep track of Input framerate
Refactor FrameCandenceAdapter to keep track of input frame rate.

Also a field trial WebRTC-FrameCadenceAdapter-UseVideoFrameTimestamp is added to control if VideoFrame timestamp should be used or local clock when calculating frame rate.
Uma is recorded to tell if input frame timestamp is monotonically increasing.

Bug: webrtc:10481, webrtc:15887
Change-Id: I6d698e9f9dcfe8c023d2d35371435c47f70102b0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/342760
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41967}
2024-03-26 10:44:29 +00:00
henrika
b7ec05777a FrameCadenceAdapter: now sets queue_overload based on encoder load
Measures the time consumed by OnFrame (e.g. the encoding time) and
sets an overload flag during N subsequent frames if the time is
longer than the current frame time. N is set to the number of
received frames on the network thread while being blocked by
encoding.

The queue overload mechanism for zero hertz can be disabled using the
WebRTC-ZeroHertzQueueOverload kill switch.

Also adds a UMA called WebRTC.Screenshare.ZeroHz.QueueOverload.

Bug: webrtc:15539
Change-Id: If81481c265d3e845485f79a2a1ac03dcbcc3ffc3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/332381
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41489}
2024-01-09 14:29:04 +00:00
Zhaoliang Ma
f089d7ea54 Reland "FrameCadenceAdapter: align video encoding to metronome"
This is a reland of commit b39c2a8464c48306a495f14beccf431b91e51efd

Original change's description:
> FrameCadenceAdapter: align video encoding to metronome
>
> This CL aligns the video encoding tasks to metronome tick which
> similar with the metronome decoding.
>
> Design doc: https://docs.google.com/document/d/18PvEgS-DehClK6twCSCATOlX-j9acmXd-3vjb0tR9-Y
>
> Bug: b/304158952
> Change-Id: I262bd4a5097fdaeed559b9d7391a059ae86e2d63
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327460
> Reviewed-by: Markus Handell <handellm@webrtc.org>
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Commit-Queue: Zhaoliang Ma <zhaoliang.ma@intel.com>
> Cr-Commit-Position: refs/heads/main@{#41469}

Bug: b/304158952
Change-Id: Icf4e1ad91f5c98f3c32a88ffe4d6277e907353e6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/333464
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41479}
2024-01-08 13:54:56 +00:00
Björn Terelius
78a57efb29 Revert "FrameCadenceAdapter: align video encoding to metronome"
This reverts commit b39c2a8464c48306a495f14beccf431b91e51efd.

Reason for revert: Breaks downstream build

Original change's description:
> FrameCadenceAdapter: align video encoding to metronome
>
> This CL aligns the video encoding tasks to metronome tick which
> similar with the metronome decoding.
>
> Design doc: https://docs.google.com/document/d/18PvEgS-DehClK6twCSCATOlX-j9acmXd-3vjb0tR9-Y
>
> Bug: b/304158952
> Change-Id: I262bd4a5097fdaeed559b9d7391a059ae86e2d63
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327460
> Reviewed-by: Markus Handell <handellm@webrtc.org>
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Commit-Queue: Zhaoliang Ma <zhaoliang.ma@intel.com>
> Cr-Commit-Position: refs/heads/main@{#41469}

Bug: b/304158952
Change-Id: I6f7a3d45cc24b63bc1fe92a93bf5c8d5058f32a8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/333482
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Björn Terelius <terelius@webrtc.org>
Auto-Submit: Björn Terelius <terelius@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41471}
2024-01-04 20:02:49 +00:00
Zhaoliang Ma
b39c2a8464 FrameCadenceAdapter: align video encoding to metronome
This CL aligns the video encoding tasks to metronome tick which
similar with the metronome decoding.

Design doc: https://docs.google.com/document/d/18PvEgS-DehClK6twCSCATOlX-j9acmXd-3vjb0tR9-Y

Bug: b/304158952
Change-Id: I262bd4a5097fdaeed559b9d7391a059ae86e2d63
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327460
Reviewed-by: Markus Handell <handellm@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Zhaoliang Ma <zhaoliang.ma@intel.com>
Cr-Commit-Position: refs/heads/main@{#41469}
2024-01-04 04:14:12 +00:00
henrika
8f16ce98c2 FrameCadenceAdapter: improves performance under repeat and high load
This change ensures that the FCA now is informed about a new max fps
when VideoStreamEncoder::OnVideoSourceRestrictionsUpdated is called.

The latest restricted frame rate which is provided to the FCA will
only affect the cadence of repeated non-idle (quality has not
converged) frames and the main goal is to ensure that the FCA reduces
its repeat rate in situations where the video source is constrained.

UpdateVideoSourceRestrictions is added to the FrameCadenceAdapter API
and it is called from the VideoStreamEncoder when its source
parameters (resolution and/or frame rate) are restricted.

This modification has no effect on the flow driven by
ProcessOnDelayedCadence (non repeated frames).

Bug: webrtc:15539
Change-Id: I26dee6480e5137f82c5ccf57091b737cad82dbf6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/328300
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41308}
2023-12-04 13:37:24 +00:00
Markus Handell
84c016a024 FrameCadenceAdapter: use distinct signal for queue overload.
The FrameCadenceAdapterInterface::Callback::OnFrame method in
VideoStreamEncoder only changed frame handling on
frames_scheduled_for_processing being 1. This CL changes
the parameter to more explicitly signal queue overload via
boolean parameter.

Bug: None
Change-Id: I1eb46b34fc4d748b7e2f1921642497c939adf197
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/327761
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41226}
2023-11-23 17:31:37 +00:00
henrika
644025c51f FrameCadenceAdapter: keeps max_fps in sync with constraints during 0Hz
Before this change, the FCA did not not update its cadence when
max_fps was changed and zero-hertz was already enabled.

See https://paste.googleplex.com/6300124249587712 for more details.

Bug: chromium:1400204
Change-Id: I95d80bdfa85ecac8681784b2b29e98d1a587ba53
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/326105
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41100}
2023-11-07 19:21:47 +00:00
Markus Handell
fb98b01061 FrameCadenceAdapter: stop delayed refresh frame calls on dtor.
The FrameCadenceAdapter starts a delayed task to request a
new refresh frame on receiving frame drop. However, the
resulting RepeatingTaskHandle was not Stop()ed on destruction,
leading to UAF.

Fixed: chromium:1478944
Change-Id: Iba441420953e989cfc7fcfd2f358b5b30f375786
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/320200
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40747}
2023-09-14 11:31:52 +00:00
Markus Handell
f2827c4b1a FrameCadenceAdapter: schedule repeats before issuing decodes.
The code currently issues frames for encode before scheduling
a new repeat. Swap this order to account for time taken by for
slow encodes.

Bug: webrtc:15456
Change-Id: I74177069e30c1bf65268231ffba033411a0f7b9a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/318580
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40690}
2023-09-04 15:16:46 +00:00
Markus Handell
8fa8619d7e FrameCadenceAdapter: account for encode sequence contention.
The synthetic delay added in ZeroHzAdapterMode::OnFrame does not
account for delay with respect to the initial frame post from
FrameCadenceAdapter::OnFrame. Fix this to account for time spent
in contention on the encode sequence.

Bug: webrtc:15456
Change-Id: I63446e8dfe8f62b09d972434a705e912f8a73d69
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/318420
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40675}
2023-08-31 17:45:51 +00:00
Markus Handell
06130548fc Remove unused histograms.
This CL removes histograms which are no longer of use.

Bug: chromium:1255737
Change-Id: I7eb7e2cfbb03126257b51bfaa30d764b37dd9bd0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/306200
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40114}
2023-05-23 09:04:31 +00:00
Markus Handell
5a77e51c17 FrameCadenceAdapter: survive layer updates for unconfigured layers.
The frame cadence adapter would previously crash on encountering
unconfigured layer updates. Fix this to silently ignore such updates.

Fixed: webrtc:14417
Change-Id: I524aa196f6aed10ffb8cd4ddcb4b053c71dfabba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/273325
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37980}
2022-09-01 13:37:17 +00:00
Philipp Hancke
a204ad210d clean up misc TimeDelta use
follow-up from https://webrtc-review.googlesource.com/c/src/+/262810

* replace Time::Millis(0) and TimeDelta::Millis(0) with ::Zero()
* drop unnecessary webrtc namespace from some TimeDeltas
* make TimeDelta do the unit conversion for stats

BUG=webrtc:13756

Change-Id: Ic60625ae0fc7959a47a6be9f5051851feaf76373
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265875
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37664}
2022-08-02 13:52:36 +00:00
Danil Chapovalov
95eeaa7aca Migrate video/ to absl::AnyInvocable based TaskQueueBase interface
Bug: webrtc:14245
Change-Id: Ibd98d3a0c548443578953ef3e25aee9919eea3d3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267980
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37465}
2022-07-06 14:40:25 +00:00
Artem Titov
c374d11fac Move to_queued_task.h and pending_task_safety_flag.h into public API
Bug: b/235812579
Change-Id: I9fa3dc4a65044df8b44fff4e9bfeac7233fa381c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/266080
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37248}
2022-06-17 09:20:39 +00:00
Markus Handell
f5a507955a FrameCadenceAdapter: ensure frame arrival after drop.
This CL accomplishes three things:

1) It enables feeding frame drop indications into the
AdaptedVideoTrackSource for the benefit of downstream projects.

2) Under zero hertz source delivery, a discarded frame ending a
sequence of frames which happened to contain important information
can be seen as a capture freeze. Avoid this by starting requesting
refresh frames after a grace period.

3) It changes the duration until first refresh frame requests on new
streams to three frame periods.

Bug: chromium:1324120, chromium:1336952
Change-Id: I0214852f1a26540588f6c193dd88a65c34ec0d99
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265871
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37238}
2022-06-16 14:12:20 +00:00
Niels Möller
105711e9ad Move rtc::make_ref_counted to api/
Bug: webrtc:12701
Change-Id: If49095b101c1a1763c2a44a0284c0d670cce953f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265390
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37219}
2022-06-15 09:47:38 +00:00
Markus Handell
f7f0b2108f ZeroHz: Repeat refresh frame requests until a frame is received.
When MediaStreamVideoSource::RequestRefreshFrame is called, the
capturer most often emits a refresh frame. Due to various
conditions such as for example timing of prior delivery,
these frames can be dropped at various places in the input
pipeline into WebRTC.

This change ensures the frame cadence adapter repeatedly
requests refresh frames at max fps frequency until one is
received, in which case the requests cease.

Fixed: chromium:1324120
Change-Id: I90f85d31b132b6c441aa1c28c5eff85e3dc365ad
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/263520
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36998}
2022-05-25 07:58:49 +00:00
Markus Handell
9d04a78f73 ZeroHertz: expose time to first frame statistic.
Bug: chromium:1324120
Change-Id: Ie609da309427ca5d2ff8585a8dc730586553c725
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262247
Auto-Submit: Markus Handell <handellm@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36867}
2022-05-12 17:35:13 +00:00
Jonas Oreland
e62c2f2c77 WebRTC-DeprecateGlobalFieldTrialString/Enabled/ - part 12/inf
rename WebRtcKeyValueConfig to FieldTrialsView

Bug: webrtc:10335
Change-Id: If725bd498c4c3daf144bee638230fa089fdde833
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256965
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36365}
2022-03-29 10:14:00 +00:00
Jonas Oreland
8ca06137dc WebRTC-DeprecateGlobalFieldTrialString/Enabled/ - part 4/inf
convert almost all of video/ (and the collateral)

Bug: webrtc:10335
Change-Id: Ic94e05937f54d11ee8a635b6b66fd146962d9f11
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254601
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36192}
2022-03-14 14:36:35 +00:00
Markus Handell
818e7fbc64 ZeroHertzAdapterMode: handle pending key frame requests.
The frame cadence adapter ignores key frame processing that happens
before the point where zero-hertz mode is activated, which leads to
no refresh frame requests if the key frame request comes too early.

Fix this to register a pending refresh frame request that gets
serviced when zero-hertz mode is activated. The CL changes the
FrameCadenceAdapterInterface::ProcessKeyFrameRequest from returning
whether to request a refresh frame into the frame cadence adapter
actively doing so itself via a new Callback::RequestRefreshFrame
API.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: I53c2dbf6468e883eb2a2e81498e7134b1b35c336
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242963
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35598}
2021-12-30 13:37:10 +00:00
Markus Handell
90a7e2ceba ZeroHertzAdapterMode: do not dead-reckon repeated frame timestamps.
Timestamps are currently dead-reckoned for repeated frames in
zero-hertz mode. This leads to an ever increasing
totalPacketSendDelay metric in chrome://webrtc-internals which is
bad.

Fix this by tracking the origin timestamp of the first delay and
measuring time's progression since then. A unit test was added
which fails with the previous version.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: I8627b91424f9bc56305b1dbd6a4c0624b6b3669d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242863
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35595}
2021-12-29 23:14:47 +00:00
Markus Handell
cb237f8822 ZeroHertzAdapterMode: reset convergence info on key frame requests.
The QP value of encoded key frames is normally very large. However,
the zero-hertz mode is retaining quality convergence info, leading
to only a single short repeat on key frame request when idle
repeating.

Fix this by resetting quality convergence information on key frame
requests, ensuring zero-hertz mode goes back to idle repeating
only when quality has converged again.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: Ia1ad686cc98007f01c8aaef9162011837575938c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242862
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35594}
2021-12-29 22:03:11 +00:00
Markus Handell
e59fee87fb ZeroHertzAdapterMode: activate earlier.
Careful analysis of logs related to the requesting of refresh
frames from the source revealed an uncomfortable truth:
zero-hertz mode activates first when the first frame has been
received in the VideoStreamEncoder, because the number of simulcast
layers can only be computed when frame dimensions are known. This
fact means that the currently implemented logic for requesting
refresh frames is noneffective.

Fix this by
1. Activating zero-hertz mode prior of knowing the final layer
count. This causes refresh frame requests to happen without any
frames received from the source.
2. Making layer count dynamically configurable. Zero-hertz mode
considers layer quality unconverged after such a reconfiguration.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: I0ecea4d2a8442a00e3b79b146dd39a5f4ac561d9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242860
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35593}
2021-12-29 20:28:38 +00:00
Markus Handell
2e0f4f0f37 ZeroHertzAdapterMode: handle key frame requests.
Under zero-hertz mode, provided that a frame arrived to the
VideoStreamEncoder, the receiver may experience up to a second
between incoming frames. This results in key frame requests getting
serviced with that delay, which is undesired.

What's worse is also the fact that if no frame ever arrived to the
VideoStreamEncoder, it will not service the keyframe requests at all
until the first frame comes.

This change introduces VideoSourceInterface::RequestRefreshFrame
which results in a refresh frame being sent from complying sources.
The method is used under zero-hertz mode from the VideoStreamEncoder
when frames didn't arrive to it yet (with changes to the zero-hertz
adapter).

With this change, when the frame adapter has received at least one
frame, it will conditionally repeat the last frame in response to the
key frame request.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: I6f97813b3a938747357d45e5dda54f759129b44d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242361
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35562}
2021-12-21 19:52:56 +00:00
Markus Handell
8d87c463d9 ZeroHertzAdapterMode: slow down repeats on quality convergence.
The frame cadence adapter previously resulted in unconditional
frame repeating at max FPS. Change this to slow down to an idle
rate (1 Hz) when quality convergence in all configured spatial
layers has been achieved.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: Ifa593dbf8a61aa29da20ac250da332734ae82791
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241421
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35547}
2021-12-16 12:01:30 +00:00
Markus Handell
29dd8d864b ZeroHertzAdapterMode: delay & repeat frames.
This change introduces a delay in the frame cadence forwarded to
the VideoStreamEncoder. In case the delivery of frames into the
VideoSinkInterface stops, ZeroHertzAdapterMode will repeat a
previously received frame until new frames appear.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: I689ac63a41a09951715ea2c26f491e7c4ad0d11d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/240060
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35542}
2021-12-15 12:22:29 +00:00
Markus Handell
ee22543829 Zero-hertz encoding mode: avoid encoder bitrate overshooting.
The encoders wrapped in VideoStreamEncoder grossly over-estimates
available bitrate when capture FPS falls close to zero, and frames
re-commence highly frequent delivery. Avoid this by moving the input
RateStatistics inside VSE into the frame cadence adapter, and changing
the reported framerate under zero-hertz encoding mode to always return
the configured max FPS.

Bug: chromium:1255737
Change-Id: Iaa71ef51c0755b12e24e435d86d9562122ed494e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239126
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35431}
2021-11-29 13:48:00 +00:00
Markus Handell
9a478b527d VideoStreamEncoder: expect frame entry on the encoder queue.
This change switches the sequence used by the FrameCadenceAdapter
to be the encoder_queue, enabling VideoStreamEncoder::OnFrame to be
invoked directly on the encoder_queue and eliminates the contained
PostTasks.

Bug: chromium:1255737
Change-Id: Ib86fc96ad2be9a38585fef2535855e3f9cc7e57c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238171
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35380}
2021-11-18 15:35:17 +00:00
Markus Handell
28c7180999 VideoStreamEncoder: simplify threading.
VideoStreamEncoder receives frames on an undefined threading
context with the only requirement being that frames are serially
arriving. This CL changes this to post all frames arriving at the
FrameCadenceAdapter to the worker thread before further
processing, transitively leading to frame entry into the
VideoStreamEncoder on the worker thread.

Bug: chromium:1255737
Change-Id: I04d69cb4a5048d671d2dcd3bd6d669fbcda52b3f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237142
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35320}
2021-11-08 10:42:48 +00:00
Markus Handell
b4e96d48a2 VideoStreamEncoder: Introduce frame cadence adapter.
This change introduces a new FrameCadenceAdapter class which takes the
role of being a VideoFrameSinkInterface<> instead of VideoStreamEncoder.
The FrameCadenceAdapter will see its functionality grow in future CLs
and eventually enable screenshare capture sources to have zero hertz as
the minimum capture frequency.

This CL moves logic related to UMA collection and constraints into the
adapter.

The adapter has two major modes. Future functionality is planned to be
added under the WebRTC-ZeroHertzScreenshare field trial. Unit tests are
added that verify passthrough operation when WebRTC-ZeroHertzScreenshare
isn't specified or disabled.

Just specifying the WebRTC-ZeroHertzScreenshare field trial isn't
enough to activate the feature, but the caller has to additionally
configure screen content type, minimum FPS 0, and maximum FPS > 0 for
the new mode.

go/rtc-0hz-present

Bug: chromium:1255737
Change-Id: I1799110ed40843152786ad80df10acfb83a608b1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236682
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35315}
2021-11-05 12:37:45 +00:00