Field telemetry has shown the combination of min_fps = 0 and max_fps >
0 is unused in the wild. Therefore it's safe to turn the
WebRTC-ZeroHertzScreenshare field trial default on unless the field
trial is disabled.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: Iea701218aa178b569333087b004106ffe2e85133
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/244086
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35621}
The frame cadence adapter ignores key frame processing that happens
before the point where zero-hertz mode is activated, which leads to
no refresh frame requests if the key frame request comes too early.
Fix this to register a pending refresh frame request that gets
serviced when zero-hertz mode is activated. The CL changes the
FrameCadenceAdapterInterface::ProcessKeyFrameRequest from returning
whether to request a refresh frame into the frame cadence adapter
actively doing so itself via a new Callback::RequestRefreshFrame
API.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I53c2dbf6468e883eb2a2e81498e7134b1b35c336
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242963
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35598}
Timestamps are currently dead-reckoned for repeated frames in
zero-hertz mode. This leads to an ever increasing
totalPacketSendDelay metric in chrome://webrtc-internals which is
bad.
Fix this by tracking the origin timestamp of the first delay and
measuring time's progression since then. A unit test was added
which fails with the previous version.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I8627b91424f9bc56305b1dbd6a4c0624b6b3669d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242863
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35595}
The QP value of encoded key frames is normally very large. However,
the zero-hertz mode is retaining quality convergence info, leading
to only a single short repeat on key frame request when idle
repeating.
Fix this by resetting quality convergence information on key frame
requests, ensuring zero-hertz mode goes back to idle repeating
only when quality has converged again.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: Ia1ad686cc98007f01c8aaef9162011837575938c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242862
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35594}
Careful analysis of logs related to the requesting of refresh
frames from the source revealed an uncomfortable truth:
zero-hertz mode activates first when the first frame has been
received in the VideoStreamEncoder, because the number of simulcast
layers can only be computed when frame dimensions are known. This
fact means that the currently implemented logic for requesting
refresh frames is noneffective.
Fix this by
1. Activating zero-hertz mode prior of knowing the final layer
count. This causes refresh frame requests to happen without any
frames received from the source.
2. Making layer count dynamically configurable. Zero-hertz mode
considers layer quality unconverged after such a reconfiguration.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I0ecea4d2a8442a00e3b79b146dd39a5f4ac561d9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242860
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35593}
Under zero-hertz mode, provided that a frame arrived to the
VideoStreamEncoder, the receiver may experience up to a second
between incoming frames. This results in key frame requests getting
serviced with that delay, which is undesired.
What's worse is also the fact that if no frame ever arrived to the
VideoStreamEncoder, it will not service the keyframe requests at all
until the first frame comes.
This change introduces VideoSourceInterface::RequestRefreshFrame
which results in a refresh frame being sent from complying sources.
The method is used under zero-hertz mode from the VideoStreamEncoder
when frames didn't arrive to it yet (with changes to the zero-hertz
adapter).
With this change, when the frame adapter has received at least one
frame, it will conditionally repeat the last frame in response to the
key frame request.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I6f97813b3a938747357d45e5dda54f759129b44d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242361
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35562}
The frame cadence adapter previously resulted in unconditional
frame repeating at max FPS. Change this to slow down to an idle
rate (1 Hz) when quality convergence in all configured spatial
layers has been achieved.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: Ifa593dbf8a61aa29da20ac250da332734ae82791
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241421
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35547}
This change introduces a delay in the frame cadence forwarded to
the VideoStreamEncoder. In case the delivery of frames into the
VideoSinkInterface stops, ZeroHertzAdapterMode will repeat a
previously received frame until new frames appear.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I689ac63a41a09951715ea2c26f491e7c4ad0d11d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/240060
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35542}
Fix multiple use-after-move issues in VideoReceiveStream2, as found by
clang-tidy:
video/video_receive_stream2.cc:259:
'config' used after it was moved
video/video_receive_stream2.cc:199:
move occurred here
Bug: chromium:1122844
Change-Id: I6367dc835f002718a5353c3e0b64c2a154e79925
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241201
Auto-Submit: Maksim Ivanov <emaxx@chromium.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35530}
Addresses case where 540*960 would not get a 135*240 layer.
Bug: webrtc:13469
Change-Id: Icc291c65114fb400cc71659d76a786e359e5996c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239820
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Konrad Hofbauer <hofbauer@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35507}
This class is superseded by VideoReceiveStream2.
Bug: webrtc:11489
Change-Id: I02b844868bafe67ce3e924fc23029ec300e934a7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/240063
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@google.com>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35491}
This was a remenant leftover from a previous design, which was no longer
valid after the switch to TaskQueues. ReturnReason::kStopped was not
used at all, and so Timeout or FrameFound can be inferred from whether
the frame is null or not.
Bug: webrtc:13343, webrtc:13346
Change-Id: Ib0f847b1e1192e32ea11208e48f5a3892703521e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239651
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35490}
When a frame is assembled `packet_infos` is moved and must be
re-initialized before potentially being used in another iteration of the
loop. Clear `packet_infos` immediately instead of relying on it being
implicitly cleared in the next iteration of the loop.
Bug: None
Change-Id: I954aaa0c6df296cc2a27b3ab496e49fac200f135
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238981
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Emil Lundmark <lndmrk@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35441}
This test being flaky impeded the landing of
https://webrtc-review.googlesource.com/c/src/+/239126. Fix by
ensuring the test's OnSendRtp guts don't execute past all streams
stopped.
Bug: None
Change-Id: Ie8aefb3bb03c09d2a9514acecd162e7c079c77c7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239363
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35432}
The encoders wrapped in VideoStreamEncoder grossly over-estimates
available bitrate when capture FPS falls close to zero, and frames
re-commence highly frequent delivery. Avoid this by moving the input
RateStatistics inside VSE into the frame cadence adapter, and changing
the reported framerate under zero-hertz encoding mode to always return
the configured max FPS.
Bug: chromium:1255737
Change-Id: Iaa71ef51c0755b12e24e435d86d9562122ed494e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239126
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35431}
This change moves the responsibility of posting
EncoderSwitchRequestCallback calls closer to the top-level
users which has a better idea about threading requirements.
The change is planned to be followed-up with more changes removing
the need for VSE to post to the worker thread.
Bug: webrtc:13414, chromium:1255737
Change-Id: I57a2962a70e9f245460c59c0d61824371394b952
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238420
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35387}
This change switches the sequence used by the FrameCadenceAdapter
to be the encoder_queue, enabling VideoStreamEncoder::OnFrame to be
invoked directly on the encoder_queue and eliminates the contained
PostTasks.
Bug: chromium:1255737
Change-Id: Ib86fc96ad2be9a38585fef2535855e3f9cc7e57c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238171
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35380}
Add implementation of RTC_DCHECK_NOTREACHED equal to the RTC_NOTREACHED.
The new macros will replace the old one when old one's usage will be
removed. The idea of the renaming to provide a clear signal that this
is debug build only macros and will be stripped in the production build.
Bug: webrtc:9065
Change-Id: I4c35d8b03e74a4b3fd1ae75dba2f9c05643101db
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237802
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35348}
This CL set the spatial id in LibaomAv1Encoder and set correct number
of spatial layers for AV1 in FrameEncodeMetadataWriter.
Bug: None
Change-Id: I40092e45be88ec9ab75f228d9ca84c44e3cad326
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237662
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Zhaoliang Ma <zhaoliang.ma@intel.com>
Cr-Commit-Position: refs/heads/main@{#35339}
VideoStreamEncoder receives frames on an undefined threading
context with the only requirement being that frames are serially
arriving. This CL changes this to post all frames arriving at the
FrameCadenceAdapter to the worker thread before further
processing, transitively leading to frame entry into the
VideoStreamEncoder on the worker thread.
Bug: chromium:1255737
Change-Id: I04d69cb4a5048d671d2dcd3bd6d669fbcda52b3f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237142
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35320}
https://chromium-swarm.appspot.com/task?id=56dfaf30fa11e510
Due to recent changes, this test sometimes crashes because of
use after free.
Fix this by adding a PendingTaskSafetyFlag to not access `stream_`
after it has been deleted.
Bug: webrtc:13315, webrtc:13351
Change-Id: I7cb180bcab1d79b39737c53704c5fe8a2ca28b7e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236660
Commit-Queue: Byoungchan Lee <daniel.l@hpcnt.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35319}
The test has a ModuleRtpRtcpImpl2 which is destroyed long
after test transports are removed, leading to a UAF.
Fix by overriding OnStreamsStopped which is called before
transports are removed.
TESTED=Asan now passes 1000/1000, failed 4/1000 before.
Fixed: chromium:1235251, b:192567426
Change-Id: Ie9135685e81712e38c4b00355debfc67c1f603bd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237345
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35318}
This change introduces a new FrameCadenceAdapter class which takes the
role of being a VideoFrameSinkInterface<> instead of VideoStreamEncoder.
The FrameCadenceAdapter will see its functionality grow in future CLs
and eventually enable screenshare capture sources to have zero hertz as
the minimum capture frequency.
This CL moves logic related to UMA collection and constraints into the
adapter.
The adapter has two major modes. Future functionality is planned to be
added under the WebRTC-ZeroHertzScreenshare field trial. Unit tests are
added that verify passthrough operation when WebRTC-ZeroHertzScreenshare
isn't specified or disabled.
Just specifying the WebRTC-ZeroHertzScreenshare field trial isn't
enough to activate the feature, but the caller has to additionally
configure screen content type, minimum FPS 0, and maximum FPS > 0 for
the new mode.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I1799110ed40843152786ad80df10acfb83a608b1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236682
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35315}
The sequences of threads entering the VideoStreamEncoder has been
unclear. Fix this by renaming the uninformational |main_queue_| to
|worker_queue_|, and introduce a new |network_queue_| which is set
on construction.
Bug: chromium:1255737
Change-Id: Ic4d3a5b8188b8cc98e60b72aee2c09c9afbc7356
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236523
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35283}
The synchronously waiting SendTask() helper method should never be
called from within OnSendRtp() as that risks a deadlock with the
shutdown of the test.
This CL reverts an earlier disabling which did not correctly identify
the root cause.
Bug: webrtc:13291
Change-Id: Ia3c3417e0cbfb7011bb2439a52f976b946adad78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/235721
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35244}
This change adds a few UMAs to inform on the usage of frame
rate constraints related to screenshare sessions.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: Icdd011a8e7df837416d603beeb0866d9eb1918e7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/235368
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Johannes Kron <kron@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35238}
This change
- adds new type VideoTrackSourceConstraints expressing min/max FPS
constraints.
- adds new method VideoTrackSourceInterface::ProcessConstraints.
- adds new method VideoSinkInterface<>::OnConstraintsChanged.
- updates AdaptedVideoTrackSource and VideoBroadcaster to forward
the constraints to sinks.
- adds several unit tests for the added functionality.
- and finally, implements OnConstraintsChanged in VideoStreamEncoder.
Chromium will be updated in coming CLs to supply constraints set
through the MediaStream module.
go/rtc-0hz-present
Bug: chromium:1255737
No-Try: true
Change-Id: Iffef239217269c332a1aaa902ddeae2440929e22
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/235040
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35197}
Encoder info will be modified at runtime. In fact, we should reduce the
number of 'full' ReconfigureEncoder(). If only need subset of it at
runtime, consider handle it in VideoStreamEncoder::EncodeVideoFrame().
Consider two cases:
Re-configure scalers when encoder info changed. Consider two cases:
1. When the status of the scaler changes from enabled to disabled, if we
don't do this CL, scaler will adapt up/down to trigger an unnecessary
full ReconfigureEncoder() when the scaler should be banned.
2. When the status of the scaler changes from disabled to enabled, if we
don't do this CL, scaler will not work until some code trigger
ReconfigureEncoder(). In extreme cases, the scaler doesn't even work for
a long time when we expect that the scaler should work.
This CL aims to make scalers work properly when encoder info changed.
BUG: None
Change-Id: Iec17730b5fac5e642c0fb2d9b11c5b7434f0a220
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/233384
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35175}
