This CL sets speed 9 for all resolutions when two or less cores are
available, as a heuristic for a "slow" machine.
This gives a large speed bost at a relatively small quality loss.
A field-trial kill-switch is available to override this behavior.
Bug: webrtc:13888
Change-Id: I24278a45de000ad7984d0525c47d9eb6b9ab6b60
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/257421
Reviewed-by: Emil Lundmark <lndmrk@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36466}
convert almost all of video/ (and the collateral)
Bug: webrtc:10335
Change-Id: Ic94e05937f54d11ee8a635b6b66fd146962d9f11
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254601
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36192}
Some of the state that's managed in VideoStreamEncoder, is updated
and accessed on the encoder queue, but a method that's used for testing
only (GetAdaptationResources()), represents a race between PostTask
operations that update state and checking for said state on the worker
thread.
This CL removes Wait() operations related to adaptation resources from
the common path and puts one in the test path instead.
Bug: webrtc:13612
Change-Id: Ie3e018e815e24951bc0634ed70de17eaf336a508
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/249220
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35797}
Currently if encoder initialization fails WebRTC doesn't send any video.
This CL adds functionality that changes encoder type in such case and
restores the video. If encoder selector is available we switch to
encoder it recommends. Otherwise, VP8 is used as the default fallback
encoder.
Bug: webrtc:13572
Change-Id: Ifcdf707a575711f5ff81f9451caf30140c9171dc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/246960
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35761}
The frame cadence adapter ignores key frame processing that happens
before the point where zero-hertz mode is activated, which leads to
no refresh frame requests if the key frame request comes too early.
Fix this to register a pending refresh frame request that gets
serviced when zero-hertz mode is activated. The CL changes the
FrameCadenceAdapterInterface::ProcessKeyFrameRequest from returning
whether to request a refresh frame into the frame cadence adapter
actively doing so itself via a new Callback::RequestRefreshFrame
API.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I53c2dbf6468e883eb2a2e81498e7134b1b35c336
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242963
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35598}
The encoders wrapped in VideoStreamEncoder grossly over-estimates
available bitrate when capture FPS falls close to zero, and frames
re-commence highly frequent delivery. Avoid this by moving the input
RateStatistics inside VSE into the frame cadence adapter, and changing
the reported framerate under zero-hertz encoding mode to always return
the configured max FPS.
Bug: chromium:1255737
Change-Id: Iaa71ef51c0755b12e24e435d86d9562122ed494e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/239126
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35431}
This change moves the responsibility of posting
EncoderSwitchRequestCallback calls closer to the top-level
users which has a better idea about threading requirements.
The change is planned to be followed-up with more changes removing
the need for VSE to post to the worker thread.
Bug: webrtc:13414, chromium:1255737
Change-Id: I57a2962a70e9f245460c59c0d61824371394b952
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238420
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35387}
This change switches the sequence used by the FrameCadenceAdapter
to be the encoder_queue, enabling VideoStreamEncoder::OnFrame to be
invoked directly on the encoder_queue and eliminates the contained
PostTasks.
Bug: chromium:1255737
Change-Id: Ib86fc96ad2be9a38585fef2535855e3f9cc7e57c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238171
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35380}
VideoStreamEncoder receives frames on an undefined threading
context with the only requirement being that frames are serially
arriving. This CL changes this to post all frames arriving at the
FrameCadenceAdapter to the worker thread before further
processing, transitively leading to frame entry into the
VideoStreamEncoder on the worker thread.
Bug: chromium:1255737
Change-Id: I04d69cb4a5048d671d2dcd3bd6d669fbcda52b3f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237142
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35320}
This change introduces a new FrameCadenceAdapter class which takes the
role of being a VideoFrameSinkInterface<> instead of VideoStreamEncoder.
The FrameCadenceAdapter will see its functionality grow in future CLs
and eventually enable screenshare capture sources to have zero hertz as
the minimum capture frequency.
This CL moves logic related to UMA collection and constraints into the
adapter.
The adapter has two major modes. Future functionality is planned to be
added under the WebRTC-ZeroHertzScreenshare field trial. Unit tests are
added that verify passthrough operation when WebRTC-ZeroHertzScreenshare
isn't specified or disabled.
Just specifying the WebRTC-ZeroHertzScreenshare field trial isn't
enough to activate the feature, but the caller has to additionally
configure screen content type, minimum FPS 0, and maximum FPS > 0 for
the new mode.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: I1799110ed40843152786ad80df10acfb83a608b1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236682
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35315}
The sequences of threads entering the VideoStreamEncoder has been
unclear. Fix this by renaming the uninformational |main_queue_| to
|worker_queue_|, and introduce a new |network_queue_| which is set
on construction.
Bug: chromium:1255737
Change-Id: Ic4d3a5b8188b8cc98e60b72aee2c09c9afbc7356
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236523
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35283}
This change adds a few UMAs to inform on the usage of frame
rate constraints related to screenshare sessions.
go/rtc-0hz-present
Bug: chromium:1255737
Change-Id: Icdd011a8e7df837416d603beeb0866d9eb1918e7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/235368
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Johannes Kron <kron@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35238}
This change
- adds new type VideoTrackSourceConstraints expressing min/max FPS
constraints.
- adds new method VideoTrackSourceInterface::ProcessConstraints.
- adds new method VideoSinkInterface<>::OnConstraintsChanged.
- updates AdaptedVideoTrackSource and VideoBroadcaster to forward
the constraints to sinks.
- adds several unit tests for the added functionality.
- and finally, implements OnConstraintsChanged in VideoStreamEncoder.
Chromium will be updated in coming CLs to supply constraints set
through the MediaStream module.
go/rtc-0hz-present
Bug: chromium:1255737
No-Try: true
Change-Id: Iffef239217269c332a1aaa902ddeae2440929e22
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/235040
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35197}
It turned out that the negotiated rtp header extensions are not fully known in WebRtcVideoChannel::AddSendStream.
The cl also remove the unnecessary factory for creating VideoStreamEncoder.
Bug: webrtc:12000
Change-Id: If994c8deb69f3ce4212896d3ad757dac94c6e09f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198840
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32916}
VideoBitrateAllocation is instead reported through the EncoderSink.
Enable VideoBitrateAllocation reporting from WebRtcVideoChannel::AddSendStream in preparation for
using the extension RtpVideoLayersAllocationExtension instead of RTCP XR.
Bug: webrtc:12000
Change-Id: I5ea8e4f237a1c4e84a89cbfd97ac4353d4c2984f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186940
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32347}
Original description
Move reporting of target bitrate to just after the encoder has been
updated. Originall submitted as refs/heads/master@{#32275}
Patch 1 contains the original cl
,patch 2 the fix to send rtcp even if BWE does not change.
Bug: webrtc:12000
Change-Id: I16766e08229fe1f6f65f449e0e074bed03338693
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186948
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32340}
This reverts commit 39a31afb77e3ce5c4ff53b8bab06364712cae7ce.
Reason for revert: Will cause RTCP Target bitrate messages to not be sent unless BWE changes.
Original change's description:
> Refactor reporting of VideoBitrateAllocation
>
> Move reporting of target bitrate to just after the encoder has been
> updated.
>
> Bug: webrtc:12000
> Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32275}
TBR=sprang@webrtc.org,perkj@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:12000
Change-Id: Icf21e6ae28dc17c61b9243c037ffef9b623894ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186945
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32337}
Move reporting of target bitrate to just after the encoder has been
updated.
Bug: webrtc:12000
Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32275}
RequestEncoderFallback, RequestEncoderSwitch and
SetVideoCodecSwitchingEnabledRequest are now all called on the
worker thread. Before, the work already happened on that thread but
WebRtcVideoChannel adapted internally when needed.
With this CL, there are thread checks to make sure that these calls are
always made the same way, we don't need the async invoker and there
are fewer calls out from the encoder thread in VideoStreamEncoder
(reducing the chance of unintentional blocking).
Bug: webrtc:11908
Change-Id: If8738bc2a708a0fefc6fe850b32655f049f30bdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184603
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32151}
This turned out to be a bit complicated, mostly
related to the tests, but here's what's changed:
* No AsyncInvoker (and avoid ClearInternal) in
WebRtcVideoSendStream (WVSS)
* The reason it was there is due to a "design leak" from
VideoSourceSinkController/VideoStreamEncoder where the former uses
locks in all methods and is unaware of a threading model. That design
affected downstream objects, pushed the need for an async hop into
WVSS and added a lock.
A suggestion was made to address this in a follow-up change, here:
https://webrtc-review.googlesource.com/c/src/+/165684
* All methods in VideoSourceSinkController are now called on a known
and checked sequence and this CL removes the lock. This also makes
checking state consistent (i.e. calling a getter twice in a row on the
same sequence, will always return the same value, avoiding race with
other threads).
* Handling of reporting state changes from the encoder queue to the
VSSC, is done by VideoStreamEncoder.
* VideoSendStreamImpl is still instantiated on the incorrect thread [1]
but has two initialization steps [2]. The second one already runs on
the right thread. Addressing that TODO [1] is something we should do
but it has side effects to consider. For the purposes of this CL
the steps relating to the encoder (setting the sink pointer) have
been moved to [2].
[1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94
[2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115
Bug: webrtc:11222, webrtc:11908
Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32150}
This is a reland of ba8abbb630cdd9d05e22c830d0845e920762850d
This can be relanded as the queuing issues that were causing a
crash in the WebRTC roll in Chromium have been resolved. I have
added the Chromium failing targets to the CQ for this commit and
they have succeeded.
Original change's description:
> [Adaptation] Remove QualityScalerResource when disabled.
>
> Bug: webrtc:11843
> Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31924}
Bug: webrtc:11843
Change-Id: I228331293060ef996f1dd7f8e18d52b0818f526b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182080
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31996}
Resource adaptation needs refactoring for async adaptations. For now
the resource adaptation processor can work on the encoder thread, until
it is refactored to support async adaptation.
Bug: webrtc:11867
Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31991}
It was not used by any class and all future uses can use the
VideoSourceRestrictionsListener.
Bug: webrtc:11834
Change-Id: I5c71b93cc503f458dce0ccdd78b91b5a1debc56d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181062
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31896}
This is one less dependency on the task queue, and will make
things like removing resources and cleanup much easier in the future.
Bug: webrtc:11754
Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31686}
This is a step needed for multi-stream and new mitigations. It also
cleans up needing to signal adaptation changes in mutiple places
from ResourceAdaptationProcessor.
R=hbos@webrtc.org
Bug: webrtc:11754
Change-Id: Ib185dc9f66fbb4a087eb9e970c68c3f47eafb17f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178874
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31684}
This patch creates a new class which provides the DegradationPreference
thread safe to all classes that need if (BalancedConstraint and
QpScalerResource). It also broadcasts to all listeners when the
preferences are updated, so the ResourceAdaptationProcessor can update
the video if needed.
In future work, this could be used to remove the need for two task
queues for the VideoStreamEncoder resources.
Bug: webrtc:11700
Change-Id: I05480db8b7157b5643f6f86ec9c64850839b3e76
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177522
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31623}
This moves this responsibility from the ResourceAdaptaitonProcessor
to the VideoStreamAdapter. A new interface for listening to adaptation
changes was added, and the ResourceAdaptationProcessor now listens on
the VideoStreamAdapter for those changes.
This means that going forward,
1. We can do adaptations outside of resource limitations, like setting
prior adaptations on a resource like initial frame dropper is designed
to.
2. Adaptations can be on a different queue than the
ResourceAdaptaitonProcessor's queue since updates are pushed through
the listener.
Bug: webrtc:11700
Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31615}
This CL adds AddAdaptationResource to Call and
AddAdaptationResource/GetAdaptationResources method to relevant
VideoSendStream and VideoStreamEncoder interfaces and implementations.
Unittests are added to ensure that resources can be added to the Call
both before and after the creation of a VideoSendStream and that the
resources always gets added to the streams.
In a follow-up CL, we will continue to plumb the resources all the way
to PeerConnectionInterface, and an integration test will then be added
to ensure that injected resources are capable of triggering adaptation.
Bug: webrtc:11525
Change-Id: I499e9c23c3e359df943414d420b2e0ce2e9b2d56
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177002
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31499}
This is a prerequisite to implementing it externally.
Bug: webrtc:11525
Change-Id: I9cb3b4418396485d3eb9f25cafa51cbff6db7817
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176401
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31418}
IsAdaptationUpAllowed is moved from Resource to AdaptationConstraint.
OnAdaptationApplied is moved from Resource to AdaptationListener.
In a future CL, Resource will be moved to api/, but
AdaptationConstraint and AdaptationListener will stay in call/.
The processor, encode stream and manager are updated to keep track of
both resources, constraints and listeners. Fakes and tests are updated.
After this CL, the manager's inner classes that prevent adaptation
implement AdaptationConstraint instead of Resource.
Bug: webrtc:11525
Change-Id: Ie9cd5b1ba7d8e161951e131ab8f6bd9d5cf765bf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176368
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31409}
No code reads these properties.
They are recalculated by RtpPayloadParams
and set directly into RTPVideoHeader::generic.
Bug: webrtc:10342
Change-Id: Ib7bda4e6e5b0d0b3a585a848e3312fb66f8ae36d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/175127
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31269}
