Some hardware H.264 encoders does not place average QP delta in
slice_qp_delta field. Adding an optional flag in EncoderInfo to notify
quality scaler about this.
Bug: webrtc:12942
Change-Id: I3ee29c5ae9bd7bb34d26eba7e6bede3798ca44b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226921
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34627}
This reverts commit 31c5c9da35209fe65ed15cb3a804823cd2789259.
Reason for revert: made QP parser thread-safe https://webrtc.googlesource.com/src/+/0e42cf703bd111fde235d06d08b02d3a7b02c727
Original change's description:
> Revert "Reland "Enable quality scaling when allowed""
>
> This reverts commit 0021fe77937f386e6021a5451e3b0d78d7950815.
>
> Reason for revert: Broken on linux_tsan bot: https://ci.chromium.org/ui/p/webrtc/builders/ci/Linux%20Tsan%20v2/25329/overview
>
> Original change's description:
> > Reland "Enable quality scaling when allowed"
> >
> > This reverts commit eb449a979bc561a8b256cca434e582f3889375e2.
> >
> > Reason for revert: Added QP parsing in https://webrtc.googlesource.com/src/+/8639673f0c098efc294a7593fa3bd98e28ab7508
> >
> > Original change's description:
> > Before this CL quality scaling was conditioned on scaling settings
> > provided by encoder. That should not be a requirement since encoder
> > may not be aware of quality scaling which is a WebRTC feature. In M90
> > chromium HW encoders do not provide scaling settings (chromium:1179020).
> > The default scaling settings provided by these encoders are not correct
> > (b/181537172).
> >
> > This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
> > is set to true in singlecast with normal video feed (not screen sharing)
> > mode. If quality scaling is allowed it is enabled no matter whether
> > scaling settings are present in encoder info or not. Setting from
> > QualityScalingExperiment are used in case if not provided by encoder.
> >
> > Bug: chromium:1179020
> > Bug: webrtc:12511
> > Change-Id: I97911fde9005ec25028a640a3f007d12f2bbc2e5
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211349
> > Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> > Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> > Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#33438}
>
> TBR=brandtr@webrtc.org,ilnik@webrtc.org,ssilkin@webrtc.org,rubber-stamper@appspot.gserviceaccount.com
>
> Change-Id: Id7633a1e98f95762e81487887f83a0c35f89195c
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: chromium:1179020
> Bug: webrtc:12511
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211352
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33439}
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: chromium:1179020
Bug: webrtc:12511
Change-Id: I3a31e1c1fdf7da93226f8c1e0a96b43fe0b786df
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212026
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33481}
This reverts commit 0021fe77937f386e6021a5451e3b0d78d7950815.
Reason for revert: Broken on linux_tsan bot: https://ci.chromium.org/ui/p/webrtc/builders/ci/Linux%20Tsan%20v2/25329/overview
Original change's description:
> Reland "Enable quality scaling when allowed"
>
> This reverts commit eb449a979bc561a8b256cca434e582f3889375e2.
>
> Reason for revert: Added QP parsing in https://webrtc.googlesource.com/src/+/8639673f0c098efc294a7593fa3bd98e28ab7508
>
> Original change's description:
> Before this CL quality scaling was conditioned on scaling settings
> provided by encoder. That should not be a requirement since encoder
> may not be aware of quality scaling which is a WebRTC feature. In M90
> chromium HW encoders do not provide scaling settings (chromium:1179020).
> The default scaling settings provided by these encoders are not correct
> (b/181537172).
>
> This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
> is set to true in singlecast with normal video feed (not screen sharing)
> mode. If quality scaling is allowed it is enabled no matter whether
> scaling settings are present in encoder info or not. Setting from
> QualityScalingExperiment are used in case if not provided by encoder.
>
> Bug: chromium:1179020
> Bug: webrtc:12511
> Change-Id: I97911fde9005ec25028a640a3f007d12f2bbc2e5
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211349
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33438}
TBR=brandtr@webrtc.org,ilnik@webrtc.org,ssilkin@webrtc.org,rubber-stamper@appspot.gserviceaccount.com
Change-Id: Id7633a1e98f95762e81487887f83a0c35f89195c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1179020
Bug: webrtc:12511
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211352
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33439}
This reverts commit eb449a979bc561a8b256cca434e582f3889375e2.
Reason for revert: Added QP parsing in https://webrtc.googlesource.com/src/+/8639673f0c098efc294a7593fa3bd98e28ab7508
Original change's description:
Before this CL quality scaling was conditioned on scaling settings
provided by encoder. That should not be a requirement since encoder
may not be aware of quality scaling which is a WebRTC feature. In M90
chromium HW encoders do not provide scaling settings (chromium:1179020).
The default scaling settings provided by these encoders are not correct
(b/181537172).
This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
is set to true in singlecast with normal video feed (not screen sharing)
mode. If quality scaling is allowed it is enabled no matter whether
scaling settings are present in encoder info or not. Setting from
QualityScalingExperiment are used in case if not provided by encoder.
Bug: chromium:1179020
Bug: webrtc:12511
Change-Id: I97911fde9005ec25028a640a3f007d12f2bbc2e5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211349
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33438}
This reverts commit 83be84bb74133343358bba22e4e5106ecc385721.
Reason for revert: Suspect of crbug.com/1185276
Original change's description:
> Reland "Enable quality scaling when allowed"
>
> This reverts commit 609b524dd3ff36719b5c4470b85d37dcdadfb1f8.
>
> Reason for revert: Disable QualityScalingAllowed_QualityScalingEnabled on iOS.
>
> Original change's description:
> Before this CL quality scaling was conditioned on scaling settings
> provided by encoder. That should not be a requirement since encoder
> may not be aware of quality scaling which is a WebRTC feature. In M90
> chromium HW encoders do not provide scaling settings (chromium:1179020).
> The default scaling settings provided by these encoders are not correct
> (b/181537172).
>
> This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
> is set to true in singlecast with normal video feed (not screen sharing)
> mode. If quality scaling is allowed it is enabled no matter whether
> scaling settings are present in encoder info or not. Setting from
> QualityScalingExperiment are used in case if not provided by encoder.
>
> Bug: chromium:1179020
> Bug: webrtc:12511
> Change-Id: Ia0923e5a62acdfdeb06f9aad5d670be8a0f8d746
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209643
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33385}
Bug: chromium:1179020
Bug: webrtc:12511
Change-Id: I7004014c5936176f8c125aeb55da91ce095b266e
TBR: ssilkin@webrtc.org
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209708
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Commit-Queue: Guido Urdaneta <guidou@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33393}
It read from simulcast_layers[0] even if vector is empty.
Bug: none
Change-Id: I293890feda70022beae4247ab10cf8182b4cf4a5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209706
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33391}
This reverts commit 609b524dd3ff36719b5c4470b85d37dcdadfb1f8.
Reason for revert: Disable QualityScalingAllowed_QualityScalingEnabled on iOS.
Original change's description:
Before this CL quality scaling was conditioned on scaling settings
provided by encoder. That should not be a requirement since encoder
may not be aware of quality scaling which is a WebRTC feature. In M90
chromium HW encoders do not provide scaling settings (chromium:1179020).
The default scaling settings provided by these encoders are not correct
(b/181537172).
This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
is set to true in singlecast with normal video feed (not screen sharing)
mode. If quality scaling is allowed it is enabled no matter whether
scaling settings are present in encoder info or not. Setting from
QualityScalingExperiment are used in case if not provided by encoder.
Bug: chromium:1179020
Bug: webrtc:12511
Change-Id: Ia0923e5a62acdfdeb06f9aad5d670be8a0f8d746
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209643
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33385}
This reverts commit 752cbaba907de077e5f1b24a232e71feb479dccb.
Reason for revert: The test VideoStreamEncoderTest.QualityScalingAllowed_QualityScalingEnabled seems to fail on iOS.
Original change's description:
> Enable quality scaling when allowed
>
> Before this CL quality scaling was conditioned on scaling settings
> provided by encoder. That should not be a requirement since encoder
> may not be aware of quality scaling which is a WebRTC feature. In M90
> chromium HW encoders do not provide scaling settings (chromium:1179020).
> The default scaling settings provided by these encoders are not correct
> (b/181537172).
>
> This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
> is set to true in singlecast with normal video feed (not screen sharing)
> mode. If quality scaling is allowed it is enabled no matter whether
> scaling settings are present in encoder info or not. Setting from
> QualityScalingExperiment are used in case if not provided by encoder.
>
> Bug: chromium:1179020, webrtc:12511
> Change-Id: I83d55319ce4b9f4fb143187ced94a16a778b4de3
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209184
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33373}
Bug: chromium:1179020
Bug: webrtc:12511
Change-Id: Icabf2d9a034d359f79491f2c37f1044f17a7445d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209641
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Björn Terelius <terelius@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33381}
Before this CL quality scaling was conditioned on scaling settings
provided by encoder. That should not be a requirement since encoder
may not be aware of quality scaling which is a WebRTC feature. In M90
chromium HW encoders do not provide scaling settings (chromium:1179020).
The default scaling settings provided by these encoders are not correct
(b/181537172).
This CL adds is_quality_scaling_allowed to VideoEncoderConfig. The flag
is set to true in singlecast with normal video feed (not screen sharing)
mode. If quality scaling is allowed it is enabled no matter whether
scaling settings are present in encoder info or not. Setting from
QualityScalingExperiment are used in case if not provided by encoder.
Bug: chromium:1179020, webrtc:12511
Change-Id: I83d55319ce4b9f4fb143187ced94a16a778b4de3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209184
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33373}
Simulcast with one active stream:
Use pixels from single active stream if set (instead of input stream which could be larger) to avoid going below the min_pixel_per_frame limit when downgrading resolution.
Bug: none
Change-Id: I65acb12cc53e46f726ccb5bfab8ce08ff0c4cf78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208101
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33309}
Use bandwidth allocation instead of encoder target bitrate in DropDueToSize when incoming resolution increases to avoid downgrades due to target bitrate being limited by the max bitrate at low resolutions.
Bug: none
Change-Id: Ic41b31c1a86911d4e97b61b0cbc41ce0da739bd4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205622
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33168}
This could potentially lead to unnecessary restarts since it is also
started after the encoder is created. However, it is needed since the
hardware acceleration support can change even though the encoder has
not been recreated.
Bug: b/145730598
Change-Id: Iad1330e7c7bdf769a68c4ecf7abb6abbf3a4fe71
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203140
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33060}
This CL implements a Resource that aggressively reports overuse or
underuse until the encoded stream has the max pixels specified. The
pixel limit is controlled with a field trial, e.g:
--force-fieldtrials="WebRTC-PixelLimitResource/Enabled-307200/"
This caps the resolution to 307200 (=640x480). This can be used by the
TestBed to simulate being CPU limited. Note that the resource doesn't
care about degradation preference at the moment, so if the degradation
preference would be set to "maintain-resolution" the PixelLimitResource
would never stop reporting overuse and we would quickly get a low-FPS
stream.
PixelLimitResource runs a repeating task and reports overuse, underuse
or neither every 5 seconds. This ensures we quickly reach the desired
resolution.
Unit tests are added. I did not add any integration tests (I think
that's overkill for a testing-only resource) but I have manually
verified that this works as intended.
This CL also moves the FakeVideoStreamInputStateProvider into a test/
folder and exposes video_stream_adapter.cc's GetLowerResolutionThan().
Bug: webrtc:12261
Change-Id: Ifbf7c4c05e9dd2843543589bebef3f49b18c38c0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195600
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32771}
External reasons here are simulcast configuration and
source resolution change.
Initial frame dropper should be enabled in these cases because the
client can request way too big resolution for available bitrate and
usual quality scaling would take too long.
Bug: none
Change-Id: I02fbbd3c15b53b39672c083c2a1f9a780256c507
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195004
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32707}
And use it in a few places that were using RTC_CHECK(false) or FATAL()
to do the exact same job. There should be no change in behavior.
Bug: none
Change-Id: I36d5e6bcf35fd41534e08a8c879fa0811b4f1967
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/191963
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32567}
"warning: control reaches end of non-void function [-Wreturn-type]"
Reported by gcc (8.3)
In all the reported cases, the end of function is never actually
reached. Add RTC_CHECK(false) to ensure the compiler is aware that
this path is a dead-end.
Bug: webrtc:12008
Change-Id: I7f816fde3d1897ed2774057c7e05da66e1895e60
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/189784
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Fabien VALLÉE <fabien.vallee@netgem.com>
Cr-Commit-Position: refs/heads/master@{#32503}
This is a reland of ba8abbb630cdd9d05e22c830d0845e920762850d
This can be relanded as the queuing issues that were causing a
crash in the WebRTC roll in Chromium have been resolved. I have
added the Chromium failing targets to the CQ for this commit and
they have succeeded.
Original change's description:
> [Adaptation] Remove QualityScalerResource when disabled.
>
> Bug: webrtc:11843
> Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31924}
Bug: webrtc:11843
Change-Id: I228331293060ef996f1dd7f8e18d52b0818f526b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182080
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31996}
Resource adaptation needs refactoring for async adaptations. For now
the resource adaptation processor can work on the encoder thread, until
it is refactored to support async adaptation.
Bug: webrtc:11867
Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31991}
There was a small miss in https://webrtc-review.googlesource.com/c/src/+/181369
which allowed frame dropping to re-enable, and there was no test to catch this.
This case has been fixed along with a test to ensure this isn't missed in the future.
Bug: webrtc:11843
Change-Id: I201aa451d4751586c780a07dc72a7401aed78088
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181661
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31936}
Some restrictions previously were preventing adaptation up caused by the quality resource. However, it makes sense to use the same restrictions in the case of other resources. This CL removes now unneeded wire-up of reason/resource causing adaptation.
Bug: webrtc:11771
Change-Id: Iec301a59d2a41d32d23b6be340f3b5637d697e52
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181580
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31926}
Instead it should be ensured that it is started with the correct config.
This removes confusion regarding a resource state. If the resource
check is stopped then the adaptations for that resource should be
removed, and there is no way to determine that if we have one method for
stop for both reconfigure and shutdown.
Bug: webrtc:11843
Change-Id: I491f2fd1f4f803a4610124c7b0026ad75ab4a9cc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181368
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31913}
Since the degradation preference can be fetch from the thread safe
provider, this removed the need to have 2 queues.
Bug: None
Change-Id: I94cb8b7d3d2950acfe0ad0a6d46edc038ed8e286
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177523
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31898}
This way can double adapt right away instead of relying
on the qp scaler checking soon into the future.
Bug: webrtc:11830
Change-Id: I8e878168303cf6a4c3edcf3997dd8ac2413a4479
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181060
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31895}
It has no relevance in ResourceAdaptationProcessorInterface.
The logic moves to an interaction directly on the VideoStreamAdapter.
Bug: webrtc:11700
Change-Id: I4c7b3e1e1361722a69b71e8f9bde33f5909c011a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178380
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31634}
This patch creates a new class which provides the DegradationPreference
thread safe to all classes that need if (BalancedConstraint and
QpScalerResource). It also broadcasts to all listeners when the
preferences are updated, so the ResourceAdaptationProcessor can update
the video if needed.
In future work, this could be used to remove the need for two task
queues for the VideoStreamEncoder resources.
Bug: webrtc:11700
Change-Id: I05480db8b7157b5643f6f86ec9c64850839b3e76
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177522
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31623}
This moves this responsibility from the ResourceAdaptaitonProcessor
to the VideoStreamAdapter. A new interface for listening to adaptation
changes was added, and the ResourceAdaptationProcessor now listens on
the VideoStreamAdapter for those changes.
This means that going forward,
1. We can do adaptations outside of resource limitations, like setting
prior adaptations on a resource like initial frame dropper is designed
to.
2. Adaptations can be on a different queue than the
ResourceAdaptaitonProcessor's queue since updates are pushed through
the listener.
Bug: webrtc:11700
Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31615}
This introduces a new class for encapsulating the QualityRampupExperiment
R=hbos@webrtc.org
Bug: webrtc:11553
Change-Id: If2f2347cdcbd0c79821355f90e2d7ad3171143b5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176363
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31531}
A more detailed explaination is in the bug, but this changes
the way that adaptation happens when multiple resources are
limited. Only the one that is most limited can trigger an
adaptation up. If multiple resources are most limited both
need to underuse to adapt up.
Some of the changes in this patch to make it all work:
* VideoStreamEncoder unittests that did not reflect this
new behaviour have been changed.
* PeekNextRestrictions returns the adaptation counters as
well as the restrictions.
* Adaptation statstics have changed so that when adapting
up all resources are tagged as triggering the adaptation.
Additionally the statistics for the current adaptation is
now the total number of adaptations per reason, rather then
the number of adaptations due to that reason.
* PreventAdaptUpDueToActiveCounts is removed as most limited
resource is a strong implementation of that.
Bug: webrtc:11553
Change-Id: If1545a201c8e019598edf82657a1befde8b05268
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31497}
This is a prerequisite to implementing it externally.
Bug: webrtc:11525
Change-Id: I9cb3b4418396485d3eb9f25cafa51cbff6db7817
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176401
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31418}
IsAdaptationUpAllowed is moved from Resource to AdaptationConstraint.
OnAdaptationApplied is moved from Resource to AdaptationListener.
In a future CL, Resource will be moved to api/, but
AdaptationConstraint and AdaptationListener will stay in call/.
The processor, encode stream and manager are updated to keep track of
both resources, constraints and listeners. Fakes and tests are updated.
After this CL, the manager's inner classes that prevent adaptation
implement AdaptationConstraint instead of Resource.
Bug: webrtc:11525
Change-Id: Ie9cd5b1ba7d8e161951e131ab8f6bd9d5cf765bf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176368
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31409}
This CL is in preparation for moving Resource to the api/ folder. It
does not move it, but makes it such that the moving CL can be a pure
move.
In order to do this, we must stop depending on rtc_base/rtc::TaskQueue
in favor of api/webrtc::TaskQueueBase.
There are also other rtc_base/ dependencies that we do not want to
expose to the api/ folder, like critical sections and thread
annotations which are not publically exposed. To get around this, we
make Resource an abstract interface and move all of the base class
functionality into a new non-api/ class: VideoStreamEncoderResource.
The Resource now has Register/UnregisterAdaptationTaskQueue() methods.
By explicitly unregistering, we can ensure validity of the pointer even
if the Resource outlives the PeerConnection. While public interface
methods are only to be called on the adaptation task queue, posting to
the task queue happens off-queue, so a |lock_| is introduced to guard
it.
Bug: webrtc:11525
Change-Id: I50b3a30960cdec9032016c779b47001c01dad32f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176320
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31402}
This CL unblocks future Call-Level Mitigation strategies by moving the
ResourceAdaptationProcessor to a separate task queue. This signifies a
major milestone in the new resource adaptation architecture because
with this CL the threading model is in place and moving the Processor
to the Call and increasing its responsibilities is made possible.
In this CL, we still have one Processor per VideoStreamEncoder and the
VideoStreamEncoder is responsible for the creation and the destruction
of its Processor and that Processor's task queue. But the PostTasks are
in place and the decision-making is executed on a separate queue.
This CL:
- Moves ResourceAdaptationProcessor to an adaptation task queue.
It continues to be entirely single-threaded, but now operates on a
separate task queue.
- Makes Resources thread-safe: Interaction with the Processor, i.e.
OnResourceUsageStateMeasured() and IsAdaptationUpAllowed(), happens
on the adaptation task queue. State updates are pushed from the
encoder task queue with PostTasks.
- QualityScalerResource operates on both task queues; the QP usage
callbacks are invoked asynchronously.
- The VideoStreamEncoderResourceManager operates on the encoder task
queue with the following exceptions:
1) Its resources are accessible on any thread (using a mutex). This
is OK because resources are reference counted and thread safe.
This aids adding and removing resources to the Processor on the
adaptation task queue.
2) |active_counts_| is moved to the adaptation task queue. This makes
it possible for PreventAdaptUpDueToActiveCounts to run
IsAdaptationUpAllowed() on the adaptation task queue.
A side-effect of this is that some stats reporting now happen on
the adaptation task queue, but that is OK because
VideoStreamEncoderObserver is thread-safe.
The Manager is updated to take the new threading model into account:
- OnFrameDroppedDueToSize() posts to the adaptation task queue to
invoke the Processor.
- OnVideoSourceRestrictionsUpdated(), now invoked on the adaptation
task queue, updates |active_counts_| synchronously but posts to the
encoder task queue to update video source restrictions (which it
only uses to calculate target frame rate).
- MaybePerformQualityRampupExperiment() posts to the adaptation task
queue to maybe reset video source restrictions on the Processor.
|quality_rampup_done_| is made std::atomic.
Bug: webrtc:11542, webrtc:11520
Change-Id: I1cfd76e0cd42f006a6d2527f5aa2aeb5266ba6d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174441
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31231}
This CL makes the VideoStreamEncoderResourceManager's inner Resources
(PreventAdaptUpDueToActiveCounts,
PreventIncreaseResolutionDueToBitrateResource and
PreventAdaptUpInBalancedResource) not directly depend on any of the
manager's states that will continue to live on the encoder task queue
when the adaptation task queue is introduced in the next CL.
PreventAdaptUpDueToActiveCounts depends on effective degradation
preference, which it can get from the Processor, and the active counts,
which will move to the adaptation queue and is safe to use.
PreventIncreaseResolutionDueToBitrateResource depends on encoder
settings and target bitrate. This Resource now listens to these states
being updated, which may be implemented with a PostTask when the
adaptation queue is added.
PreventAdaptUpInBalancedResource depends on the effective degradation
preference, which it can get from the Processor; balanced settings,
which is a const readonly struct (thread-safe); and encoder target
bitrate, which it listens for being updated (to be PostTask'ed).
All resources depends on GetReasonFromResource() which will be callable
from the adaptation queue.
Bug: webrtc:11542, webrtc:11520
Change-Id: Ifa7bd87d9d8729988073f78f6a37c6f3b8aa4db1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174807
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31220}
In a future CL, adaptation processing and stream encoder resource
management will happen on different task queues. When this is the case,
asynchronous tasks will be posted in both directions and some resources
will have internal states used on multiple threads.
This CL makes the Resource class reference counted in order to support
posting tasks to a different threads without risk of use-after-free
when a posted task is executed with a delay. This is preferred over
WeakPtr strategies because WeakPtrs are single-threaded and preferred
over raw pointer usage because the reference counted approach enables
more compile-time and run-time assurance. This is also "future proof";
when resources can be injected through public APIs, ownership needs to
be shared between libwebrtc and the application (e.g. Chrome).
To reduce the risk of making mistakes in the future CL, sequence
checkers and task queue DCHECKs are added as well as other DCHECKs to
make sure things have been cleaned up before destruction, e.g:
- Processor gets a sequence checker. It is entirely single-threaded.
- Processor must not have any attached listeners or resources on
destruction.
- Resources must not have any listeners on destruction.
- The Manager, EncodeUsageResource and QualityScalerResource DCHECKs
they are running on the encoder queue.
- TODOs are added illustrating where we want to add PostTasks in the
future CL.
Lastly, upon VideoStreamEncoder::Stop() we delete the
ResourceAdaptationProcessor. Because the Processor is already used in
posted tasks, some if statements are added to ensure the Processor is
not used after destruction.
Bug: webrtc:11542, webrtc:11520
Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31217}
This replaces references to the ResourceAdaptationProcessor with
references to its interface. This would make it possible to have
alternative implementations or inject fake/mock implementations for
testing.
The VideoStreamAdapter is still responsible for constructing the
ResourceAdaptationProcessor, but beyond construction it is agnostic
towards the implementation.
With this CL, I claim https://crbug.com/webrtc/11222 complete.
TBR=ilnik@webrtc.org
Bug: webrtc:11222
Change-Id: I6e7a73bf1d0b5e97bc694f66180a747b27ffb018
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174160
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31148}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The ResourceListenerResponse was used to make the QualityScaler
not clear QP samples and instead increase its frequency of checking for
QP under certain circumstances, see enum description:
https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33
Because the QualityScaler depends on whether and how adaptation
happened it should listen to adaptation happening.
This CL moves the logic that was previously in VideoStreamAdapter closer
to the QualityScaler: QualityScalerResource::OnAdaptationApplied().
This would allow the VideoStreamAdapter to operate on a separate task
queue in the future, with no dependencies on any stream-specific
resources that might operate on other task queues.
Bug: webrtc:11172, webrtc:11521
Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31143}
