2318 Commits

Author SHA1 Message Date
Danil Chapovalov
0acbb7745f Pass Environment into RtcpSender
To remove usage of RtcpConfiguration fields that are passed through Environment

Bug: webrtc:362762208
Change-Id: I1a0f218efe6a893c31ef2272cf2379c66fb7b205
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361746
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42984}
2024-09-09 13:44:21 +00:00
Henrik Boström
d4c5843bae Undo recent changes to initial frame dropper, fixing a regression.
A recent bugfix[1] introduced having to reconfigure the encoder in
response to restrictions updating for the sake of not getting stuck at
the wrong resolution in some cases.

But the newly added ReconfigureEncoder() calls happened too early/often,
and the initial frame dropper got confused thinking resolution changes
were not in response to adaptation (restrictions) when they in fact were.

The CL wrongly disabled the "reset initial frame dropper" logic, when
the correct solution to this problem should have been to delay the
ReconfigurationEncoder() call to the next frame (using
`pending_encoder_reconfiguration_ = true`).

With this delay the initial frame dropper is not confused and we can
restore the old "reset" logic, fixing the regression.

[1] https://webrtc-review.googlesource.com/c/src/+/360200

Bug: b/364252657
Change-Id: I6b93f4cc44eb12b1bbbda0d8d1e9906c29b615a2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361740
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42961}
2024-09-05 12:11:11 +00:00
Jakob Ivarsson
010c189f76 Move thread handling from source tracker.
This makes it simpler to use in more contexts.

Bug: b/364184684
Change-Id: I1b08ebd24e51ba1b3f85261eed503a78cd006fd8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361480
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Jakob Ivarsson‎ <jakobi@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42956}
2024-09-05 08:45:11 +00:00
Johannes Kron
8401f56a54 Add fieldtrials WebRTC-QCM-Static-{AV1, VP8, VP9}
The fieldtrials can be used to override the static QP threshold
that is used in QualityConvergenceMonitor to determine if an
encoded video stream has reached its target quality.

The fieldtrials do not change the dynamic detection.

Bug: chromium:328598314
Change-Id: I5995860eff461f0c712293e34cf75834ce414bed
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361201
Commit-Queue: Johannes Kron <kron@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42928}
2024-09-03 11:27:39 +00:00
Fanny Linderborg
6e072e6bfb Rename is_key_frame to communicate_upper_bits in FrameInstrumentation*Data
This is to make it clear that this field indicate whether the upper bits
of the sequence number should be communicated. However, the current
implementation only sets the field if it is a key frame.

Bug: webrtc:358039777
Change-Id: Ic2c8b6d91499e4e5cf25b8ce9591d326d7044fb0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361402
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42924}
2024-09-03 10:42:18 +00:00
Fanny Linderborg
5eb8588a65 Move FrameInstrumentation*Data structs to common_video
This enables usage in, e.g., `modules/rtp_rtcp`.

R=sprang@webrtc.org

Bug: webrtc:358039777
Change-Id: I0f8aafb75938d07f338bbf7acc4ff7c15a8b0baa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361285
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42920}
2024-09-03 07:20:06 +00:00
Florent Castelli
8037fc6ffa Migrate absl::optional to std::optional
Bug: webrtc:342905193
No-Try: True
Change-Id: Icc968be43b8830038ea9a1f5f604307220457807
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361021
Auto-Submit: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42911}
2024-09-02 12:16:47 +00:00
Danil Chapovalov
af7155e3ae Propagate Environment to video RtpRtcp modules
No-Iwyu: suggests too many changes, better address them separately.
Bug: webrtc:362762208
Change-Id: I1f97895109bda2b66eb864145f765ad3abb7de21
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/361144
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42906}
2024-09-02 10:11:24 +00:00
Danil Chapovalov
e2fee23271 Propagate Environment into RtpVideoStreamReceiver2
To make it available for constructing ModuleRtpRtcpImpl2

Bug: webrtc:362762208
Change-Id: Ic6ad339170c6aedb6c0bf42419964741d4d32bcc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/360921
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42888}
2024-08-29 20:10:45 +00:00
Henrik Boström
41fffaa6f4 Fix requested_resolution bug where we get stuck with old restrictions.
Normally (scaleResolutionDownBy) restrictions are applied at the source
which changes the input frame size which triggers reconfiguration with
appropriate scaling factors.

But when requested_resolution is used, encoder settings are by
definition not relative to the input frame size. In order for
restrictions to have an effect, they are applied inside
ReconfigureEncoder(): you get the minimum between the requested
resolution and the restricted resolution.

ReconfigureEncoder() happens when you SetParameters(), but the bug
here is that we don't do it again once the restrictions are updated.
So if restrictions are 540p when you ask for 720p, you get 540p and
after restrictions change to unlimited you're still stuck in 540p.

The fix is to also trigger ReconfigureEncoder() inside
OnVideoSourceRestrictionsUpdated() when the restricted resolution is
changing and a requested_resolution is configured.

To ensure reconfiguring the encoder "on the fly" like this does not
reset initial frame dropping logic, InitialFrameDropper caring about
input frame size changing is made conditional on not using
requested_resolution.

# Slow purple bots failing but they are not affected by this change.
NOTRY=True

Bug: webrtc:361477261
Change-Id: I1389aa16cf408b0d14e0b5b6f68c2442db955be9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/360200
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42882}
2024-08-29 12:26:17 +00:00
Emil Vardar
572280fe6a Remove redundant mapping.
It looks like the map `content_specific_stats_` is just carried over into `aggregated_stats` without doing any important processing. This seems to be redundant and hence, removing it in this CL.

Bug: None
Change-Id: Ia4a5de03999ecd33d387014928cba322bb11ee93
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/360745
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42870}
2024-08-28 11:44:43 +00:00
Emil Vardar
3f1e51d599 Aggregate and log corruption score.
Bug: webrtc:358039777
Change-Id: I4dade8e6daecf41e5b1f156416935c320f513d0b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359160
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Cr-Commit-Position: refs/heads/main@{#42867}
2024-08-27 13:50:35 +00:00
Fanny Linderborg
a6186b2485 Add helper that generate filter data given a captured and an encoded frame
R=sprang@webrtc.org

Bug: b/358039777
Change-Id: I48400db23b836d45f03cfa151aa1a9f1b8f00b2a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359940
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42807}
2024-08-19 13:04:30 +00:00
Fanny Linderborg
668e905513 Add support for determining which pixels to sample
Bug: b/358039777
Change-Id: If08f9fa46df4071674605a9feb95b73e71e9a291
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359245
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42800}
2024-08-19 09:43:31 +00:00
Johannes Kron
b47cd6fbe3 Ensure calls to QP convergence controller are on the same sequence
The original CL overlooked the possibility that the encoder may be
reconfigured in the middle of a stream.

Restructure the code so that all calls to QP convergence controller
happen on the encoder queue.

A side effect of this CL is that `EncodedImage::SetAtTargetQuality()`
is never called. The information is supplied to the frame cadence
adapter directly without this intermediate step.

`EncodedImage::SetAtTargetQuality()` and
`EncodedImage::IsAtTargetQuality()` are being marked as deprecated
in https://webrtc-review.googlesource.com/c/src/+/359660.

Bug: chromium:359410061
Change-Id: I941b5f60b1a9fd7694dbedf2f3e4ff5253ccf357
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359640
Commit-Queue: Johannes Kron <kron@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42788}
2024-08-16 06:54:05 +00:00
Fanny Linderborg
4fffeabba3 Add generic mapping functions for automatic corruption detection
The functions will be used to map a frame's QP to its optimal standard
deviation for the Gaussian kernel for the filter applied in the
automatic corruption detection algorithm.

Bug: b/358039777
Change-Id: Idb3b8cfdbd4a405151c660df87205e3949f9b085
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359380
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42777}
2024-08-14 07:31:02 +00:00
Dor Hen
1921fa5ea1 Apply include-cleaner to api/test/[^/]*
e.g all files in the api/test folder not including subdirectories

Bug: webrtc:42226242
Change-Id: I18d74a18f8feec41eb252faa9acfffd1d6f45ce4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/359420
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Dor Hen <dorhen@meta.com>
Cr-Commit-Position: refs/heads/main@{#42773}
2024-08-13 15:28:34 +00:00
Fanny Linderborg
c2a82417e9 Add support for the Halton sequence
Bug: b/358039777
Change-Id: Id191d584e22cf82384a5d16ed355f41c0bb32b7b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/358981
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42764}
2024-08-12 15:13:34 +00:00
Markus Handell
e864dec2e9 VSyncEncodeAdapterMode: avoid UAF.
This CL fixes a problem where VSEAM's `queue_` was dereferenced
post destruction. A sequence triggering it is:

0. FrameCadenceAdapter (FCA) is configured with Metronome with >= 34 ms tick delay.
1. A frame is queued for processing on worker queue.
2. The FCA is destroyed. The contained VSyncEncodeAdapterMode instance is scheduled for deletion on worker queue.
3. Encode queue is destroyed.
4. Worker queue is executed, which runs a task that dereferences `queue_`.

Bug: chromium:356423094
Change-Id: Iae8dc070304ef5ec0cfb0b4f27bbb7fe86e7def7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/358660
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42745}
2024-08-08 11:02:50 +00:00
Björn Terelius
8d7642a9f7 Remove unused QpFastFilterLow method
Bug: None
Change-Id: I63665a3fc9afd57aec8f0f7d2a2a2e631452f6c6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/358080
Auto-Submit: Björn Terelius <terelius@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42704}
2024-07-31 10:40:42 +00:00
Dan Tan
96c1b9c5ea Add variables to lend unused audio bits to video
This CL only adds variables necessary for the feature, which will be
implemented in later CLs.

Bug: webrtc:350555527
Change-Id: I71e56666e629f56168d316bf693150c0df0e2ecf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356740
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Dan Tan <dwtan@google.com>
Reviewed-by: Jakob Ivarsson‎ <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42698}
2024-07-30 18:42:16 +00:00
Tony Herre
5079e8a30a Allow supplying a custom NetworkControllerInterfaceFactory per-Call in PeerConnectionDependencies
This requires making CallConfig move-only so it can hold a unique_ptr to
the factory, but as discussed with Danil, that seems fine.

Bug: chromium:355610792
Change-Id: Ie52e33faaa4a2af748daeb25f5327b7a532936e2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357862
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Tony Herre <herre@google.com>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42679}
2024-07-29 07:17:14 +00:00
Jeremy Leconte
6ed136c6ae Fix TestExtendedReportsCanSignalZeroTargetBitrate flakiness on Mac.
The test is flaky on Mac on it seems related to a timeout.
https://luci-analysis.appspot.com/p/webrtc/clusters/testname-v4/82d0b764552f0811b37cc651c0962399?tab=recent-failures

../../video/end_to_end_tests/extended_reports_tests.cc:196: Failure
Value of: Wait()
  Actual: false
Expected: true
Timed out while waiting for RTCP SR/RR packets to be sent.

Change-Id: I9b19d3952a761415ab65d15f188ae3336e43e97e
Bug: None
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357820
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42672}
2024-07-25 09:16:44 +00:00
Danil Chapovalov
ac15a137ac In RtpVideoStreamReceiver do not rely on RTP sequence number unwrap to be stable
Currently this class assumed that if the same RTP sequence number is unwrapped again result would be the same.
That might not be true when several packets were inserted in between these two calls and unwrapper changed its state

This CL propose instead to unwrap once, and save the result in the intermediate struct.
To minimize the change and the risk, only redundant unwrapping is replaced to use unwrapped sequence number

Bug: webrtc:353565743
Change-Id: I8a18c8c206a0e16010951cabcf81dd9cb1588eda
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357660
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42662}
2024-07-22 15:42:12 +00:00
Sergey Silkin
1a33aa4a8e Override stream settings in a separate function
Bug: webrtc:351644568, b/352504711
Change-Id: I706d5a85b83603613693f63c5d3faa9946e90afc
No-Try: True
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357440
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42655}
2024-07-19 13:59:27 +00:00
Sergey Silkin
5fe85d23a2 Reland "Pass true stream resolutions to GetSimulcastConfig()"
This is a reland of commit 09f03be54804e81f626c26e8fde8c86cc952545f

Use max_num_layers instead of encoder_config.number_of_streams when calculation stream resolutions in EncoderStreamFactory::GetStreamResolutions().

Original change's description:
> Pass true stream resolutions to GetSimulcastConfig()
>
> Before this change GetSimulcastConfig() received only maximum resolution as an input parameter and derived resolutions for low quality simulcast streams assuming 1/2 scaling factor [1]. These days applications can configure resolution scaling factors via RtpEncodingParameters. If the configured resolution scaling factors were different from 1/2 then we got wrong bitrate limits from GetSimulcastConfig(). Now resolutions are calculated using scaling factor from RtpEncodingParameters (or default 1/2) for all streams in EncoderStreamFactory::CreateEncoderStreams() and then passed to GetSimulcastConfig().
>
> Moved tests from simulcast_unittest.cc to encoder_stream_factory_unittest.cc. Mapping of old to new tests:
> * GetConfigWithLimitedMaxLayersForResolution -> ReducesStreamCountWhenResolutionIsLow
> * GetConfigWithLowResolutionScreenshare -> ReducesLegacyScreencastStreamCountWhenResolutionIsLow
> * GetConfigWithNotLimitedMaxLayersForResolution -> KeepsStreamCountUnchangedWhenLegacyLimitIsDisabled
> * GetConfigWithNormalizedResolution -> AdjustsResolutionWhenUnaligned
> * GetConfigWithNormalizedResolutionDivisibleBy4 -> MakesResolutionDivisibleBy4
> * GetConfigWithNormalizedResolutionDivisibleBy8 -> not needed (MakesResolutionDivisibleBy4 should be enough).
> * GetConfigForLegacyLayerLimit -> KeepsStreamCountUnchangedWhenResolutionIsHigh and ReducesStreamCountWhenResolutionIsLow
> * GetConfigForLegacyLayerLimitWithRequiredHD -> KeepsStreamCountUnchangedWhenLegacyLimitIsDisabled
>
> [1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/video/config/simulcast.cc;l=297-298;drc=1b78a7eb3f418460da03672b1d1af1d9488bb544
>
> Bug: webrtc:351644568, b/352504711
> Change-Id: I0028904ab0bb1e27b9c1b7cd3fb9a8ccf447fa35
> No-Try: true
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357280
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#42651}

Bug: webrtc:351644568, b/352504711
Change-Id: Ib3fd859257b61c2a5d695b8b8f45c95495117c0e
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357520
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42654}
2024-07-19 13:12:59 +00:00
Sergey Silkin
ede05c35e4 Revert "Pass true stream resolutions to GetSimulcastConfig()"
This reverts commit 09f03be54804e81f626c26e8fde8c86cc952545f.

Reason for revert: breaks downstream projects

Original change's description:
> Pass true stream resolutions to GetSimulcastConfig()
>
> Before this change GetSimulcastConfig() received only maximum resolution as an input parameter and derived resolutions for low quality simulcast streams assuming 1/2 scaling factor [1]. These days applications can configure resolution scaling factors via RtpEncodingParameters. If the configured resolution scaling factors were different from 1/2 then we got wrong bitrate limits from GetSimulcastConfig(). Now resolutions are calculated using scaling factor from RtpEncodingParameters (or default 1/2) for all streams in EncoderStreamFactory::CreateEncoderStreams() and then passed to GetSimulcastConfig().
>
> Moved tests from simulcast_unittest.cc to encoder_stream_factory_unittest.cc. Mapping of old to new tests:
> * GetConfigWithLimitedMaxLayersForResolution -> ReducesStreamCountWhenResolutionIsLow
> * GetConfigWithLowResolutionScreenshare -> ReducesLegacyScreencastStreamCountWhenResolutionIsLow
> * GetConfigWithNotLimitedMaxLayersForResolution -> KeepsStreamCountUnchangedWhenLegacyLimitIsDisabled
> * GetConfigWithNormalizedResolution -> AdjustsResolutionWhenUnaligned
> * GetConfigWithNormalizedResolutionDivisibleBy4 -> MakesResolutionDivisibleBy4
> * GetConfigWithNormalizedResolutionDivisibleBy8 -> not needed (MakesResolutionDivisibleBy4 should be enough).
> * GetConfigForLegacyLayerLimit -> KeepsStreamCountUnchangedWhenResolutionIsHigh and ReducesStreamCountWhenResolutionIsLow
> * GetConfigForLegacyLayerLimitWithRequiredHD -> KeepsStreamCountUnchangedWhenLegacyLimitIsDisabled
>
> [1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/video/config/simulcast.cc;l=297-298;drc=1b78a7eb3f418460da03672b1d1af1d9488bb544
>
> Bug: webrtc:351644568, b/352504711
> Change-Id: I0028904ab0bb1e27b9c1b7cd3fb9a8ccf447fa35
> No-Try: true
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357280
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#42651}

Bug: webrtc:351644568, b/352504711
Change-Id: I7aadbe49419b7ac610db4db99284fdcdce9deff5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357500
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42653}
2024-07-19 09:46:34 +00:00
Sergey Silkin
09f03be548 Pass true stream resolutions to GetSimulcastConfig()
Before this change GetSimulcastConfig() received only maximum resolution as an input parameter and derived resolutions for low quality simulcast streams assuming 1/2 scaling factor [1]. These days applications can configure resolution scaling factors via RtpEncodingParameters. If the configured resolution scaling factors were different from 1/2 then we got wrong bitrate limits from GetSimulcastConfig(). Now resolutions are calculated using scaling factor from RtpEncodingParameters (or default 1/2) for all streams in EncoderStreamFactory::CreateEncoderStreams() and then passed to GetSimulcastConfig().

Moved tests from simulcast_unittest.cc to encoder_stream_factory_unittest.cc. Mapping of old to new tests:
* GetConfigWithLimitedMaxLayersForResolution -> ReducesStreamCountWhenResolutionIsLow
* GetConfigWithLowResolutionScreenshare -> ReducesLegacyScreencastStreamCountWhenResolutionIsLow
* GetConfigWithNotLimitedMaxLayersForResolution -> KeepsStreamCountUnchangedWhenLegacyLimitIsDisabled
* GetConfigWithNormalizedResolution -> AdjustsResolutionWhenUnaligned
* GetConfigWithNormalizedResolutionDivisibleBy4 -> MakesResolutionDivisibleBy4
* GetConfigWithNormalizedResolutionDivisibleBy8 -> not needed (MakesResolutionDivisibleBy4 should be enough).
* GetConfigForLegacyLayerLimit -> KeepsStreamCountUnchangedWhenResolutionIsHigh and ReducesStreamCountWhenResolutionIsLow
* GetConfigForLegacyLayerLimitWithRequiredHD -> KeepsStreamCountUnchangedWhenLegacyLimitIsDisabled

[1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/video/config/simulcast.cc;l=297-298;drc=1b78a7eb3f418460da03672b1d1af1d9488bb544

Bug: webrtc:351644568, b/352504711
Change-Id: I0028904ab0bb1e27b9c1b7cd3fb9a8ccf447fa35
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357280
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42651}
2024-07-19 08:47:09 +00:00
Sergey Silkin
b27ac6bc83 Set min bitrate equal to kDefaultMinVideoBitrateBps
If experimental min bitrate value is not configured, set the min bitrate for the first simulcast stream equal to kDefaultMinVideoBitrateBps (=30kbps).

Min bitrate depends on resolution. At absence of the experimental min bitrate override, we got high min bitrate values for high resolutions (600kbps for VP8 720p, for example) before. That led to encode pauses [1] which is an undesired behavior.

[1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/modules/video_coding/utility/simulcast_rate_allocator.cc;l=173;drc=f317f7106a7a15a04da7cd30c2e2ddb1b3025bc6

Bug: webrtc:351644568, b/352504711
Change-Id: Ifc93cc230fb194d2c9a739368d415f24385939fd
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357420
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42649}
2024-07-19 06:54:36 +00:00
Sergey Silkin
1b78a7eb3f Remove dependency on WebRTC-LowresSimulcastBitrateInterpolation
Dependency on WebRTC-LowresSimulcastBitrateInterpolation field trial in LimitSimulcastLayerCount() is unnecessary.

Bug: webrtc:351644568, b/352504711
Change-Id: I9daf9cbfb5b6a582cd9f03ce1a86e5bbd2b2bfd4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357260
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42644}
2024-07-17 07:51:33 +00:00
Johannes Kron
f6a804826c Add QualityConvergenceController to VideoStreamEncoder
QualityConvergenceController is a layer between VideoStreamEncoder
and QualityConvergenceMonitor that takes care of the simulcast
logic.

Bug: chromium:328598314
Change-Id: Iad8a9d9138e69a60fd508a7ef038220947888f0a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356420
Commit-Queue: Johannes Kron <kron@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42642}
2024-07-16 10:20:27 +00:00
Sergey Silkin
ea615affcc Remove WebRTC-VP8ConferenceTemporalLayers field trial
WebRTC-VP8ConferenceTemporalLayers experiment is restricted to <= M126. Number of temporal layers is controlled via scalaiblity mode now.

Bug: webrtc:351644568, b/352504711,  chromium:40097057, b/140159553
Change-Id: I025f8f64e8d5144cf54fe8bf26e8b99daae6e079
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357104
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42637}
2024-07-15 10:26:18 +00:00
Sergey Silkin
c14e2cc4ca Set is_highest_layer_max_bitrate_configured outside of loop
Bug: webrtc:351644568, b/352504711
Change-Id: Ia1798e35adf8b34357103ae3aba8ab16499a458f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357081
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42636}
2024-07-15 10:25:14 +00:00
Sergey Silkin
108c94b1d4 Do not expose GetNormalSimulcastLayers and GetScreenshareLayers
This is a cleanup of simulcast.cc. Remove GetNormalSimulcastLayers and GetScreenshareLayers from simulcast.h. Move the implementations to anonymous namespace in simulcast.cc.

Bug: webrtc:351644568, b/352504711
Change-Id: Iff03161e5c44cb0e7faa60b16cfc2fc9b903d5ce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357103
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42635}
2024-07-15 10:24:12 +00:00
Sergey Silkin
3f9589ae64 Remove max_qp argument from GetSimulcastConfig()
This is a cleanup of simulcast.cc. max_qp is not needed to decide simulcast config. Move setting of max QP in VideoStream one level up, to EncoderStreamFactory::CreateEncoderStreams(), where it can be set per stream.

Bug: webrtc:351644568, b/352504711
Change-Id: Ia0e3e9d90032383574dc8867b30d362e9c5df7e8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357102
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42634}
2024-07-15 10:23:10 +00:00
Sergey Silkin
55d328dc25 Add ssilkin@webrtc to OWNERS in video/
Bug: none
Change-Id: Ie5b5e339634c07d260cc3e10312f97aad63fa552
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357180
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42633}
2024-07-15 09:11:54 +00:00
Sergey Silkin
c0a32fe01b Remove bitrate_priority argument from GetSimulcastConfig()
This is a cleanup of simulcast.cc. bitrate_priority is not needed to decide simulcast config. Move setting of bitrate priority in VideoStream one level up, to EncoderStreamFactory::CreateEncoderStreams().

Bug: webrtc:351644568
Change-Id: I002d728ccf8d141fe4bbb32b390129ce57c830cd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357101
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42629}
2024-07-12 15:02:59 +00:00
Sergey Silkin
be2f8f6ec8 Reduce number of InterpolateSimulcastFormat calls
Call it once instead of 3 times.

Also remove FindSimulcastMaxBitrate, FindSimulcastTargetBitrate, FindSimulcastMinBitrate and a one parameter less version of InterpolateSimulcastFormat.

Bug: b/337757868, webrtc:351644568
Change-Id: I7b4002fc3134c47f368bb833925c959a07ce5177
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356980
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42623}
2024-07-11 14:34:18 +00:00
Johannes Kron
38e3466837 Remove deprecated function FillTimingInfo()
The function has been deprecated in favor of
FrameEncodeMetadataWriter::FillMetadataAndTimingInfo().

Bug: chromium:328598314
Change-Id: Iaf2008e855dbd71f2d7cf412d95c5932b3645d71
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356042
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42613}
2024-07-09 23:23:55 +00:00
Tommi
187a4363c0 Remove more sstream deps
Bug: webrtc:8982
Change-Id: I7e1e2a8515b84567d6fe8127ff0e2806a2a4714a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356400
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42610}
2024-07-09 10:30:26 +00:00
Sergey Silkin
3172d16ea0 Clean up EncoderStreamFactory
* Simplified ctor. Get settings (max_qp, content_type, etc) from encoder_config passed to CreateEncoderStreams().

* Some tests assigned VideoEncoderConfig::video_stream_factory to EncoderStreamFactory they created. That's not really needed. VideoStreamEncoder creates the factory if video_stream_factory is not provided [1]. Removed video_stream_factory initialization in tests.

[1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/video/video_stream_encoder.cc;l=1002;drc=1d7d0e6e2c5002815853be251ce43fe88779ac85

Bug: b/347150850, webrtc:42233936
Change-Id: Ie0322abb6c48e1a9bd10e9ed3879e3ed484fea5d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355321
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42608}
2024-07-09 09:47:55 +00:00
Erik Språng
8ac4a464f0 Default enable new bitrate adjuster behavior.
Only minor positive changes seen in initial testing. Let's default-
enable and monitor behavior through the normal release cycle.

Bug: b/349561566
Change-Id: Id6b39daa159068bf076acc34888b5d7eaf110329
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356641
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42607}
2024-07-09 06:30:13 +00:00
Sergey Silkin
ffca3241a0 Disable AV1 screencast test on Mac
Bug: webrtc:351644561
Change-Id: I73101e22f373cd0aca8ca4faa49a2237b2a1fe8f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355961
Auto-Submit: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42599}
2024-07-08 07:33:48 +00:00
Philipp Hancke
c02488f0f9 Log last decoded frame rtp timestamp
in addition to last received frame rtp timestamp. This helps in cases
where frames continue to be received but the decoder fails to decode.

BUG=None

Change-Id: I56ad5f9ef85cc598d3c1a1971c4c697eb6b70165
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356080
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@meta.com>
Reviewed-by: Markus Handell <handellm@google.com>
Cr-Commit-Position: refs/heads/main@{#42596}
2024-07-05 15:41:33 +00:00
Erik Språng
db65fda82f Wire up trial for alternative EncoderBitrateAdjuster behavior.
Behind a flag, the new behavior changes how the "media rate" utilization
is calculated:

* Instead of per spatial & temporal layer, it's per spatial layer only.
* Overshoot is compared to real target vs adjusted target.
* Window takes quite periods/frame drops more into consideration.

This should lead to less push-back when not network constrained and
complex content is used causing bursty behavior.

Bug: b/349561566
Change-Id: I402e6531183493c963fec48ae363ce0b859b396a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356480
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42593}
2024-07-05 13:36:26 +00:00
Erik Språng
c592257953 Add rate utilization tracker helper class.
This class measures the allocated cumulative byte budget (as specified
by one or more rate updates) and the actual cumulative number of bytes
produced over a sliding window.

A utilization factor (produced bytes / budgeted bytes) is calculated
seen from the first data point timestamp until the last data point
timestamp plus the amount time needed to send that last data point
given no further updates to the rate.

Wireup to EncoderBitrateAdjuster will happen in a follow-up CL.

Bug: b/349561566
Change-Id: Id0dc183b07a96366531007be9ff1c1ec6574e9ff
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356200
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42591}
2024-07-04 17:47:38 +00:00
Åsa Persson
3b15e46a4c Get min bitrate from spatial layers for AV1 (instead of bitrate limits).
Bitrate limits should have been applied to the spatial layers in ApplySpatialLayerBitrateLimits (and usage is restricted to a single active stream/layer).

Bug: b/299588022
Change-Id: Iaae4ece28b8a95eea7d4bacba292847ba5b4000b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355841
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42588}
2024-07-04 13:16:58 +00:00
Tommi
55c3600781 Remove <ostream> dependencies
Some dependencies still exist but are a bit more complex to remove.
This CL removes either unused or easily replaced with ToString()
instances of ostream usage. In one case, moving the operator<<
implementation to the one test file that requires it.

Bug: webrtc:8982
Change-Id: Ia5c840b12a42893494af401317a3daf2fe50ba9b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/356240
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42582}
2024-07-03 12:27:55 +00:00
Erik Språng
fe4c1dd6dc Fix logic reading spatial/temporal id in VideoStreamEncoder.
The temporal id must be read from `EncodedImage` rather than codec
specifics for AV1. Furthermore, in some configs the spatial id of
`EncodedImage` is populated and set to 0 while the simulcast id can
also be simultaneously populated and set to values, including non-zero.
To solve this, just take the max of the two.

Bug: b/349561566
Change-Id: I46c61b7f0fff7a7ab8d7262c3a8d413f49b3286a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355904
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42573}
2024-07-02 14:15:52 +00:00
Johannes Kron
e0287f2797 Add default values and field trials to QualityConvergenceMonitor
Add default values and a static Create() function that determines
the parameters to use based on the specified codec and potential
field trial overrides.

Bug: chromium:328598314
Change-Id: I7a9331a1fd0ed4bd258788760592ea84e535e43b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355903
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@google.com>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42567}
2024-07-01 22:37:46 +00:00