2373 Commits

Author SHA1 Message Date
Qiu Jianlin
d171832b6c Set default simulcast temporal layer to 1 if not configured.
For H.265 when scalability mode is not configured for simulcast layers,
the default mode of L1T1 should be assumed instead of L1T3, as that is
the most commonly supported temporal scalability on all devices for
H.265.

Bug: chromium:41480904
Change-Id: Ia9bc91729eb393850dfe5e8fb04280b4f784560d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/369080
Commit-Queue: Jianlin Qiu <jianlin.qiu@intel.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43452}
2024-11-26 04:07:52 +00:00
Erik Språng
e5f6f1fab4 Add optional corruption filter settings to EncodedImage.
This is a prerequisite for enabling implementation-specific filter
settings for automatic corruption detection.

Bug: webrtc:358039777
Change-Id: I363c592aa35164f690dd4ad1204e90afc0277d8b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/368940
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43443}
2024-11-22 12:10:31 +00:00
Erik Språng
9aeed0c5f4 Avoid potential deadlock due to queue in corruption detection.
In particular, some platforms have a limited pool of frames in the
capturer stack, so we need to avoid stashing raw frames in the frame
instrumentation generator that may be dropped by limiting the size of
the queue and avoid putting anything in there until we know we will
send it to the encoder.

Bug: webrtc:358039777
Change-Id: I054ae53dd5e6ac6a22da39c5049f47788561e77a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/368641
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43432}
2024-11-20 22:50:41 +00:00
Harald Alvestrand
752235261e Remove all references to codec-level transport-cc functions and flags.
This seems to have no effect on tests, so it appears that these were
not used after all.
The goal is to make transport-cc a media-section-level attribute.

Bug: webrtc:378698658
Change-Id: Ia20ca5b91472b02db30f911ad1a1892cf36cd682
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/368440
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43411}
2024-11-18 10:20:01 +00:00
Qiu Jianlin
f54707cd71 Reuse VP9 simulcast stream limits for H.265.
H.265 should have limits probably between VP9 and AV1, instead of using
VP8 tables. For now we reuse VP9 tables but eventually we may create
table for H.265.

Bug: chromium:41480904
Change-Id: I6dc2ec629142ee06f1c82a2df30d753ec1353496
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/368240
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Jianlin Qiu <jianlin.qiu@intel.com>
Cr-Commit-Position: refs/heads/main@{#43404}
2024-11-15 10:58:16 +00:00
Emil Vardar
4c171e84c3 Prevent upscaling when calculating sample values.
Bug: webrtc:358039777
Change-Id: I33edc12f312d0d37eac0c39a913313a1aa6f1de5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366942
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43394}
2024-11-13 14:57:14 +00:00
Qiu Jianlin
4405d06b97 Add H.265 to codecs that supports temporal scalability.
Also updated the test to cover IsTemporalLayersSupported() for all types
of codecs.

Bug: chromium:41480904
Change-Id: I25788a87737aba7308b1d6980ad5b2c26b0e225f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/367570
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jianlin Qiu <jianlin.qiu@intel.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43369}
2024-11-07 14:24:42 +00:00
Evan Shrubsole
7589689774 Replace cricket::LeastCommonMultiple and cricket::GreatestCommonDivisor with std::lcm and std::gcd.
The std::lcm and std::gcd functions are part of the C++ standard
library. The existing functions are marked as deprecated rather than
deleted in the case of possible third party uses.

#rtc_cleanup

Bug: webrtc:377205743
Change-Id: I174e663f152d750c984a35dc7136bc18dc01bc8e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/367440
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Auto-Submit: Evan Shrubsole <eshr@google.com>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43368}
2024-11-07 13:30:28 +00:00
Emil Vardar
a5676c17b3 Change scale factor to 0.5
The tests shows that using a scale factor around 0.5 gives the best precision and F1 score.

Bug: webrtc:358039777
Change-Id: I22557eb9e6318ecaa726b56d3ccb2125fdf65f7e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/367681
Reviewed-by: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Cr-Commit-Position: refs/heads/main@{#43366}
2024-11-07 10:32:41 +00:00
Evan Shrubsole
b444820216 Update freshness of adaptation documentation
Checked wording and that all links are functional.

Bug: b/363184959
Change-Id: Idc15a5671b49d7ab1bf71cee00c704c51bee7474
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/367780
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Auto-Submit: Evan Shrubsole <eshr@google.com>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43362}
2024-11-06 14:53:01 +00:00
Qiu Jianlin
faef5de87c Cleanup H.265 TODOs.
Cleanup some of the TODOs for H.265. They are either invalid or their handling should be merged with other codec types.

Bug: chromium:41480904
Change-Id: I76263354b1b87035e240d77283b21a9a26dcb45b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366044
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Jianlin Qiu <jianlin.qiu@intel.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43359}
2024-11-05 14:06:18 +00:00
Danil Chapovalov
037ab2627d In tests replace AudioProcessingBuilder with BuiltinAudioProcessingBuilder
To move towards deprecating AudioProcessingBuilder

Bug: webrtc:369904700
Change-Id: I7998b331eca26c2185c94c39c1310ef7b6faa717
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/367221
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Auto-Submit: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43347}
2024-11-01 12:38:34 +00:00
Danil Chapovalov
141dfb036d Remove webrtc::ToLogString as no longer needed
These function were replaced with AbslStringify

Bug: None
Change-Id: Ia34b98ed4e0ed18bb52fe9370cff7a6f70caae6d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364621
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Auto-Submit: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43346}
2024-11-01 11:12:52 +00:00
Palak Agarwal
c4f61fbde3 Rename capture_time_identifier to presentation_timestamp
After landing this change, we can change the corresponding usage in
blink to start using presentation_timestamp as well and then delete
the remaining usage of capture_time_identifier.


Bug: webrtc:373365537
Change-Id: I0c4f2b6b3822df42d6e3387df2c243c3684d8a41
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365640
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Commit-Queue: Palak Agarwal <agpalak@google.com>
Cr-Commit-Position: refs/heads/main@{#43317}
2024-10-28 12:11:38 +00:00
Henrik Boström
e8c97c0d09 Reland "Rename requested_resolution to scale_resolution_down_to."
This is a reland of commit 82617ac51e7825db53451818f4d1ad52b69761fd

The reason for the revert was a downstream use of
`rtc::VideoSinkWants::requested_resolution`, so in this reland we don't
rename this field, it's fine just to rename the one in
RtpEncodingParameters for now.

Original change's description:
> Rename `requested_resolution` to `scale_resolution_down_to`.
>
> This is a pure refactor/rename CL without any changes in behavior.
>
> This field is called scaleResolutionDownTo in the spec and JavaScript.
> Let's make C++ match to avoid confusion.
>
> In order not to break downstream during the transition a variable with
> the old name being a pure reference to the renamed attribute is added.
> This means we have to add custom constructors, but we can change this
> back to "= default" when the transition is completed, which should only
> be a couple of CLs away.
>
> Bug: webrtc:375048799
> Change-Id: If755102ccd79d46020ce5b33acd3dfcc29799e47
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366560
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
> Commit-Queue: Henrik Boström <hbos@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#43300}

NOTRY=True

Bug: webrtc:375048799
Change-Id: Ic4ee156c1d50aa36070a8d84059870791dcbbe5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366660
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43304}
2024-10-25 08:39:49 +00:00
Sergio Garcia Murillo
b5289d72be Unregister previous external decoders when creating a new one.
When switching between payload types on same ssrc, a HW decoder is only
used the first payload type received, falling back to SW decoding if
payload type is changed.

This change unregister any external decoder previously registered so it
can be re-initialized if received again.

Bug: webrtc:375097852
Change-Id: Ic04951c5676d9a3854eefb2ab8836ef8a2645d78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366580
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43302}
2024-10-24 15:12:05 +00:00
Florent Castelli
af44d8ff06 Revert "Rename requested_resolution to scale_resolution_down_to."
This reverts commit 82617ac51e7825db53451818f4d1ad52b69761fd.

Reason for revert: Break downstream projects

Original change's description:
> Rename `requested_resolution` to `scale_resolution_down_to`.
>
> This is a pure refactor/rename CL without any changes in behavior.
>
> This field is called scaleResolutionDownTo in the spec and JavaScript.
> Let's make C++ match to avoid confusion.
>
> In order not to break downstream during the transition a variable with
> the old name being a pure reference to the renamed attribute is added.
> This means we have to add custom constructors, but we can change this
> back to "= default" when the transition is completed, which should only
> be a couple of CLs away.
>
> Bug: webrtc:375048799
> Change-Id: If755102ccd79d46020ce5b33acd3dfcc29799e47
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366560
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
> Commit-Queue: Henrik Boström <hbos@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#43300}

Bug: webrtc:375048799
Change-Id: Ie41723a39420e12e7b5b681d3d00ccd14f66b4b1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366642
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#43301}
2024-10-24 14:51:29 +00:00
Henrik Boström
82617ac51e Rename requested_resolution to scale_resolution_down_to.
This is a pure refactor/rename CL without any changes in behavior.

This field is called scaleResolutionDownTo in the spec and JavaScript.
Let's make C++ match to avoid confusion.

In order not to break downstream during the transition a variable with
the old name being a pure reference to the renamed attribute is added.
This means we have to add custom constructors, but we can change this
back to "= default" when the transition is completed, which should only
be a couple of CLs away.

Bug: webrtc:375048799
Change-Id: If755102ccd79d46020ce5b33acd3dfcc29799e47
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366560
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43300}
2024-10-24 11:38:21 +00:00
Olov Brändström
05666b4db7 Function that Converts NtpTime to a Timestamp with UTC epoch in Clock.
danilchap@webrtc.org suggested to add a converter for NtpTime <-> UTC Timestamp for in https://webrtc-review.googlesource.com/c/src/+/365641.

This CL add a NtpTime -> UTC Timestamp in Clock, and change code to start to use the new function.

Bug: None
Change-Id: If4af6cb8e31c1731692edfb8358e67b7a43226a0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/366001
Commit-Queue: Olov Brändström <brandstrom@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43293}
2024-10-23 14:19:08 +00:00
Fanny Linderborg
b280cb95c6 Add a basic end-to-end test for corruption detection.
This adds a Call-based test, that sets up video-pipeline with a VP8
encoder and the corruption detection header extension configured.
It then verifies that the corruption likelihood metrics are populated
in the receive stream stats.

Bug: webrtc:358039777
Change-Id: Ide005459a801778de4238e786f13efc8c3245f3f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365860
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43254}
2024-10-17 09:25:41 +00:00
Olov Brändström
558c2dc539 Change timestamps type from int64 to Timestamp in MediaReceiverInfo.
We should use the Timestamp type, rather then int64, to store timestamps. In https://webrtc-review.googlesource.com/c/src/+/365001/ an additional int64 timestamp was added (last_sender_report_timestamp_ms).

This CL fixes the new timestamp, as well as other similar timestamps in MediaReceiverInfo (last_sender_report_utc_timestamp_ms and last_sender_report_remote_utc_timestamp_ms).

Bug: webrtc:372393493
Change-Id: I0e473730e85a69ec595b421e2c3db920364008eb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365641
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Olov Brändström <brandstrom@google.com>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43248}
2024-10-16 11:02:37 +00:00
Emil Vardar
129f228f59 Post corruption score aggregation to worker thread.
Bug: webrtc:358039777
Change-Id: Ia7196436aaa024019869a7521243da0576dbb148
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365600
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Auto-Submit: Emil Vardar (xWF) <vardar@google.com>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43238}
2024-10-15 09:43:40 +00:00
Harald Alvestrand
d8bddfef88 Split up the call/video_stream_api target
The split shows that some places don't need it at all. Most other
places will depend on both send and receive stream targets.

Bug: webrtc:373151158
Change-Id: I788136a2ee84180c16345a7929b7f7bf3f97507b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365460
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43230}
2024-10-14 08:26:16 +00:00
Emil Vardar
346cf7c4e5 Add support for frame pair corruption score calculation.
With this changes users can calculate the corruption score on two frames e.g. in test scenarios where one has access to the input and output file.

Bug: webrtc:358039777
Change-Id: Id864010115aa040284ec09b42d0279ccb45960b9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364161
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Cr-Commit-Position: refs/heads/main@{#43222}
2024-10-11 09:28:56 +00:00
Fanny Linderborg
518bd61cec Forward the corruption score from the decoder to ReceiveStatisticsProxy
Bug: webrtc:358039777
Change-Id: Iace01daa53d08b5d0c484b5f55da73ba230317da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365095
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43213}
2024-10-10 10:45:35 +00:00
Fanny Linderborg
e1adfc05ac Rename FrameToRender to OnFrameToRender
This is to make the name consistent with the other methods in the
interface and additionally to in the future not have a function that has
the same name as the `FrameToRender` struct.

Bug: webrtc:358039777
Change-Id: Iac727d93ab9e020a073477bd33d0f67f9983a0aa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364961
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43195}
2024-10-08 16:29:01 +00:00
Olov Brändström
b9c4c242d4 rename timestamps to show epoch
I missed one timestamp in https://webrtc-review.googlesource.com/c/src/+/363946, meaning that the config flag that was added do not yet work for all timestamps in RTCStats objects. The RTCRemoteOutboundRtpStreamStats still has UTC timestamps even if the config flag is set.

I will solve this by saving both an UTC (existing) and env (to be added) timestamp, and then let rtc_stats_collector choose timestamp based on the value of the config flag (just like RTCRemoteInboundRtpStreamStats is done in the 363946 commit).

Before adding the new env_ timestamp I want to make this change. I rename the existing timestamp to show what epoch it uses (NTP or UTC). This will later make it clear which timestamp is which.

So this CL will make no logical change, just renaming members.

I only need to rename the last_sender_report_timestamp_ms, but opted to rename the remote timestamp as well, to be consistent with the naming convention I add in this CL.

Bug: chromium:369369568
Change-Id: Icfe7cf274995b39799e1478a1bb8cdf5134f0b16
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364782
Commit-Queue: Olov Brändström <brandstrom@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43194}
2024-10-08 16:27:58 +00:00
Fanny Linderborg
a507a08904 Calculate corruption score once the frame is decoded
Bug: webrtc:358039777
Change-Id: I291e8e505f2ea7f9f95da4c83cd7679b49f2bc56
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364600
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43192}
2024-10-08 13:26:00 +00:00
Fanny Linderborg
215401f651 Reland "Add a FrameToRender argument struct as input to FrameToRender"
This is a reland of commit 01f91c81f7660be842fa44e96bf804a8b2402f47

Original change's description:
> Add a FrameToRender argument struct as input to FrameToRender
>
> This is to make it easier to add new arguments to the method in the
> future. We will remove the already existing method accordingly to WebRTCs deprecation rules.
>
> Bug: webrtc:358039777
> Change-Id: Id0706de5216fbd0182cac80ebfccfc4a6a055ee8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364642
> Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#43181}

Bug: webrtc:358039777
Change-Id: I404bb9660d9f4436c0658814fd3ac7d74e483f0f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364900
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43188}
2024-10-08 06:22:03 +00:00
Jeremy Leconte
5680d8199a Revert "Add a FrameToRender argument struct as input to FrameToRender"
This reverts commit 01f91c81f7660be842fa44e96bf804a8b2402f47.

Reason for revert: break downstream projects.

Original change's description:
> Add a FrameToRender argument struct as input to FrameToRender
>
> This is to make it easier to add new arguments to the method in the
> future. We will remove the already existing method accordingly to WebRTCs deprecation rules.
>
> Bug: webrtc:358039777
> Change-Id: Id0706de5216fbd0182cac80ebfccfc4a6a055ee8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364642
> Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#43181}

Bug: webrtc:358039777
Change-Id: Id59633023a428fb63aadeb266421b09040e590bb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364841
Owners-Override: Jeremy Leconte <jleconte@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#43184}
2024-10-07 12:46:24 +00:00
Fanny Linderborg
01f91c81f7 Add a FrameToRender argument struct as input to FrameToRender
This is to make it easier to add new arguments to the method in the
future. We will remove the already existing method accordingly to WebRTCs deprecation rules.

Bug: webrtc:358039777
Change-Id: Id0706de5216fbd0182cac80ebfccfc4a6a055ee8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364642
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43181}
2024-10-07 11:47:17 +00:00
Henrik Boström
b23b3dd9b1 Improve simulcast CPU adaptation when requested_resolution API is used.
In simulcast, BW adaptation causes layers to be disabled rather than
downscaling layers. But CPU adaptation restricts the resolution of all
layers, this means that a 540p restriction on 180p:360p:720p results in
180p:360p:540p, which is fine but a) it's inconsistent with BW
adaptation and b) it's not ideal for performance, because non power of
two scaling factors means we can't use a single encoder instance to
produce all layers (the CPU adaptation could actually result in even
more CPU usage and further adaptation as a result).

This CL disables top layers by limiting `max_num_layers` based on
`restrictions_` and the layers' `requested_resolution`, the end result
is 180p:360p:- when CPU adaptation kicks in.

Note that the problem described (and therefore the solution) is
specific to the `requested_resolution` API. If instead the
`scale_resolution_down_by` API is used, all scaling is relative and we
get 135p:270p:540p, which is problematic for other reasons (180p and
360p no longer sent, middle layer no longer HW accelerated).

Bug: webrtc:366415118
Change-Id: I2e238b1b87470413c21623b21d0ce20eadf6c8c7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364660
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43172}
2024-10-04 13:53:55 +00:00
Fanny Linderborg
b63c05d1cc Remove unused misspelled function
Bug: webrtc:358039777
Change-Id: I5573a8ab40a42663cfc2d24576b90e1100972e7b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363942
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43165}
2024-10-03 13:55:44 +00:00
Fanny Linderborg
4f6f92a986 Convert CorruptionDetectionMessage to FrameInstrumentationSyncData
Bug: webrtc:358039777
Change-Id: I7504573cdee40ee3224242e19c254de815e0311b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364485
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43163}
2024-10-03 13:27:33 +00:00
Olov Brändström
4baeed3b97 Use environment monotonic timestamps (i.e. not UTC) in RTCStats.
Add media config for using environment monotonic timestamps (i.e. not UTC) in RTCStats constructor, and implemented the usage of the flag.

Bug: chromium:369369568
Change-Id: Ia93d048742c28af201164fe7b2152b791bb6d0b6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363946
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Olov Brändström <brandstrom@google.com>
Cr-Commit-Position: refs/heads/main@{#43156}
2024-10-03 09:07:17 +00:00
Fanny Linderborg
1869afa63a Parse extension and store it in RTPVideoHeader
Bug: webrtc:358039777
Change-Id: Ib70046662877efa5f8d0cbe559b44d138f4733e2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364481
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43146}
2024-10-02 13:46:13 +00:00
Fanny Linderborg
a49ab28fca Set CodecSpecific.FrameInstrumentationData in RtpFrameObject ctor
Bug: webrtc:358039777
Change-Id: Ib0a663f06b293c62a4eb0689b82b3bf919cff25f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364282
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43136}
2024-10-02 07:09:11 +00:00
Fanny Linderborg
55398a7612 Add helper for comparing FrameInstrumentationData with a VideoFrame
Bug: webrtc:358039777
Change-Id: Ibe597160658dbc66aba427f4e30dade4d6fe56e2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363701
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43134}
2024-10-02 05:38:48 +00:00
Fanny Linderborg
052cc9d92a Fix misspelled function
Bug: webrtc:358039777
Change-Id: I640256a33c6a2f998042555607e053aa0b09e626
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363803
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43091}
2024-09-27 07:19:52 +00:00
Fanny Linderborg
0a3a6908e8 Ensure both corruption detection tests are formulated the same way
DoesNotPopulateFrameInstrumentationDataWhenSetNotTo should be formulated equivalently to PopulatesFrameInstrumentationDataWhenSetTo.

Bug: webrtc:358039777
Change-Id: I22d487d0a88cd3e1badb3bb8bf304a0322f9d53e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363862
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43090}
2024-09-27 07:18:49 +00:00
Fanny Linderborg
a61a25a0b1 Correct format for TODO comments
Bug: webrtc:358039777
Change-Id: Iad5f8867bcbca883b16617819ea24a965160d23e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363802
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43089}
2024-09-27 07:17:40 +00:00
Emil Vardar
965f134b2b Number of samples need to be more than 0 when calculating corruption score.
Bug: webrtc:358039777
Change-Id: I28597185731ba3d9485103f7c24813f2bdd7110a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363120
Reviewed-by: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Cr-Commit-Position: refs/heads/main@{#43075}
2024-09-24 13:09:44 +00:00
Fanny Linderborg
5a294aeea3 Allow the standard deviation for GetSampleValuesForFrame to be 0
Setting the standard deviation to 0 is valid and should be interpreted
as directly using the sample value at the coordinates without weighting.
This is made explicit in the documentation for the Corruption Detection
extension:
http://www.webrtc.org/experiments/rtp-hdrext/corruption-detection

Also, change stddev to std_dev in halton_frame_sampler files.

Bug: webrtc:358039777
Change-Id: Id5aa4110194f7f2b2fe9914c94304c90afd64198
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363300
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43070}
2024-09-23 15:17:58 +00:00
Mirko Bonadei
a8829eb5f3 macro cleanup: "(const override)" -> "(const, override)"
Bug: None
Change-Id: Iffd5db39b1a5ae70b403193b40054df04cf5600b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/362800
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Auto-Submit: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43065}
2024-09-22 18:30:29 +00:00
Henrik Boström
b6ee51b7a5 Don't restrict max simulcast layers when requested_resolution is used.
The code that restricts the maximum number of simulcast layers based on
resolution is a spec-compliance bug and doesn't make much sense: if the
app asks for 3 layers it should get 3 layers. Since the app knows the
size of the track, it could very easily ask for 1 layer when resolution
is small if that is the behavior it wanted. If the app doesn't ask to
disable layers, WebRTC shouldn't disable layers on its behalf.

This behavior makes even less sense with this "new" API since the app
is explicitly controlling the send resolution in absolute terms.

Removing this behavior in the general case is out of scope since it
would break backwards compatibility, but since `requested_resolution`
has not been exposed to the web yet and existing usage is small, this
is an opportunity to fix the compliance bug for this API.

This CL makes the last web platform test for "scaleResolutionDownTo"
pass.

Bug: chromium:363544347
Change-Id: Ic6fadf3cad69d3beec4ae03d3d031e8062382ad9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/363100
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43061}
2024-09-20 11:22:18 +00:00
Fanny Linderborg
2f106d683a Add FrameInstrumentationGenerator to VideoStreamEncoder
The FrameInstrumentationGenerator is responsible for generating the
instrumentation data that will be used to detect corruption. The data is
then passed to the encoder in the CodecSpecificInfo.

Bug: webrtc:358039777
Change-Id: I79d0534920b4c7fa001e1138371dfd36c13424fb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/362584
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Fanny Linderborg <linderborg@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43060}
2024-09-20 11:12:21 +00:00
Emil Vardar
4a201de10d Add support for corruption classification.
This class calculates the corruption score based on the given samples from two frames.

Bug: webrtc:358039777
Change-Id: Ib036f91ec16609e827137cc35d342a2c49764737
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/362801
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Emil Vardar (xWF) <vardar@google.com>
Cr-Commit-Position: refs/heads/main@{#43043}
2024-09-18 13:38:41 +00:00
Fanny Linderborg
f045dbd67c Modify sequence index on key frames
For key frames: increase the sequence index until the last 7 bits are
all zeroes. If this results in an overflow, wraparound to 0.

Also:
* Allow setting and getting the sequence index
* Allow getting LayerId

Bug: webrtc:358039777
Change-Id: Ibe16689a3d1eb5706d4fce5c9220770046f26896
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/362540
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Fanny Linderborg <linderborg@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43042}
2024-09-18 09:29:47 +00:00
Henrik Boström
825e4f19ce VideoAdapter: Interpret requested resolution as max restriction.
The `requested_resolution` API must not change aspect ratio, example:
- Frame is 60x30
- Requested is 30x30
- We expect 30x15 (not 30x30!) as to maintain aspect ratio.

This bug was previously fixed by making VideoAdapter unaware of the
requested resolution behind a flag: this seemed OK since the
VideoStreamEncoder ultimately decides the resolution, whether or not
the incoming frame is adapted.

But this is not desired for some non-Chrome use cases. This CL attempts
to make both Chrome and non-Chrome use cases happy by implementing the
aspect ratio preserving restriction inside VideoAdapter too.

This allows us to get rid of the "use_standard_requested_resolution"
flag and change the "VideoStreamEncoderResolutionTest" TEST_P to
TEST_F.

Bug: webrtc:366067962, webrtc:366284861
Change-Id: I1dfd10963274c5fdfd18d0f4443b2f209d2e9a4b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/362720
Reviewed-by: Jonas Oreland <jonaso@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43037}
2024-09-17 14:33:26 +00:00
Sergey Silkin
e81ba30897 Increase AV1 QP threshold for quality convergence from 40 to 60.
Bug: chromium:328598314
Change-Id: I132b4c30f132ace2bbef6359edd994c1ad75c9ad
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/362620
Reviewed-by: Johannes Kron <kron@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43035}
2024-09-17 14:12:44 +00:00