5549 Commits

Author SHA1 Message Date
Etienne Pierre-doray
03bce3f49d Reland "[Battery]: Delay start of TaskQueuePacedSender." Take 3
This is a reland of 89cb65ed663a9000b9f7c90a78039bd85731e9ae
... and f28aade91dcc2cb8f590dc1379ac7ab5c1981909
... and 2072b87261a6505a88561bdeab3e7405d7038eaa

Reason for revert: Failing DuoGroupsMediaQualityTest due to missing
TaskQueuePacedSender::EnsureStarted() in google3.
Fix: This CL adds the logic behind TaskQueuePacedSender::EnsureStarted,
but initializes with |is_started| = true. Once the caller in google3 is
updated, |is_started| can be switched to false by default.

> Original change's description:
> Reason for revert: crashes due to uninitialized pacing_bitrate_
> crbug.com/1190547
> Apparently pacer() is sometimes being used before EnsureStarted()
> Fix: Instead of delaying first call to SetPacingRates(),
> this CL no-ops MaybeProcessPackets() until EnsureStarted()
> is called for the first time.

> Original change's description:
> > [Battery]: Delay start of TaskQueuePacedSender.
> >
> > To avoid unnecessary repeating tasks, TaskQueuePacedSender is started
> > only upon RtpTransportControllerSend::EnsureStarted().
> >
> > More specifically, the repeating task happens in
> > TaskQueuePacedSender::MaybeProcessPackets() every 500ms, using a self
> > task_queue_.PostDelayedTask().
> >
> > Bug: chromium:1152887
> > Change-Id: I72c96d2c4b491d5edb45a30b210b3797165cbf48
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208560
> > Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
> > Reviewed-by: Henrik Boström <hbos@webrtc.org>
> > Reviewed-by: Erik Språng <sprang@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#33421}
>
> Bug: chromium:1152887
> Change-Id: I9aba4882a64bbee7d97ace9059dea8a24c144f93
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212880
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#33554}

Bug: chromium:1152887
Change-Id: Ie365562bd83aefdb2757a65e20a4cf3eece678b9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/213000
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
Cr-Commit-Position: refs/heads/master@{#33629}
2021-04-06 16:59:12 +00:00
Fyodor Kyslov
b454767f10 AV1: Use AOM_USAGE_REALTIME when creating encoder
libaom is compiled with REALTIME_ONLY option. Soon it will be impossible
to create encoder or request default config with usage other than
AOM_USAGE_REALTIME. Fixing the wrapper to use proper usage parameter

Bug: None
Change-Id: I862741a724e4a8524f22ae79700b3da6517dbfb2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/214100
Commit-Queue: Fyodor Kyslov <kyslov@google.com>
Reviewed-by: Marco Paniconi <marpan@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33624}
2021-04-06 02:38:34 +00:00
Alessio Bazzica
8aaa604375 AGC2 new data dumps
Bug: webrtc:7494
Change-Id: Id288dd426e1c2754805bc548fbffe0eaeaacf3da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/213420
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Jesus de Vicente Pena <devicentepena@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33605}
2021-03-31 14:55:42 +00:00
Alessio Bazzica
841d74ea80 AGC2 periodically reset VAD state
Bug: webrtc:7494
Change-Id: I880ef3991ade4e429ccde843571f069ede149c0e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/213342
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Jesus de Vicente Pena <devicentepena@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33604}
2021-03-31 14:15:10 +00:00
Alessio Bazzica
b995bb86df AGC2 size_t -> int
Bug: webrtc:7494
Change-Id: I5ecf242e83b509931c1764a37339d11506c5afc6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/213341
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33600}
2021-03-31 11:18:30 +00:00
Philipp Hancke
883fea1548 red: pass through calls to underlying encoder
BUG=webrtc:11640

Change-Id: I87e6f7c91c80d61e64127574485bbdcaedc8120c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181063
Reviewed-by: Minyue Li <minyue@webrtc.org>
Commit-Queue: Philipp Hancke <philipp.hancke@googlemail.com>
Cr-Commit-Position: refs/heads/master@{#33595}
2021-03-30 13:51:51 +00:00
Markus Handell
eca855197a VCMEncodedFrame: add basic support for AV1.
This change adds basic support for setting codecType kVideoCodecAV1 in
VCMEncodedFrames.

Bug: chromium:1191972
Change-Id: I258b39ff89c8b92ebbb288ef32c88b900a35d10e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/213182
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33594}
2021-03-30 11:45:00 +00:00
Mirko Bonadei
6e6411c099 Revert "Add fuzzer to validate libvpx vp9 encoder wrapper"
This reverts commit c184047fef005b86a6dd76f03b0eb5ec01de3c5c.

Reason for revert: Breaks the WebRTC->Chromium roll:

ERROR Unresolved dependencies.
//third_party/webrtc/test/fuzzers:vp9_encoder_references_fuzzer(//build/toolchain/win:win_clang_x64)
  needs //third_party/webrtc/modules/video_coding:mock_libvpx_interface(//build/toolchain/win:win_clang_x64)

We need to add tryjob to catch these. The fix is to make 
//third_party/webrtc/modules/video_coding:mock_libvpx_interface
visible in built_with_chromium builds by moving the target
out of this "if" https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/modules/video_coding/BUILD.gn;l=615;drc=3889de1c4c7ae56ec742fb9ee0ad89657f638169.

Original change's description:
> Add fuzzer to validate libvpx vp9 encoder wrapper
>
> Fix simulcast svc controller to reuse dropped frame configuration,
> same as full svc and k-svc controllers do.
> This fuzzer reminded the issue was still there.
>
> Bug: webrtc:11999
> Change-Id: I74156bd743124723562e99deb48de5b5018a81d0
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212281
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33568}

TBR=danilchap@webrtc.org,sprang@webrtc.org

Change-Id: I1676986308c6d37ff168467ff2099155e8895452
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:11999
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212973
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33573}
2021-03-26 11:17:00 +00:00
Jeremy Leconte
b258c56267 Send and Receive VideoFrameTrackingid RTP header extension.
Bug: webrtc:12594
Change-Id: I2372a361e55d0fdadf9847081644b6a3359a2928
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212283
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/master@{#33570}
2021-03-25 21:57:29 +00:00
Danil Chapovalov
c184047fef Add fuzzer to validate libvpx vp9 encoder wrapper
Fix simulcast svc controller to reuse dropped frame configuration,
same as full svc and k-svc controllers do.
This fuzzer reminded the issue was still there.

Bug: webrtc:11999
Change-Id: I74156bd743124723562e99deb48de5b5018a81d0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212281
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33568}
2021-03-25 18:52:38 +00:00
Jeremy Leconte
4f88a9d1c3 Create a VideoFrameTrackingId RTP header extension.
Bug: webrtc:12594
Change-Id: I518b549b18143f4711728b4637a4689772474c45
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212084
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/master@{#33567}
2021-03-25 17:25:18 +00:00
Ying Wang
4c555cca2d Revert "Reland "[Battery]: Delay start of TaskQueuePacedSender." Take 2"
This reverts commit 2072b87261a6505a88561bdeab3e7405d7038eaa.

Reason for revert: Causing test failure.

Original change's description:
> Reland "[Battery]: Delay start of TaskQueuePacedSender." Take 2
>
> This is a reland of 89cb65ed663a9000b9f7c90a78039bd85731e9ae
> ... and f28aade91dcc2cb8f590dc1379ac7ab5c1981909
>
> Reason for revert: crashes due to uninitialized pacing_bitrate_
> crbug.com/1190547
> Apparently pacer() is sometimes being used before EnsureStarted()
> Fix: Instead of delaying first call to SetPacingRates(),
> this CL no-ops MaybeProcessPackets() until EnsureStarted()
> is called for the first time.
>
> Original change's description:
> > [Battery]: Delay start of TaskQueuePacedSender.
> >
> > To avoid unnecessary repeating tasks, TaskQueuePacedSender is started
> > only upon RtpTransportControllerSend::EnsureStarted().
> >
> > More specifically, the repeating task happens in
> > TaskQueuePacedSender::MaybeProcessPackets() every 500ms, using a self
> > task_queue_.PostDelayedTask().
> >
> > Bug: chromium:1152887
> > Change-Id: I72c96d2c4b491d5edb45a30b210b3797165cbf48
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208560
> > Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
> > Reviewed-by: Henrik Boström <hbos@webrtc.org>
> > Reviewed-by: Erik Språng <sprang@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#33421}
>
> Bug: chromium:1152887
> Change-Id: I9aba4882a64bbee7d97ace9059dea8a24c144f93
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212880
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#33554}

TBR=hbos@webrtc.org,sprang@webrtc.org,etiennep@chromium.org

Change-Id: I430fd31c7602702c8ec44b9e38e68266abba8854
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1152887
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212965
Reviewed-by: Ying Wang <yinwa@webrtc.org>
Commit-Queue: Ying Wang <yinwa@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33559}
2021-03-25 10:50:53 +00:00
philipel
02b1321b47 Clean up video_coding namespace snipets.
Bug: webrtc:12579
Change-Id: I487fe017f30746e2fe83a122123b236295d96d28
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212962
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33558}
2021-03-25 10:44:40 +00:00
Aaron Clauson
5d6abbddf4 Adds missing header to fix compilation error when compiling with use_custom_libcxx set to false.
Fixed: webrtc:12584
Change-Id: I8830095f887e7ee8887bc37106da847b60c1e996
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211762
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33557}
2021-03-25 09:57:00 +00:00
Etienne Pierre-doray
2072b87261 Reland "[Battery]: Delay start of TaskQueuePacedSender." Take 2
This is a reland of 89cb65ed663a9000b9f7c90a78039bd85731e9ae
... and f28aade91dcc2cb8f590dc1379ac7ab5c1981909

Reason for revert: crashes due to uninitialized pacing_bitrate_
crbug.com/1190547
Apparently pacer() is sometimes being used before EnsureStarted()
Fix: Instead of delaying first call to SetPacingRates(),
this CL no-ops MaybeProcessPackets() until EnsureStarted()
is called for the first time.

Original change's description:
> [Battery]: Delay start of TaskQueuePacedSender.
>
> To avoid unnecessary repeating tasks, TaskQueuePacedSender is started
> only upon RtpTransportControllerSend::EnsureStarted().
>
> More specifically, the repeating task happens in
> TaskQueuePacedSender::MaybeProcessPackets() every 500ms, using a self
> task_queue_.PostDelayedTask().
>
> Bug: chromium:1152887
> Change-Id: I72c96d2c4b491d5edb45a30b210b3797165cbf48
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208560
> Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33421}

Bug: chromium:1152887
Change-Id: I9aba4882a64bbee7d97ace9059dea8a24c144f93
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212880
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Etienne Pierre-Doray <etiennep@chromium.org>
Cr-Commit-Position: refs/heads/master@{#33554}
2021-03-24 18:46:51 +00:00
Jonas Oreland
90c3981773 Fix RtpVideoLayersAllocationExtension::Write of invalid allocation
This patch is a follow up to https://webrtc-review.googlesource.com/c/src/+/212743
which broke downstream fuzzer :(

prior to https://webrtc-review.googlesource.com/c/src/+/212743,
RtpVideoLayersAllocationExtension::AllocationIsValid returns
false if rtp_stream_index > max(layer.rtp_stream_index)

After https://webrtc-review.googlesource.com/c/src/+/212743,
0 spatial layers is supported, so the AllocationIsValid is
updated to allow any value if not layers are present.

Bug: webrtc:12000
Change-Id: Ib3e64ecb621f795b9126442c50969f5178c85a37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212901
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33551}
2021-03-24 13:53:13 +00:00
Henrik Boström
56db9ff1e1 VideoStreamEncoder: Don't map kNative video frame buffers.
Follow-up CL to VP8 and VP9 encoders taking care of mapping.
Context again:
  This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.

In this CL, VideoStreamEncoder no longer calls GetMappedFrameBuffer() on
behalf of the encoders, since the encoders are now able to either do the
mapping or performs ToI420() anyway.

- Tests for old VSE behaviors are updated to test the new behavior (i.e.
  that native frames are pretty much always forwarded).
- The "having to call ToI420() twice" workaround to Android bug
  https://crbug.com/webrtc/12602 is added to H264 and AV1 encoders.

Bug: webrtc:12469
Change-Id: Ibdc2e138d4782a140f433c8330950e61b9829f43
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211940
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33548}
2021-03-24 09:43:11 +00:00
Alessio Bazzica
5cf8c2c501 Fix unspecified time origin for lastPacketReceivedTimestamp
`RTCInboundRtpStreamStats.lastPacketReceivedTimestamp` must be a time
value in milliseconds with Unix epoch as time origin (see
bugs.webrtc.org/12605#c4).

This change fixes both audio and video `RTCInboundRtpStreamStats` stats.

Tested: verified from chrome://webrtc-internals during an appr.tc call

Bug: webrtc:12605
Change-Id: I68157fcf01a5933f3d4e5d3918b4a9d3fbd64f16
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212865
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33547}
2021-03-24 09:36:41 +00:00
Fyodor Kyslov
26abdaf478 AV1: Use Default TX type for encoding
This will further speed up intra frame encoding

Bug: None
Change-Id: I3c836502cdcb1037e3128850a085b92acd8fc7ad
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212821
Reviewed-by: Marco Paniconi <marpan@webrtc.org>
Commit-Queue: Fyodor Kyslov <kyslov@google.com>
Cr-Commit-Position: refs/heads/master@{#33544}
2021-03-23 17:19:27 +00:00
philipel
ca18809ee5 Move RtpFrameObject and EncodedFrame out of video_coding namespace.
Bug: webrtc:12579
Change-Id: Ib7ecd624eb5c54abb77fe08440a014aa1e963865
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212860
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33542}
2021-03-23 14:22:47 +00:00
Jonas Oreland
93ee168671 Allow empty video layer allocation extension
This patch adds support for sending zero video layer allocations
header extensions. This can be used to signal that a stream is
turned off.

Bug: webrtc:12000
Change-Id: Id18fbbff2216ca23179c58ef7bbe2ebea5e242af
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212743
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33541}
2021-03-23 12:36:59 +00:00
Henrik Boström
3889de1c4c Support native scaling of VideoFrameBuffers in LibvpxVp8Encoder.
This is a follow-up to the VP9, fixing VP8 this time. Context again:

This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
- To achieve this, WebRTC encoders are updated to map kNative video
  buffers so that in a follow-up CL VideoStreamEncoder can stop mapping
  intermediate buffer sizes.

Bug: webrtc:12469, chromium:1157072
Change-Id: I026527ae77e36f66d02e149ad6fe304f6a8ccb05
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212600
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33537}
2021-03-23 09:08:58 +00:00
philipel
6a6715042a Move RtpFrameReferenceFinder out of video_coding namespace.
Namespace used because of copy-pasting an old pattern, should never have been used in the first place. Removing it now to make followup refactoring prettier.

Bug: webrtc:12579
Change-Id: I00a80958401cfa368769dc0a1d8bbdd76aaa4ef5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212603
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33536}
2021-03-23 08:48:37 +00:00
Austin Orion
1cdbabde53 Update WgcCaptureSession to handle portrait oriented screen capture.
WgcCaptureSession would crash when copying the frame data for an image
from a portrait oriented monitor. This is because we were using the
height of the image multiplied by the rowpitch of the buffer to
determine the size of the data to be copied. However, in portrait
mode the height measures the same dimension as the rowpitch, leading
to us overrunning the frame buffer.

The fix is to use the height and width of the image multiplied by
the number of bytes per pixel to determine how much data to copy
out of the buffer, and only use the rowpitch to advance the pointer
in the source data buffer. This has the added benefit of giving us
contiguous data, reducing the size of the DesktopFrame that we output.

Bug: webrtc:12490
Change-Id: I4c26f8864cb57ac566a742af70fea1da504b9706
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209501
Reviewed-by: Joe Downing <joedow@chromium.org>
Commit-Queue: Austin Orion <auorion@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#33532}
2021-03-22 20:50:50 +00:00
Henrik Boström
bd9e4a95eb Support native scaling of VideoFrameBuffers in LibvpxVp9Encoder.
This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
- To achieve this, WebRTC encoders are updated to map kNative video
  buffers so that in a follow-up CL VideoStreamEncoder can stop mapping
  intermediate buffer sizes.

In this CL LibvpxVp9Encoder is updated to map kNative buffers of pixel
formats it supports and convert ToI420() if the kNative buffer is
something else. A fake native buffer that keeps track of which
resolutions were mapped, MappableNativeBuffer, is added.

Because VP9 is currently an SVC encoder and not a simulcast encoder, it
does not need to invoke CropAndScale.

This CL also fixes MultiplexEncoderAdapter, but because it simply
forwards frames it only cares about the pixel format when
|supports_augmented_data_| is true so this is the only time we map it.
Because this encoder is not used with kNative in practise, we don't care
to make this path optimal.

Bug: webrtc:12469, chromium:1157072
Change-Id: I74edf85b18eccd0d250776bbade7a6444478efce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33526}
2021-03-22 13:35:35 +00:00
Niels Möller
490c1503d9 Delete unowned buffer in EncodedImage.
Bug: webrtc:9378
Change-Id: Ice48020c0f14905cbc185b52c88bbb9ac3bb4c93
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/128575
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33510}
2021-03-19 14:12:28 +00:00
Fyodor Kyslov
f8776cba62 Revert "AV1: Use Default TX type for encoding"
This reverts commit b0dc518f82b2a50c1b3e3062d3b58ee221d2e028.

Reason for revert: corresponding change  https://chromium-review.googlesource.com/c/chromium/src/+/2765225 from chromium has not been backported to webrtc yet.


Original change's description:
> AV1: Use Default TX type for encoding
>
> This will further speed up intra frame encoding
>
> Bug: None
> Change-Id: I1a105c6d2cdd9dc82f84d0039dbea3f0d090ab93
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212320
> Reviewed-by: Marco Paniconi <marpan@webrtc.org>
> Commit-Queue: Fyodor Kyslov <kyslov@google.com>
> Cr-Commit-Position: refs/heads/master@{#33492}

TBR=jianj@google.com,marpan@webrtc.org,kyslov@google.com

Change-Id: I7ff93537942ab34706db0b71c6b5f8535209619d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212340
Reviewed-by: Andrey Logvin <landrey@webrtc.org>
Commit-Queue: Fyodor Kyslov <kyslov@google.com>
Cr-Commit-Position: refs/heads/master@{#33494}
2021-03-17 19:16:50 +00:00
Fyodor Kyslov
b0dc518f82 AV1: Use Default TX type for encoding
This will further speed up intra frame encoding

Bug: None
Change-Id: I1a105c6d2cdd9dc82f84d0039dbea3f0d090ab93
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212320
Reviewed-by: Marco Paniconi <marpan@webrtc.org>
Commit-Queue: Fyodor Kyslov <kyslov@google.com>
Cr-Commit-Position: refs/heads/master@{#33492}
2021-03-17 18:43:28 +00:00
Erik Språng
f19aec829b Updates ulpfec reader to accept padding on media packets.
Bug: webrtc:12530
Change-Id: I659c430d50a88d49cb4c3c21d00710fac78f1e0d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212081
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33483}
2021-03-16 19:47:09 +00:00
Danil Chapovalov
ab63350411 Delete RtpRtcp::RemoteRTCPStat in favor of GetLatestReportBlockData
Bug: webrtc:10678
Change-Id: I1cff0230208e22f56f26cf2eba976f66d9b5bafc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212020
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33479}
2021-03-16 10:31:35 +00:00
Per Åhgren
19775cbd29 Reland "Reduce complexity in the APM pipeline when the output is not used"
This is a reland of aa6adffba325f4b698a1e94aeab020bfdc47adec

What was changed in the reland is that the merging of the bands is
excluded from the code that is not run when the output is not used.
I.e., the merging is always done.

This is important to have since some clients may apply muting before APM,
and still flag to APM that the signal is muted. If the merging is not
always done, those clients will get nonzero output from APM during muting.


Original change's description:
> Reduce complexity in the APM pipeline when the output is not used
>
> This CL selectively turns off parts of the audio processing when
> the output of APM is not used. The parts turned off are such that
> don't need to continuously need to be trained, but rather can be
> temporarily deactivated.
>
> The purpose of this CL is to allow CPU to be reduced when the
> client is muted.
>
> The CL will be follow by additional CLs, adding similar functionality
> in the echo canceller and the noiser suppressor
>
> Bug: b/177830919
> Change-Id: I72d24505197a53872562c0955f3e7b670c43df6b
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209703
> Commit-Queue: Per Åhgren <peah@webrtc.org>
> Reviewed-by: Sam Zackrisson <saza@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33431}

Bug: b/177830919
Change-Id: Ib74dd1cefa173d45101e26c4f2b931860abc6d08
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211760
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33478}
2021-03-16 09:33:02 +00:00
Per Åhgren
15179a9986 Allowing reduced computations in the noise suppressor when the output is not used
This CL adds functionality in the noise suppressor that allows the
computational complexity to be reduced when the output of APM is not used.

Bug: b/177830919
Change-Id: I849351ba9559fae770e4667d78e38abde5230eed
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211342
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33477}
2021-03-16 09:28:42 +00:00
Per Åhgren
8ee1ec82e4 Allowing reduced computations in the AEC3 when the output is not used
This CL adds functionality in AEC3 that allows the computational
complexity to be reduced when the output of APM is not used.

Bug: b/177830919
Change-Id: I08121364bf966f34311f54ffa5affbfd8b4db1e2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211341
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33476}
2021-03-16 09:16:32 +00:00
Fyodor Kyslov
bff6489c94 AV1: Disable several intra coding tools.
This will speed up key frame encoding (together with libaom changes)
3x-4x times with ~13% BDRate loss on key frames only

Bug: None
Change-Id: I24332f4f7285811cdc6619ba29844fe564cae95e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212040
Reviewed-by: Marco Paniconi <marpan@webrtc.org>
Commit-Queue: Fyodor Kyslov <kyslov@google.com>
Cr-Commit-Position: refs/heads/master@{#33468}
2021-03-15 23:20:08 +00:00
Per Åhgren
db5d728878 Add refined handling of the internal scaling of the audio in APM
This CL adds functionality that allows adjusting the audio levels
internally in APM. The main purpose of the functionality is to allow
APM to optionally be moved to an integration that does not provide an
analog gain to control, and the implementation of this has been
tailored specifically to meet the requirements for that.

More specifically, this CL does
-Add a new variant of the pre-amplifier gain that is intended to replace
 the pre-amplifier gain (but at the moment can coexist with that). The
 main differences with the pre-amplifier gain is that an attenuating
 gain is allowed, the gain is applied jointly with any emulated analog
 gain, and that its packaging fits better with the post gain.
-Add an emulation of an analog microphone gain. The emulation is
 designed to match the analog mic gain functionality in Chrome OS (which
 is digital) but should be usable also on other platforms.
-Add a post-gain which is applied after all processing has been applied.
 The purpose of this gain is for it to work well with the integration
 in ChromeOS, and be used to compensate for the offset that there is
 applied on some USB audio devices.


Bug: b/177830918
Change-Id: I0f312996e4088c9bd242a713a703eaaeb17f188a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209707
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33466}
2021-03-15 19:12:02 +00:00
philipel
b3159517c3 Remove incorrect DCHECKs from LibaomAv1Encoder::SetRates.
Bug: none
Change-Id: I6474418e04538151cfc1588a63e9ffa476e7fd7b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211870
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33465}
2021-03-15 15:20:14 +00:00
Gustaf Ullberg
fdd6099348 Rework transient suppressor configuration in audioproc_f
The transient suppressor can be configured as:
0 - Deactivated
1 - Activated with key events from aecdump
2 - Activated with continuous key events (for debugging purposes)

Bug: webrtc:5298
Change-Id: I116eb08ad50178dc5116d5d967084e6c9967f258
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211869
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33464}
2021-03-15 15:19:09 +00:00
Sergey Silkin
0e42cf703b Reland "Parse encoded frame QP if not provided by encoder"
This reverts commit 727d2afc4330efebc904e0e4f366e885d7b08787.

Reason for revert: Use thread-safe wrapper for H264 parser.

Original change's description:
> Revert "Parse encoded frame QP if not provided by encoder"
>
> This reverts commit 8639673f0c098efc294a7593fa3bd98e28ab7508.
>
> Reason for revert: linux_tsan fails https://ci.chromium.org/ui/p/webrtc/builders/ci/Linux%20Tsan%20v2/25329/overview
>
> Original change's description:
> > Parse encoded frame QP if not provided by encoder
> >
> > Bug: webrtc:12542
> > Change-Id: Ic70b46e226f158db7a478a9f20e1f940804febba
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210966
> > Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> > Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#33434}
>
> TBR=asapersson@webrtc.org,ssilkin@webrtc.org
>
> Change-Id: Ie251d8f70f8e87fd86b63730aefd2ef3f941e4bb
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: webrtc:12542
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211355
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33441}

# Not skipping CQ checks because this is a reland.

Bug: webrtc:12542
Change-Id: Ib7601fd6f2f26bceddbea2b4ba54d67a281f3a59
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211660
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33458}
2021-03-15 10:11:22 +00:00
Per Åhgren
55bc077b45 Add one frame (10 ms) of silence in APM output after unmuting
This CL adds one frame (10 ms) of silence in APM output after unmuting to mask
audio resulting from the turning on the processing that was deactivated
during the muting.

Bug: b/177830919
Change-Id: If44cfb0ef270dde839dcd3f0b98d1c91e81668dd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211343
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33454}
2021-03-13 01:05:45 +00:00
Alessio Bazzica
bc1c93dc6e Add remote-outbound stats for audio streams
Add missing members needed to surface `RTCRemoteOutboundRtpStreamStats`
via `ChannelReceive::GetRTCPStatistics()` - i.e., audio streams.

`GetSenderReportStats()` is added to both `ModuleRtpRtcpImpl` and
`ModuleRtpRtcpImpl2` and used by `ChannelReceive::GetRTCPStatistics()`.

Bug: webrtc:12529
Change-Id: Ia8f5dfe2e4cfc43e3ddd28f2f1149f5c00f9269d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211041
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33452}
2021-03-12 20:39:50 +00:00
Erik Språng
c80f955114 Avoid log spam when decoder implementation changes
A refactoring (https://webrtc-review.googlesource.com/c/src/+/196520)
of decoder metadata handling introduced a bug which causes us to log an
info-level entry for every frame decoded if the implementation changes
during runtime (e.g. due to software fallback).

This CL fixes that to avoid spamming the logs.

Bug: webrtc:12271
Change-Id: I89016351b8752b259299c4cf56c6feddcca43460
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211664
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33451}
2021-03-12 17:12:25 +00:00
philipel
7c7885c016 Remove NTP timestamp from PacketBuffer::Packet.
Bug: webrtc:12579
Change-Id: I64ca0ddb6f5c20bef5e9503955e0e4b4c484a1e7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211662
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33448}
2021-03-12 15:19:35 +00:00
Danil Chapovalov
048e9c2f45 In full svc controller reuse unused frame configuration
vp9 encoder wrapper rely on that behavioue
when generates vp9-specific temporal references

Bug: webrtc:11999
Change-Id: Ie1b4cb60adf290992cc3307b56397a88eda78be4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211661
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33446}
2021-03-12 14:50:39 +00:00
Danil Chapovalov
8da67f6165 In ksvc controller reuse unused frame configuration
vp9 encoder wrapper rely on that behaviour
to generate vp9-specific temporal references

Bug: webrtc:11999
Change-Id: I35536af4eca76450e2f72777e06ad3af872a5800
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211340
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33445}
2021-03-12 13:39:38 +00:00
Sergey Silkin
727d2afc43 Revert "Parse encoded frame QP if not provided by encoder"
This reverts commit 8639673f0c098efc294a7593fa3bd98e28ab7508.

Reason for revert: linux_tsan fails https://ci.chromium.org/ui/p/webrtc/builders/ci/Linux%20Tsan%20v2/25329/overview 

Original change's description:
> Parse encoded frame QP if not provided by encoder
>
> Bug: webrtc:12542
> Change-Id: Ic70b46e226f158db7a478a9f20e1f940804febba
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210966
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33434}

TBR=asapersson@webrtc.org,ssilkin@webrtc.org

Change-Id: Ie251d8f70f8e87fd86b63730aefd2ef3f941e4bb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:12542
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211355
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33441}
2021-03-11 17:06:06 +00:00
Sergey Silkin
8639673f0c Parse encoded frame QP if not provided by encoder
Bug: webrtc:12542
Change-Id: Ic70b46e226f158db7a478a9f20e1f940804febba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210966
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33434}
2021-03-11 11:48:38 +00:00
Ilya Nikolaevskiy
8097935df3 Revert "Reduce complexity in the APM pipeline when the output is not used"
This reverts commit aa6adffba325f4b698a1e94aeab020bfdc47adec.

Reason for revert: breaks webrtc-importer

Original change's description:
> Reduce complexity in the APM pipeline when the output is not used
>
> This CL selectively turns off parts of the audio processing when
> the output of APM is not used. The parts turned off are such that
> don't need to continuously need to be trained, but rather can be
> temporarily deactivated.
>
> The purpose of this CL is to allow CPU to be reduced when the
> client is muted.
>
> The CL will be follow by additional CLs, adding similar functionality
> in the echo canceller and the noiser suppressor
>
> Bug: b/177830919
> Change-Id: I72d24505197a53872562c0955f3e7b670c43df6b
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209703
> Commit-Queue: Per Åhgren <peah@webrtc.org>
> Reviewed-by: Sam Zackrisson <saza@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33431}

Bug: b/177830919
Change-Id: I937cd61dedcd43150933eb1b9d65aebe68401e91
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211348
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33433}
2021-03-11 11:41:20 +00:00
Per Åhgren
aa6adffba3 Reduce complexity in the APM pipeline when the output is not used
This CL selectively turns off parts of the audio processing when
the output of APM is not used. The parts turned off are such that
don't need to continuously need to be trained, but rather can be
temporarily deactivated.

The purpose of this CL is to allow CPU to be reduced when the
client is muted.

The CL will be follow by additional CLs, adding similar functionality
in the echo canceller and the noiser suppressor

Bug: b/177830919
Change-Id: I72d24505197a53872562c0955f3e7b670c43df6b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209703
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33431}
2021-03-11 10:06:58 +00:00
Per Åhgren
34fdc92119 Add audioproc_f support for testing the runtime settings of whether the output is used
Bug: b/177830919
Change-Id: Iddcb79000f471eac165e3f44f14fad41435e6ccb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211241
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33426}
2021-03-10 23:19:30 +00:00
Alessio Bazzica
048adc7136 Add missing remote-outbound stats to RTCPReceiver::NTP
In order to add `RTCRemoteOutboundRtpStreamStats` (see [1]), the
following stats must be added:
- sender's packet count (see [2])
- sender's octet count (see [2])
- total number of RTCP SR blocks sent (see [3])

[1] https://www.w3.org/TR/webrtc-stats/#dom-rtcremoteoutboundrtpstreamstats
[2] https://tools.ietf.org/html/rfc3550#section-6.4.1
[3] https://www.w3.org/TR/webrtc-stats/#dom-rtcremoteoutboundrtpstreamstats-reportssent

Bug: webrtc:12529
Change-Id: I47ac2f79ba53631965d1cd7c1062f3d0f158d66e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210963
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33423}
2021-03-10 16:36:48 +00:00