This CL adds one frame (10 ms) of silence in APM output after unmuting to mask
audio resulting from the turning on the processing that was deactivated
during the muting.
Bug: b/177830919
Change-Id: If44cfb0ef270dde839dcd3f0b98d1c91e81668dd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211343
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33454}
This reverts commit aa6adffba325f4b698a1e94aeab020bfdc47adec.
Reason for revert: breaks webrtc-importer
Original change's description:
> Reduce complexity in the APM pipeline when the output is not used
>
> This CL selectively turns off parts of the audio processing when
> the output of APM is not used. The parts turned off are such that
> don't need to continuously need to be trained, but rather can be
> temporarily deactivated.
>
> The purpose of this CL is to allow CPU to be reduced when the
> client is muted.
>
> The CL will be follow by additional CLs, adding similar functionality
> in the echo canceller and the noiser suppressor
>
> Bug: b/177830919
> Change-Id: I72d24505197a53872562c0955f3e7b670c43df6b
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209703
> Commit-Queue: Per Åhgren <peah@webrtc.org>
> Reviewed-by: Sam Zackrisson <saza@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33431}
Bug: b/177830919
Change-Id: I937cd61dedcd43150933eb1b9d65aebe68401e91
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211348
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33433}
This CL selectively turns off parts of the audio processing when
the output of APM is not used. The parts turned off are such that
don't need to continuously need to be trained, but rather can be
temporarily deactivated.
The purpose of this CL is to allow CPU to be reduced when the
client is muted.
The CL will be follow by additional CLs, adding similar functionality
in the echo canceller and the noiser suppressor
Bug: b/177830919
Change-Id: I72d24505197a53872562c0955f3e7b670c43df6b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209703
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33431}
Currently the echo canceller reference signal is high-pass filtered to
avoid the need of modeling the capture-side high-pass filter as part of
the echo path.
This can lead to the lowest frequency bins of the linear filter
diverging as there is little low-frequency content available for
training. Over time the filter can output an increasing amount of
low-frequency power, which in turn affects the filter's ability to
adapt properly.
Disabling the high-pass filtering of the echo canceller reference solves
this issue, resulting in improved filter convergence.
Bug: webrtc:12265
Change-Id: Ic526a4b1b73e1808cfcd96a8cdee801b96a27671
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208288
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33322}
This remove webrtc-specific macro that has no reason to be webrtc specific
ABSL_DEPRECATED takes a message parameter encouraging to write text how class or function is deprecated.
Bug: webrtc:12484
Change-Id: I89f1398f91dacadc37f7db469dcd985e3724e444
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208282
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33314}
This CL adds the initial support for letting APM know when its output
will be used or not.
It also adds a new method for passing RuntimeInformation to APM that
returns a bool indicating the success of the passing of information.
Bug: b/177830919
Change-Id: Ic2e1b92c37241d74ca6394b785b91736ca7532aa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/206061
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33201}
This CL adds the ability in audioproc_f and unpack_aecdump to:
-Clearly identify the Init events and when those occur.
-Optionally only process a specific Init section of an aecdump.
-Optionally selectively turn on dumping of internal data for a
specific init section, and a specific time interval.
-Optionally let unpack_aecdump produce file names based on inits.
Bug: webrtc:5298
Change-Id: Id654b7175407a23ef634fca832994d87d1073239
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196160
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33181}
We need to be able build chromium with rtc_include_tests = true. It
reveals a lot of targets that are not compatible with chromium but
aren't marked so.
`rtc_include_tests=true` has been considered a way to disable targets for the Chromium build, causing an overload on rtc_include_tests while the meaning of the two GN args (rtc_include_tests and build_with_chromium) should be kept separated.
Bug: webrtc:12404
Change-Id: I2f72825445916eae7c20ef9338672d6a07a9b9ff
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203890
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Andrey Logvin <landrey@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33124}
This CL allows separate dump sets to be used when dumping internal
APM data using audioproc_f, opening up for reducing the amount of
data to be dumped.
Bug: webrtc:5298
Change-Id: I8286933ceed10db074f2064414cc08e8b12653fa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196089
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33047}
- The injection of the AGC2 level estimator into `AgcManagerDirect`
is not used anymore
- `ExperimentalAgc::enabled_agc2_level_estimator` can also be removed
- 3 ctors of `ExperimentalAgc` are unused
- `AgcManagerDirectStandaloneTest::AgcMinMicLevelExperiment` can be
split into separate unit tests (better code clarity)
Bug: webrtc:7494
Change-Id: I5843147c38cf7cb5ee484b0a72fe13dcf363efaf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202025
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33027}
This is a reland of 69241a93fb14f6527a26d5c94dde879013012d2a
Fix: The problem was related to NO_MAIN_THREAD_WRAPPING, which
affects https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/rtc_base/thread.cc;l=257-263;drc=7acc2d9fe3a6e3c4d8881d2bdfc9b8968a724cd5.
The original CL didn't attach the definition of the macro
NO_MAIN_THREAD_WRAPPING when building for Chromium (which doesn't have
to be related to //rtc_base anymore but to //rtc_base:threading).
Original change's description:
> Refactor rtc_base build targets.
>
> The "//rtc_base:rtc_base" build target has historically been one of the
> biggest targets in the WebRTC build. Big targets are the main source of
> circular dependencies and non-API types leakage.
>
> This CL is a step forward into splitting "//rtc_base:rtc_base" into
> smaller targets (as originally started in 2018).
>
> The only non-automated changes are (like re-wiring the build system):
> * The creation of //rtc_base/async_resolver.{h,cc} which allows to
> break a circular dependency (is has been extracted from
> //rtc_base/net_helpers.{h,cc}).
> * The creation of //rtc_base/internal/default_socket_server.{h,cc} to
> break another circular dependency.
>
> Bug: webrtc:9987
> Change-Id: I0c8f5e7efe2c8fd8e6bffa0d6dd2dd494cf3df02
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196903
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32941}
Bug: webrtc:9987
Change-Id: I7cdf49d2aac8357f1f50f90010bf2c2f62fa19f6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202021
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33001}
This reverts commit 69241a93fb14f6527a26d5c94dde879013012d2a.
Reason for revert: Breaks WebRTC roll into Chromium.
Original change's description:
> Refactor rtc_base build targets.
>
> The "//rtc_base:rtc_base" build target has historically been one of the
> biggest targets in the WebRTC build. Big targets are the main source of
> circular dependencies and non-API types leakage.
>
> This CL is a step forward into splitting "//rtc_base:rtc_base" into
> smaller targets (as originally started in 2018).
>
> The only non-automated changes are (like re-wiring the build system):
> * The creation of //rtc_base/async_resolver.{h,cc} which allows to
> break a circular dependency (is has been extracted from
> //rtc_base/net_helpers.{h,cc}).
> * The creation of //rtc_base/internal/default_socket_server.{h,cc} to
> break another circular dependency.
>
> Bug: webrtc:9987
> Change-Id: I0c8f5e7efe2c8fd8e6bffa0d6dd2dd494cf3df02
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196903
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32941}
TBR=mbonadei@webrtc.org,hta@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
No-Try: True
Bug: webrtc:9987
Change-Id: I1e36ad64cc60092f38d6886153a94f1a58339256
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/201840
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32986}
residual_echo_detector and level_estimation is not part of gain_controller2
in AudioProcessing::Config.
Bug: None
Change-Id: Ic31b8c6bb4d9ed59b4f337e5dc8b8c587ae54202
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/200924
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32972}
The "//rtc_base:rtc_base" build target has historically been one of the
biggest targets in the WebRTC build. Big targets are the main source of
circular dependencies and non-API types leakage.
This CL is a step forward into splitting "//rtc_base:rtc_base" into
smaller targets (as originally started in 2018).
The only non-automated changes are (like re-wiring the build system):
* The creation of //rtc_base/async_resolver.{h,cc} which allows to
break a circular dependency (is has been extracted from
//rtc_base/net_helpers.{h,cc}).
* The creation of //rtc_base/internal/default_socket_server.{h,cc} to
break another circular dependency.
Bug: webrtc:9987
Change-Id: I0c8f5e7efe2c8fd8e6bffa0d6dd2dd494cf3df02
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196903
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32941}
Results: RNN VAD realtime factor improved from 140x to 195x (+55x)
Test device: Pixel 2 XL
Benchmark setup: max clock speed forced on all the cores by
setting "performance" as scaling governor
Bug: webrtc:10480
Change-Id: I3e92f643853ad1fe990db909c578ce78ee826c03
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198842
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32888}
This change just affects the ERLE metric that is reported. The rest is unaffected and bitexact.
Bug: webrtc:12280
Change-Id: I2d28ef14a9b704c83aba18b624f67671eec4a042
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196649
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Jesus de Vicente Pena <devicentepena@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32809}
The implementations for the fully connected layer can be simlpified by
using `VectorMath:DotProduct()`. In this way, it is also possible to
remove (nearly) duplicated SIMD code, reduce the binary size and more
easily maintain the code.
This CL also forces unoptimized code for the output layer of the VAD,
which is a FC 24x1 layer. A slight improvement of the realtime has
been measured (delta ~ +5x).
Bug: webrtc:10480
Change-Id: Iee93bd59f7905ebf96275dbbfeb3c921baf4e8db
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195580
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32806}
Using `VectorMath::DotProduct()` in GatedRecurrentLayer to reuse existing
SIMD optimizations. Results:
- When SSE2/AVX2 is avilable, the GRU layer takes 40% of the unoptimized
code
- The realtime factor for the VAD improved as follows
- SSE2: from 570x to 630x
- AVX2: from 610x to 680x
This CL also improved the GRU layer benchmark by (i) benchmarking a GRU
layer havibng the same size of that used in the VAD and (ii) by prefetching
a long input sequence.
Bug: webrtc:10480
Change-Id: I9716b15661e4c6b81592b4cf7c172d90e41b5223
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195545
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32803}
After the refined filter has been determined to perform better than
the coarse filter, and the coefficients of the coarse filters are
overwritten by the ones from the refined filter, at least 100 ms have
to pass before the adaptation of the refined filter is allowed to speed
up due to good coarse filter performance.
This change solves the vicious circle described in webrtc:12265, where
the coarse and refined filters can diverge over time.
This feature can be disabled remotely via a kill-switch. When disabled
the AEC output is bit-exact to before the change.
Bug: webrtc:12265,chromium:1155477
Change-Id: Iacd6e325e987dd8a475bb3e8163fee714c65b20a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196501
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32801}
This CL makes the static variable kChunkSizeMs constexpr to avoid
potential issues on some compilers.
Bug: None
Change-Id: I8e2b1819561ec96fb17d3899af95405cc36a6097
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196640
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32795}
Handle the case in which neither WEBRTC_ARCH_X86_FAMILY nor
WEBRTC_HAS_NEON are defined.
Bug: webrtc:10480
Change-Id: I241583911d8e5645dfbd39b60337dd20b2d9f046
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196525
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32786}
This will soon become a compile-time error. Fix class hierarchies that
wrap StrictMock in a NiceMock or vice-versa by removing redundant
wrappings and removing inheritance from Nice/StrictMock and fixing the
call sites as appropriate.
Bug: b/173702213
Change-Id: Ic90b1f270c180f7308f40e52e358a8f6a6baad86
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196461
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32783}
Refactoring done to more easily and cleanly add SIMD optimizations and
to remove `GatedRecurrentLayer` from the RNN VAD api.
Bug: webrtc:10480
Change-Id: Ie1dffdd9b19c57c03a0b634f6818c0780456a66c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195445
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32770}
Refactoring done to more easily and cleanly add SIMD optimizations and
to remove `FullyConnectedLayer` from the RNN VAD api.
Minor improvements (readability, API):
- `FullyConnectedLayer` gets the ActivationFunction enum and not
a function view anymore
- SSE2 optimization moved into `FullyConnectedLayer::ComputeOutputSse2`
- layer name added for improved logs
Bug: webrtc:10480
Change-Id: Ida4903a67655e19ef0464f378c433c1f6e96dca7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195444
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32766}
Plus a few minor code readability improvements.
Bug: webrtc:10480
Change-Id: I590d8e203b1d05959a8c15373841e37abe83237e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195334
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32764}
The HMM based transparent mode classifier is disabled until an issue
with diverging filters is resolved.
Bug: chromium:1155071
Change-Id: Iee249869f6ece1e48e834b3a4b9249c69a51286c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196341
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32756}
This CL brings a large improvement to the RNN VAD CPU performance
by finally using `VectorMath::DotProduct()` for pitch search.
The realtime factor improved from about 390x to 570x for SSE2
(+180x, 45% faster) and to 610x for AVX2 (+235x, 60% faster).
RNN VAD benchmark results:
```
+-----+-------+------+------+
| run | none* | SSE2 | AVX2 |
+-----+-------+------+------+
| 1 | 393x | 572x | 618x |
| 2 | 388x | 568x | 607x |
| 3 | 393x | 564x | 599x |
+-----+-------+------+------+
```
*: baseline, no SIMD used for pitch search, but SSE2 used for the RNN
Results obtained as follows:
1. Force SSE2 in `DISABLED_RnnVadPerformance` for the RNN part in
order to measure the baseline correctly:
```
RnnBasedVad rnn_vad({/*sse2=*/true, /*avx2=*/true, /*neon=*/false});
```
2. Run the test:
```
$ ./out/release/modules_unittests \
--gtest_filter=*RnnVadTest*DISABLED_RnnVadPerformance* \
--gtest_also_run_disabled_tests --logs
```
Bug: webrtc:10480
Change-Id: I89a2bd420265540026944b9c0f1fdd4bfda7f475
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195001
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32755}
This CL adds a new library for the RNN VAD that provides (optimized)
vector math ops. The scheme is the same of the `VectorMath` class of AEC3
to ensure correct builds across different platforms.
Bug: webrtc:10480
Change-Id: I96bcfbf930ca27388ab5f2d52c022ddb73acf8e6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/194326
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Per Åhgren <peah@webrtc.org>
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32741}
In preparation for adding AVX2 code, a safe scheme to support
different SIMD optimizations is added.
Safety features:
- AVX2 kill switch to stop using it even if supported by the
architecture
- struct indicating the available CPU features propagated from
AGC2 to each component; in this way
- better control over the unit tests
- no need to propagate individual kill switches but just
set to false features that are turned off
Note that (i) this CL does not change the performance of the RNN VAD
and (ii) no AVX2 optimization is added yet.
Bug: webrtc:10480
Change-Id: I0e61f3311ecd140f38369cf68b6e5954f3dc1f5a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193140
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32739}
