This makes sense since the buffered data is only used by
the echo subtraction method. Furthermore, it simplifies the
upcoming modifications to the echo subtraction method since
the way the buffering is done can then be specific for the
echo subtraction implementation used.
The change is bitexact and this was verified using a fairly
extensive bitexactness suite.
BUG=
Review URL: https://codereview.webrtc.org/1639773002
Cr-Commit-Position: refs/heads/master@{#11547}
The level of the error signal after linear echo cancellation was based on non-buffered signal while that of the near-end and far-end signal based on buffered signal. This discrepancy made the comparison of them unfair.
This CL is to make calculating the error level rely on the same buffering.
BUG=
Review URL: https://codereview.webrtc.org/1510873004
Cr-Commit-Position: refs/heads/master@{#11408}
In AEC, audio levels are calculated in frequency domain. This makes the calculation dependent on FFT. We now make the calculation performed in time domain. The complexity is the same, but the dependence on FFT is removed.
BUG=
Review URL: https://codereview.webrtc.org/1542573002
Cr-Commit-Position: refs/heads/master@{#11357}
Macro incorrectly displays DISABLED_ON_ANDROID in test names for
parameterized tests under --gtest_list_tests, causing tests to be
disabled on all platforms since they contain the DISABLED_ prefix rather
than their expanded variants.
This expands the macro variants to inline if they're disabled or not,
and removes building some tests under configurations where they should
fail, instead of building them but disabling them by default.
The change also removes gtest_disable.h as an unused include from many
other files.
BUG=webrtc:5387, webrtc:5400
R=kjellander@webrtc.org, phoglund@webrtc.orgTBR=henrik.lundin@webrtc.org
Review URL: https://codereview.webrtc.org/1547343002 .
Cr-Commit-Position: refs/heads/master@{#11150}
Currently, FFT is performance when AEC buffers farend signal. This has some drawbacks
1. memory inefficiency: two ring buffers are needed;
2. computation inefficiency: if ringbuffer gets wrapped around, some FFT computation will be wasted;
3. accessibility: the main AEC function looses accessibility to the time-domain signal.
Therefore, this CL tries to buffer time domain data, which is buffered any way if a debugging macro is defined, and calculate the FFTs where they are actually used.
BUG=
Review URL: https://codereview.webrtc.org/1512573003
Cr-Commit-Position: refs/heads/master@{#11091}
-Extended the InverseFft function to be more generally
applicable.
-Included the previous external extra scaling into the
preexisting InverseFft call.
-Moved the updating of aec->delayEstCtr to where it is
actually used.
-Refactored the output production and comfort noise
addition using the InverseFft function.
-Removed the if-statements checking the value of the
constant flagHbandCn as any value different from 1 would
crash the program. Also removed the constant
The changes have been tested for bitexactness.
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1492343002
Cr-Commit-Position: refs/heads/master@{#11054}
-Changed the type for the frequency estimate of the comfort noise for the
higher band to be a two dimensional float array instead of a complex_t array.
This makes sense since all the other frequency estimate (apart from the
coherence) use this format and doing this change allows bundling the
IFFT operations into using the InverseFFT method.
-Moved the memset of the frequency estimate of the comfort noise to where it is used and made it conditional so that it is only performed when used.
-Harmonized the if-statements for when the frequency estimate of the comfort noise is computed in the different optimized ComfortNoise computation methods.
The changes have been tested for bitexactness.
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1494133002
Cr-Commit-Position: refs/heads/master@{#11050}
-Moved memsets to where their variables are used.
-Removed redundant.
-Changed a pointer scalar to be accessed in pointer notation rather than
in array notation.
The change has been tested for bitexactness.
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1494473006
Cr-Commit-Position: refs/heads/master@{#10963}
-Moved filter reset from the echo suppression
into the echo subtraction code where it belongs
(the echo subtractor should own its filter reset).
-Moved the selection between using the microphone sinal and
the echo subtractor output down to the lowest level in the
EchoSuppression function. This makes sense as that selection
was very hidden in an unrelated sub-sub-function call and
as the selection is critical for what the AEC outputs.
The changes have been tested for bitexactness.
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1499573003
Cr-Commit-Position: refs/heads/master@{#10956}
to simplify future refactoring and development.
In more detail:
1) Moved the updating of eBuf from the EchoSubtraction method
to the EchoSuppression method as it is only used in the latter.
2) Moved the computation of efw and dfw from the SubbandCoherence method
as those are actually the analysis filterbank computation that is not
directly related to the coherence.
3) As a consequence of 2) 3 functions needed to be replaced by the
generic function pointer scheme used in WebRTCAec as they have
optimized versions for SSE2 and NEON (which before were local to each
of the aec_core*.c files.
Motivation:
Apart from making sense from a logical point of view, the changes will
a) Allow eBuf stored in half the size on the state.
b) Allow simpler switching between using the the microphone signal
and echo subtractor output in the echo suppressor.
c) Allow further refactoring that move all the changes to eBuf to one method
(currently those are happening in at least 4 different methods.
Drawbacks:
i) dfw is moved to EchoSuppression which increases the stack usage for that
method. This will, however, be improved once further refactoring can be done.
The changes have been tested for bitexactness on Linux using a quite extensive dataset.
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1494563002
Cr-Commit-Position: refs/heads/master@{#10954}
-Renamed the TimeToFrequency and FrequencyToTime functions.
-Moved the windowing from the TimeToFrequency function.
-Simplified the EchoSubtraction function.
Note that the aec state is still an input to the EchoSubtraction function, and it currently needs to be that in order to support the output of the debug file. The longer-term goal is, however, to order the state into substates. This will simplify the parameter lists to the EchoCancellation function as well as replace the aec state as a parameter
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1456123003
Cr-Commit-Position: refs/heads/master@{#10830}
This works aims to:
-More clearly separate the functionalities in the AEC.
-Make the inputs and outputs to functions more clear (currently the state struct is often passed as a parameter to the functions and the functions use members of the state both as inputs and outputs, which reduces the readability of the code and makes it difficult to change/refactor.
What is done in this CL:
-Most of what belongs to the echo subtraction functionality has been moved to a separate function.
-The NonLinearProcessing function has been renamed to EchoSuppressor which I think is more appropriate.
-Part of the code was replaced by a call to the TimeToFrequency function (which was also suggested by an existing todo).
-For consistency, a function FrequencyToTime doing the opposite of TimeToFrequency was added and part of the code was moved to that.
-The ScaleErrorSignal function was changed to no longer have the state as an input parameter. This entailed also changing the corresponding assembly optimized files accordingly.
Testing:
-The changes have been tested for bitexactness on Linux using a fairly extensive test.
-All the unittests pass on linux.
BUG=webrtc:5201
Review URL: https://codereview.webrtc.org/1455163006
Cr-Commit-Position: refs/heads/master@{#10764}
-Made the component error messages generic to be an unspecified error message.
BUG=webrtc:5099
Review URL: https://codereview.webrtc.org/1404743003
Cr-Commit-Position: refs/heads/master@{#10570}
Replaced the wav file dumping functionality in aec_core.c with the newly added corresponding macros
Added macros for logging of AEC internal data
BUG=
Review URL: https://codereview.webrtc.org/1272403003
Cr-Commit-Position: refs/heads/master@{#9808}
We use this Config struct for enabling/disabling the delay agnostic
AEC. This change renames it to DelayAgnostic for readability reasons.
NOTE: The logic is reversed in this CL. The old ReportedDelay config
turned DA-AEC off, while the new DelayAgnostic turns it on.
The old Config is kept in parallel with the new during a transition
period. This is to avoid problems with API breakages. During this
period, ReportedDelay is disabled or DelayAgnostic is enabled, DA-AEC
is engaged in APM.
BUG=webrtc:4651
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1211053006
Cr-Commit-Position: refs/heads/master@{#9531}
Original review at https://codereview.webrtc.org/1180423006
SystemDelayTests was not updated w.r.t. extended_filter mode and some tests were disabled on Android since DA-AEC is automatically set.
All tests have now been updated for both extended_filter mode as well as DA-AEC, hence are now enabled on Android.
Also
* Moves default settings of extended_filter and DA-AEC form Init() to Create() to avoid unintentional loss of state during a reset.
* Fixes a potential bug of starting from scratch in extended_filter mode + DA-AEC.
This reverts commit 01c9b012e9171c813ace9e405c32fc75f4262bf6.
BUG=
R=henrik.lundin@webrtc.org
Review URL: https://codereview.webrtc.org/1187943005.
Cr-Commit-Position: refs/heads/master@{#9458}
The code only affects DA-AEC, but since DA-AEC is the default AEC if run on Android tests failed. Reverting to fix that test.
This reverts commit 9002cc426dab7a576f5247f45ba888cd081a39f0.
BUG=
TBR=henrik.lundin@webrtc.org
Review URL: https://codereview.webrtc.org/1183243003.
Cr-Commit-Position: refs/heads/master@{#9453}
We've seen that if we get a buffer underrun followed by a sudden buffer build up the DA-AEC can't really catch up even though it should be possible to estimate the upcoming difference. We have a feature for this already, but that is only used in the regular AEC. This CL turns that feature on also for DA-AEC.
- Adds a helper function MoveFarReadPtrWithoutSystemDelayUpdate()
- Only apply conservative correction for positive delays, where we can put the AEC into a non-causal state
- Stuff the farend buffer if we don't have enough data to process w.r.t. to current nearend buffer.
- Always run delay estimation based on reported delays to catch buffer starvation.
BUG=
R=henrik.lundin@webrtc.org
Review URL: https://codereview.webrtc.org/1180423006.
Cr-Commit-Position: refs/heads/master@{#9452}
This includes changes like:
* Attempt to break lines at better positions
* Use "override" in more places, don't use "virtual" with it
* Use {} where the body is more than one line
* Make declaration and definition arg names match
* Eliminate unused code
* EXPECT_EQ(expected, actual) (but use (actual, expected) for e.g. _GT)
* Correct #include order
* Use anonymous namespaces in preference to "static" for file-scoping
* Eliminate unnecessary casts
* Update reference code in comments of ARM assembly sources to match actual current C code
* Fix indenting to be more style-guide compliant
* Use arraysize() in more places
* Use bool instead of int for "boolean" values (0/1)
* Shorten and simplify code
* Spaces around operators
* 80 column limit
* Use const more consistently
* Space goes after '*' in type name, not before
* Remove unnecessary return values
* Use "(var == const)", not "(const == var)"
* Spelling
* Prefer true, typed constants to "enum hack" constants
* Avoid "virtual" on non-overridden functions
* ASSERT(x == y) -> ASSERT_EQ(y, x)
BUG=none
R=andrew@webrtc.org, asapersson@webrtc.org, henrika@webrtc.org, juberti@webrtc.org, kjellander@webrtc.org, kwiberg@webrtc.org
Review URL: https://codereview.webrtc.org/1172163004
Cr-Commit-Position: refs/heads/master@{#9420}
(This reverts commit 3fbf3f8841b5460503fb646eaedcb063620434a8.)
The original submission was reverted because it broke the Chrome build. This is fixed in patch set 2 of this change by keeping the old MediaConstraintsInterface string kExperimentalEchoCancellation. It will be removed once the Chrome code has been updated.
Original description:
"We use this Config struct for enabling/disabling Extended filter mode in AEC. This change renames it to ExtendedFilter for readability reasons. The corresponding media constraint is also renamed to kExtendedFilterEchoCancellation.
The old Config is kept in parallel with the new during a transition period. This is to avoid problems with API breakages. During this period, if any of the two Configs are enabled, the extended filter mode is engaged in APM. That is, the two Configs are combined with an "OR" operation.
This change also renames experimental_aec in AudioOptions to extended_filter_aec."
BUG=webrtc:4696
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1151573021.
Cr-Commit-Position: refs/heads/master@{#9401}
We use this Config struct for enabling/disabling Extended filter mode
in AEC. This change renames it to ExtendedFilter for readability
reasons. The corresponding media constraint is also renamed to
kExtendedFilterEchoCancellation.
The old Config is kept in parallel with the new during a transition
period. This is to avoid problems with API breakages. During this
period, if any of the two Configs are enabled, the extended filter
mode is engaged in APM. That is, the two Configs are combined with an
"OR" operation.
This change also renames experimental_aec in AudioOptions to extended_filter_aec.
BUG=4696
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/54659004
Cr-Commit-Position: refs/heads/master@{#9378}
The delay agnostic AEC uses a signal based delay correction method to adjust buffer synchronization between loudspeaker and microphone. On Mac in particular we have seen deviations in UMA stats that point towards an echo already at startup. This is likely due to an early and incorrect correction based on poor audio data.
By waiting 15 seconds before we turn on the ability to correct we can avoid a majority of these.
The reported delay values are in general accurate enough and relying on them in the beginning is fine. The value 15 seconds is chosen because we have seen from UMA data that a significant amount of calls tend to end before 15 seconds when being in the UseDelayAgnosticAEC Finch experiment.
We turn this "feature" on for all platforms but Android, where the reported system delays are inaccurate and we want to take action as soon as possible.
In addition, the set of "good" delay values has been increased from 25% to 75% of the filter length.
BUG=webrtc:3504
R=henrik.lundin@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/50199004
Cr-Commit-Position: refs/heads/master@{#9376}
Merge WEBRTC_ARCH_ARM64_NEON and WEBRTC_ARCH_ARM_NEON into one
WEBRTC_HAS_NEON.
Replace WEBRTC_DETECT_ARM_NEON by WEBRTC_DETECT_NEON.
Replace WEBRTC_ARCH_ARM by WEBRTC_ARCH_ARM64 for arm64 cpu.
BUG=4002
R=andrew@webrtc.org, jridges@masque.com, kjellander@webrtc.org
Change-Id: I870a4d0682b80633b671c9aab733153f6d95a980
Review URL: https://webrtc-codereview.appspot.com/49309004
Cr-Commit-Position: refs/heads/master@{#9228}
When running AEC in extended_filter mode there is no startup phase to evaluate the reported system delay values.
Instead we simply use the first value and scale by two to avoid over compensating when synchronizing render and capture.
We don't need to be too accurate since we have extended the filter length.
On Android we use fixed (measured) reported delay values.
There is no need to be extra conservative here, because that is already built-in in the measured value.
In fact, the difference between devices is large and with such an extra conservative approach the true delay can not be caught by the filter length.
With this change we can improve performance on some devices.
BUG=4472
TESTED=offline on recordings from various devices
R=kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/49909004
Cr-Commit-Position: refs/heads/master@{#9144}
In AEC a fixed fft size is used, but processing can in the lower band be in either 8 or 16 kHz.
Therefore we need a multiplier/rate factor to, for example, map frequency bands in Hz to frequency bins.
The multiplier/rate factor can only be either 1 or 2, but when 48 kHz support was added it was assigned 3.
BUG=crbug.com/482424
R=aluebs@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/43329004
Cr-Commit-Position: refs/heads/master@{#9117}
There is no point in returning an error when Free() fails. In fact it can only happen if we have a null pointer as object. There is further no place where the return value is used.
Affected components are
- aec
- aecm
- agc
- ns
BUG=441
R=kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/50579004
Cr-Commit-Position: refs/heads/master@{#8966}
If built-in Echo Cancellation is available on a device it is automatically enabled. The reason is that it in most cases performs better than the WebRTC software echo control for mobile. The drawback is that we can not develop, test and rollout the delay agnostic AEC (DA-AEC) on Android as for desktops.
This CL includes
- adding a media constraint to enable/disable DA-AEC.
- automatically turning on echo cancellation if DA-AEC is enabled.
- a fix in the AEC that enables delay estimation when DA-AEC is enabled, but delay metrics is disabled.
- sets the Config struct ReportedDelay, which controls DA-AEC internally in the AEC.
The test code to verify that it works in AppRTCDemo can be found here:
https://webrtc-codereview.appspot.com/50479004/
BUG=4472
TESTED=locally on N7, N6, Android One
R=glaznev@webrtc.org, perkj@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/48699004
Cr-Commit-Position: refs/heads/master@{#8861}
The known clients (GetStats and UMA histogram in Chrome) use at least 5 second aggregation window. There is no particular value in calculating the metrics more often.
The CL also includes a small refactoring moving a declaration inside an if statement.
BUG=2994
TEST=N/A
R=kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/40219004
Cr-Commit-Position: refs/heads/master@{#8619}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8619 4adac7df-926f-26a2-2b94-8c16560cd09d
There exist devices with runtime checks for NEON, but where the device is not NEON. One such device is Tegra2 on which currently NEON code is running.
This fix adds a missing feature check when initializing the AEC.
BUG=4304
R=kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/42159004
Cr-Commit-Position: refs/heads/master@{#8559}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8559 4adac7df-926f-26a2-2b94-8c16560cd09d
To more easily determine if for example the AEC is not working properly one could monitor how often the estimated delay is out of bounds. With out of bounds we mean either being negative or too large, where both cases will break the AEC.
A new delay metric is added telling the user how often poor delay values were estimated. This is measured in percentage since last time the metrics were calculated.
All APIs have been updated with a third parameter with EchoCancellation::GetDelayMetrics() giving the option to exclude the new metric not to break existing code.
The new metric has been added to audio_processing_unittests with an additional protobuf member, and reference files accordingly updated.
voe_auto_test has not been updated to display the new metric.
BUG=4246
TESTED=audioproc on files
R=aluebs@webrtc.org, andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/39739004
Cr-Commit-Position: refs/heads/master@{#8230}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8230 4adac7df-926f-26a2-2b94-8c16560cd09d
Previously, the delay estimate history was reset every time the metrics were pulled. This required all clients to be on the same thread and make use of one call.
Now we use a fixed aggregation window of one second and when a client pulls the metrics you get the latest value.
Under certain circumstances like tests you would like to have the aggregation window set to the recording length. We therefore turn on the fixed aggregation window after the first call.
BUG=2994
TESTED=locally on Mac and trybots
R=kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/38759004
Cr-Commit-Position: refs/heads/master@{#8170}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8170 4adac7df-926f-26a2-2b94-8c16560cd09d
Doing something similar for the band 16-24kHz to what is done for the band 8-16kHz. The only difference is that there is no comfort noise added in this band. Could not test how this sounds because there are no aecdumps with 48kHz sample rate as nfar as I know.
Tested for 32kHz sample rate and the output is bitexact with how it was before this CL.
BUG=webrtc:3146
R=andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/28319004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8080 4adac7df-926f-26a2-2b94-8c16560cd09d
On non-Android the delay estimator in audio_processing/aec has solely been used for logging purposes. The maximum possible observed delay has been 236 ms. We have seen longer delays for which the delay estimate at best ends up at 236 ms, but can also be 'random'. reported delays are clamped to 500 ms.
This cl extends the delay estimation window to match that.
BUG=4086, 3504, 4113
TESTED=locally on Linux and trybots
R=kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/36569004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@7989 4adac7df-926f-26a2-2b94-8c16560cd09d
