Adds two new stats to RTCMediaStreamTrackStats:
* totalSamplesReceived is the total number of samples received on
the audio channel and includes real and synthetic samples.
* concealedSamples is the total number of synthetic samples
received on the audio channel used to conceal packet loss.
Bug: webrtc:8076
Change-Id: I36e9828525ec341490cf3310a972b56a8b443667
Reviewed-on: https://chromium-review.googlesource.com/615902
Commit-Queue: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Taylor Brandstetter <deadbeef@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19506}
In practice, this change will make AudioFrame::muted_ replicate the
explicit muted variable, passed as a pointer to NetEq::GetAudio.
BUG=webrtc:7944
Review-Url: https://codereview.webrtc.org/2965203002
Cr-Commit-Position: refs/heads/master@{#18914}
I used a command like this to update the paths:
perl -pi -e "s/webrtc\/base/webrtc\/rtc_base/g" `find webrtc/rtc_base -name "*.cc" -o -name "*.h"`
The only manual edit is to add an include of webrtc/rtc_base/checks.h in
webrtc/modules/audio_device/android/opensles_common.h, which likely
was needed due to changed include paths due to 'git cl format'.
BUG=webrtc:7634
NOTRY=True
NOPRESUBMIT=True
Review-Url: https://codereview.webrtc.org/2969653002
Cr-Commit-Position: refs/heads/master@{#18871}
LastDecoderError was only used in tests. LastError was only used in
conjunction with RemovePayloadType, and always to distinguish between
"decoder not found" and "other error". In AcmReceiver, "decoder not
found" was not treated as an error.
With this change, calling NetEq::RemovePayloadType with a payload type
that is not registered is no longer considered to be an error. This
allows to rewrite the code in AcmReceiver, such that it no longer has
to call LastError.
The internal member variables NetEqImpl::error_code_ and
NetEqImpl::decoder_error_code_ are removed, since they were no longer
read.
Bug: none
Change-Id: Ibfe97265954a2870c3caea4a34aac958351d7ff1
Reviewed-on: https://chromium-review.googlesource.com/535533
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#18588}
const int16_t* data() const;
int16_t* mutable_data();
- data() returns a zeroed static buffer on muted frames (to avoid unnecessary zeroing of the member buffer) and directly returns AudioFrame::data_ on unmuted frames.
- mutable_data(), lazily zeroes AudioFrame::data_ if the frame is currently muted, sets muted=false, and returns AudioFrame::data_.
These accessors serve to "force" callers to be aware of the mute state field, i.e. lazy zeroing is not the primary motivation.
This change only optimizes handling of muted frames where it is somewhat trivial to do so. Other improvements requiring more significant structural changes will come later.
BUG=webrtc:7343
TBR=henrika
Review-Url: https://codereview.webrtc.org/2750783004
Cr-Commit-Position: refs/heads/master@{#18543}
RTP packets can be padded with extra data at the end of the payload. The usable
payload length of the packet should then be reduced with the padding length,
since the padding must be discarded. This was not the case; instead, the entire
payload, including padding data, was forwarded to the audio channel and in the
end to the decoder.
A special case of padding is packets which are empty except for the padding.
That is, they carry no usable payload. These packets are sometimes used for
probing the network and were discarded in
RTPReceiverAudio::ParseAudioCodecSpecific. The result is that NetEq never sees
those empty packets, just the holes in the sequence number series; this can
throw off the target buffer calculations.
With this change, the empty (after removing the padding) packets are let through,
all the way down to NetEq, to a new method called NetEq::InsertEmptyPacket. This
method notifies the DelayManager that an empty packet was received.
BUG=webrtc:7610, webrtc:7625
Review-Url: https://codereview.webrtc.org/2870043003
Cr-Commit-Position: refs/heads/master@{#18083}
After a Merge operation, the statistics for number of samples
generated using Expand must be corrected, and the correction can in
fact be negative. However, a bug was introduced in
https://codereview.webrtc.org/1230503003 which uses a size_t to
represent the correction, which leads to wrap-around of the negative
value. This is not a problem in itself, since this value is added to
another size_t, with the effect that the desired subtraction happens
anyway.
The actual problem arises if the statistics are polled/reset before a
subtraction happens -- that is, between an Expand and a Merge
operation. This will lead to an actual wrap-around of the stats value,
and large expand_rate (16384) is reported.
BUG=webrtc:7554
Review-Url: https://codereview.webrtc.org/2859483005
Cr-Commit-Position: refs/heads/master@{#18029}
This CL adds a few methods to the NetEq API that will be used for
delay analysis and plotting.
BUG=webrtc:7467
Review-Url: https://codereview.webrtc.org/2839163002
Cr-Commit-Position: refs/heads/master@{#17889}
Reason for revert:
Downstream roadblock should be cleared by now. Relanding original patch.
Original issue's description:
> Revert of Change NetEq::InsertPacket to take an RTPHeader (patchset #2 id:20001 of https://codereview.webrtc.org/2807273004/ )
>
> Reason for revert:
> Broke downstream dependencies.
>
> Original issue's description:
> > Change NetEq::InsertPacket to take an RTPHeader
> >
> > It used to take a WebRtcRTPHeader as input, which has an RTPHeader as
> > a member. None of the other member in WebRtcRTPHeader where used in
> > NetEq.
> >
> > This CL adapts the production code; tests and tools will be converted
> > in a follow-up CL.
> >
> > BUG=webrtc:7467
> >
> > Review-Url: https://codereview.webrtc.org/2807273004
> > Cr-Commit-Position: refs/heads/master@{#17652}
> > Committed: 4d027576a6
>
> TBR=ivoc@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:7467
>
> Review-Url: https://codereview.webrtc.org/2812933002
> Cr-Commit-Position: refs/heads/master@{#17657}
> Committed: 10d095d4f7R=ivoc@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_rel_ng
BUG=webrtc:7467
Review-Url: https://codereview.webrtc.org/2835093002 .
Cr-Commit-Position: refs/heads/master@{#17843}
Reason for revert:
Broke downstream dependencies.
Original issue's description:
> Change NetEq::InsertPacket to take an RTPHeader
>
> It used to take a WebRtcRTPHeader as input, which has an RTPHeader as
> a member. None of the other member in WebRtcRTPHeader where used in
> NetEq.
>
> This CL adapts the production code; tests and tools will be converted
> in a follow-up CL.
>
> BUG=webrtc:7467
>
> Review-Url: https://codereview.webrtc.org/2807273004
> Cr-Commit-Position: refs/heads/master@{#17652}
> Committed: 4d027576a6TBR=ivoc@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:7467
Review-Url: https://codereview.webrtc.org/2812933002
Cr-Commit-Position: refs/heads/master@{#17657}
It used to take a WebRtcRTPHeader as input, which has an RTPHeader as
a member. None of the other member in WebRtcRTPHeader where used in
NetEq.
This CL adapts the production code; tests and tools will be converted
in a follow-up CL.
BUG=webrtc:7467
Review-Url: https://codereview.webrtc.org/2807273004
Cr-Commit-Position: refs/heads/master@{#17652}
Reason for revert:
Makes perf and Chromium FYI bots unhappy.
Original issue's description:
> WebRtcVoiceMediaChannel::AddRecvStream: Don't call SetRecPayloadType
>
> This removes one more place where we were unable to handle codecs not
> in the built-in set.
>
> BUG=webrtc:5805
>
> Review-Url: https://codereview.webrtc.org/2686043006
> Cr-Commit-Position: refs/heads/master@{#17370}
> Committed: 1724cfbdbaTBR=ossu@webrtc.org,solenberg@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5805
Review-Url: https://codereview.webrtc.org/2772043002
Cr-Commit-Position: refs/heads/master@{#17374}
This removes one more place where we were unable to handle codecs not
in the built-in set.
BUG=webrtc:5805
Review-Url: https://codereview.webrtc.org/2686043006
Cr-Commit-Position: refs/heads/master@{#17370}
different sample rate frequency.
BUG=webrtc:7327
Problems before the fix:
1. NetEqImpl::timestamp_ is inconsistent. Initially it is set to
the original RTP timestamp, but later gets updated with the
scaled timestamp.
2. NetEqImpl::InsertPacketInternal::main_timestamp is set with
the original RTP timestamp, but later gets compared with the
NetEqImpl::timestamp_ which may or may not be with the same
sample rate frequency and this results in major problems.
3. IncreaseEndTimestamp(main_timestamp - timestamp_) will be
incorrect when SSRC is changed and not the first packet.
4. delay_manager_->Update() may not be always invoked, since
the (main_timestamp - timestamp_) >= 0 will not be true when
the previous scaled timestamp_ is bigger than the main_timestamp
(current RTP timestamp) even if the current RTP timestamp is
bigger than the previous RTP timestamp.
5. delay_manager_->Update() parameters are main_timestamp
which increments with the RTP sample rate frequency and the
fs_hz_ which is the decoder sample rate frequency. When these
two frequencies are different as is the case with g.722, the
DelayManager::Update() will misfire and calculate incorrect
packet_len_ms and inter-arrival time (IAT) as a result. This
in effect will cause neteq to enter kPreemptiveExpand operation
and will keep expanding the jitter buffer even if the RTP packets
arrive with no jitter at all.
The fix corrects all these problems by making sure the
main_timestamp and the timestamp_ are always set with the scaled
timestamp and increment with the decoder sample rate frequency.
Review-Url: https://codereview.webrtc.org/2743063005
Cr-Commit-Position: refs/heads/master@{#17232}
It's the faster, less strict cousin of checked_cast.
BUG=none
Review-Url: https://codereview.webrtc.org/2714063002
Cr-Commit-Position: refs/heads/master@{#16958}
PacketList is now list<Packet> instead of list<Packet*>.
Splicing the lists in NetEqImpl::InsertPacketInternal instead of
moving packets. Avoid moving the packet when doing Rfc3389Cng.
Removed PacketBuffer::DeleteFirstPacket and DeleteAllPackets.
BUG=chromium:657300
Review-Url: https://codereview.webrtc.org/2425223002
Cr-Commit-Position: refs/heads/master@{#14747}
Only three items in the (rather large) header were actually used after
InsertPacket: payloadType, timestamp and sequenceNumber. They are now
put directly into Packet. This saves 129 bytes per Packet that no
longer need to be allocated and deallocated.
This also works towards decoupling NetEq from RTP. As part of that,
I've moved the NACK code earlier in InsertPacketInternal, together
with other things that directly reference the RTPHeader.
BUG=webrtc:6549
Review-Url: https://codereview.webrtc.org/2411183003
Cr-Commit-Position: refs/heads/master@{#14658}
NetEq already uses SdpAudioFormat internally; this CL adds an
AudioCodingModule::RegisterReceiveCodec overload that accepts
SdpAudioFormat, and propagates it through AcmReceiver into NetEq.
The intention is to get rid of the other ways to specify decoders and
always use SdpAudioFormat. (And eventually to do the same for encoders
too.)
NOTRY=true
BUG=5801
Review-Url: https://codereview.webrtc.org/2365653004
Cr-Commit-Position: refs/heads/master@{#14506}
NetEqDecoder is still used in the external interfaces, but this change
opens up the ability to use SdpAudioFormats directly, once appropriate
interfaces have been added.
BUG=webrtc:5805
Review-Url: https://codereview.webrtc.org/2355503002
Cr-Commit-Position: refs/heads/master@{#14368}
The biggest change to NetEq is the move from a primary flag, to a
Priority with two separate levels: one set by RED splitting and one
set by the codec itself. This allows us to unambigously prioritize
"fallback" packets from these two sources. I've chosen what I believe
is the sensible ordering: packets that the codec prioritizes are
chosen first, regardless of if they are secondary RED packets or
not. So if we were to use Opus w/ FEC in RED, we'd only do Opus FEC
decoding if there was no RED packet that could cover the time slot.
With this change, PayloadSplitter now only deals with RED
packets. Maybe it should be renamed RedPayloadSplitter?
BUG=webrtc:5805
Review-Url: https://codereview.webrtc.org/2342443005
Cr-Commit-Position: refs/heads/master@{#14347}
There's still some code run specifically for Opus w/ FEC. It will be
addressed in a separate CL.
BUG=webrtc:5805
Review-Url: https://codereview.webrtc.org/2326003002
Cr-Commit-Position: refs/heads/master@{#14319}
It requires a new NetEq method, but it can no longer fail. And we no
longer need to use AcmReceiver::decoders_, which we're trying to
eliminate.
(This is a re-land of https://codereview.webrtc.org/2342313002.)
BUG=webrtc:5801
Review-Url: https://codereview.webrtc.org/2348233002
Cr-Commit-Position: refs/heads/master@{#14304}
It allows the decoder to split the input up into usable chunks before
they are put into NetEq's PacketBuffer. Eventually, all packet splitting
will move into ParsePayload.
There's currently a base implementation of ParsePayload. It will
generate a single Frame that calls the underlying AudioDecoder for
getting Duration() and to Decode.
BUG=webrtc:5805
BUG=chromium:428099
Review-Url: https://codereview.webrtc.org/2326953003
Cr-Commit-Position: refs/heads/master@{#14300}
Reason for revert:
Seems to have broken Chromium tests.
Original issue's description:
> AcmReceiver: Look up last decoder in NetEq's table of decoders
>
> AcmReceiver::decoders_ is now one step closer to being unused.
>
> BUG=webrtc:5801
>
> Committed: https://crrev.com/1e4d8b574cde64d93b98d89c7b817fb93185a307
> Cr-Commit-Position: refs/heads/master@{#14274}
TBR=ossu@webrtc.org,henrik.lundin@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5801
Review-Url: https://codereview.webrtc.org/2348123002
Cr-Commit-Position: refs/heads/master@{#14279}
Reason for revert:
Seems to have broken Chromium tests.
Original issue's description:
> AcmReceiver: Ask NetEq to delete all decoders at once instead of one by one
>
> It requires a new NetEq method, but it can no longer fail. And we no
> longer need to use AcmReceiver::decoders_, which we're trying to
> eliminate.
>
> BUG=webrtc:5801
>
> Committed: https://crrev.com/f6232b43a176e1717354b671a0a52b887d70de59
> Cr-Commit-Position: refs/heads/master@{#14275}
TBR=ossu@webrtc.org,henrik.lundin@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5801
Review-Url: https://codereview.webrtc.org/2349973002
Cr-Commit-Position: refs/heads/master@{#14278}
It requires a new NetEq method, but it can no longer fail. And we no
longer need to use AcmReceiver::decoders_, which we're trying to
eliminate.
BUG=webrtc:5801
Review-Url: https://codereview.webrtc.org/2342313002
Cr-Commit-Position: refs/heads/master@{#14275}
AcmReceiver::decoders_ is now one step closer to being unused.
BUG=webrtc:5801
Review-Url: https://codereview.webrtc.org/2339953002
Cr-Commit-Position: refs/heads/master@{#14274}
I could not find a single place it was used, outside of the unittests
for the sync packet support itself.
Review-Url: https://codereview.webrtc.org/2309303002
Cr-Commit-Position: refs/heads/master@{#14130}
The sample uses are from when I debugged bug 617124. The change in neteq_network_stats_unittest.cc is a fix for a minor unrelated bug found by the try bots when I tried to land this CL (a test was passing uninitialized packet data to NetEq).
BUG=chromium:617124
Review-Url: https://codereview.webrtc.org/2293893002
Cr-Commit-Position: refs/heads/master@{#14034}
That is, rather than keeping a separate pointer and size.
This helps automate memory management in NetEq and will be useful in the
work to minimize the AudioDecoder interface as part of the injectable
audio codec work.
I'm planning a follow-up that will change the current management of Packet* to wrapping them in unique_ptr instead.
Review-Url: https://codereview.webrtc.org/2289093003
Cr-Commit-Position: refs/heads/master@{#14002}
With this change, the value 0xFF is no longer used to flag that the RTP
type is unknown. Instead, an empty value for the rtc::Optional is used.
Review-Url: https://codereview.webrtc.org/2290153002
Cr-Commit-Position: refs/heads/master@{#13989}
The current_rtp_payload_type_ should only be updated when the packet is
actually inserted into the packet buffer, since then the payload type
has been validated. This CL removes an unvalidated setting of this value
that happened after SSRC change or upon first packet.
BUG=webrtc:5447
Review-Url: https://codereview.webrtc.org/2270793003
Cr-Commit-Position: refs/heads/master@{#13910}
If an error happens in the GetAudio call, for instance when corrupt
payloads are inserted, GetAudio wil return an error. In this case, the
audio frame is not always correctly populated, which is to be expected.
BUG=webrtc:5447
Review-Url: https://codereview.webrtc.org/2272963002
Cr-Commit-Position: refs/heads/master@{#13902}
The new method returns the current total delay (packet buffer and sync
buffer) in ms, with smoothing applied to even out short-time
fluctuations due to jitter. The packet buffer part of the delay is not
updated during DTX/CNG periods.
This CL also pipes the new metric through ACM and uses it in
VoiceEngine. It replaces the previous method of estimating the buffer
delay (where an inserted packet's RTP timestamp was compared with the
last played timestamp from NetEq). The new method works better under
periods of DTX/CNG.
Review-Url: https://codereview.webrtc.org/2262203002
Cr-Commit-Position: refs/heads/master@{#13855}
I'll be rewriting AcmReceiver soon and am trying to reduce the amount of
old stuff that needs to be supported.
I've manually checked the outputs of the AcmReceiver bitexactness
tests with this change. A large part of the tests are still bitexact,
with one section only differing slightly in timings. Nothing audible
unless playing the old and new versions back simultaneously.
The output of NetEqDecoderTest were also changed due to this CL, although only on android. I built and ran the test locally and compared the audio output manually - the changes were the same as for the other tests; i.e. very slight timing changes for a part of the output.
I updated the network stats checksum for android without analyzing it further. I expect it goes hand-in-hand with the changes to the output; i.e. the changes in it are fine because the audio output is fine. Likely, the stats will show changes in the usage of CNG, since that is what the code changes.
BUG=webrtc:1361
Review-Url: https://codereview.webrtc.org/2117763002
Cr-Commit-Position: refs/heads/master@{#13415}
This allows us to get rid of the function that computes it, which gets
us one step closer to getting rid of the NetEqDecoder type.
BUG=webrtc:5801
Review-Url: https://codereview.webrtc.org/2021063002
Cr-Commit-Position: refs/heads/master@{#12974}
The NetEqNetworkStatistics::expand_rate was not incremented during muted
state, which caused under-reporting of that metric. This change fixes
that.
BUG=chromium:613321, webrtc:5608
Review-Url: https://codereview.webrtc.org/2003203004
Cr-Commit-Position: refs/heads/master@{#12894}
