Implements RTCAudioSourceStats members:
- audioLevel
- totalAudioEnergy
- totalSamplesDuration
In this CL description these are collectively referred to as the audio
levels.
The audio levels are removed from sending "track" stats (in Chrome,
these are now reported as undefined instead of 0).
Background:
For sending tracks, audio levels were always reported as 0 in Chrome
(https://crbug.com/736403), while audio levels were correctly reported
for receiving tracks. This problem affected the standard getStats() but
not the legacy getStats(), blocking some people from migrating. This
was likely not a problem in native third_party/webrtc code because the
delivery of audio frames from device to send-stream uses a different
code path outside of chromium.
A recent PR (https://github.com/w3c/webrtc-stats/pull/451) moved the
send-side audio levels to the RTCAudioSourceStats, while keeping the
receive-side audio levels on the "track" stats. This allows an
implementation to report the audio levels even if samples are not sent
onto the network (such as if an ICE connection has not been established
yet), reflecting some of the current implementation.
Changes:
1. Audio levels are added to RTCAudioSourceStats. Send-side audio
"track" stats are left undefined. Receive-side audio "track" stats
are not changed in this CL and continue to work.
2. Audio level computation is moved from the AudioState and
AudioTransportImpl to the AudioSendStream. This is because a) the
AudioTransportImpl::RecordedDataIsAvailable() code path is not
exercised in chromium, and b) audio levels should, per-spec, not be
calculated on a per-call basis, for which the AudioState is defined.
3. The audio level computation is now performed in
AudioSendStream::SendAudioData(), a code path used by both native
and chromium code.
4. Comments are added to document behavior of existing code, such as
AudioLevel and AudioSendStream::SendAudioData().
Note:
In this CL, just like before this CL, audio level is only calculated
after an AudioSendStream has been created. This means that before an
O/A negotiation, audio levels are unavailable.
According to spec, if we have an audio source, we should have audio
levels. An immediate solution to this would have been to calculate the
audio level at pc/rtp_sender.cc. The problem is that the
LocalAudioSinkAdapter::OnData() code path, while exercised in chromium,
is not exercised in native code. The issue of calculating audio levels
on a per-source bases rather than on a per-send stream basis is left to
https://crbug.com/webrtc/10771, an existing "media-source" bug.
This CL can be verified manually in Chrome at:
https://codepen.io/anon/pen/vqRGyq
Bug: chromium:736403, webrtc:10771
Change-Id: I8036cd9984f3b187c3177470a8c0d6670a201a5a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143789
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28480}
This reverts commit 8fa7151e4bbad40fec1f964fe0c003b8787bb78a.
Reason for revert: Speculative revert to fix roll of webrtc into chrome. Right now tests related to RTCRtpReceiver failing and looks like it is main candidate, who can affect that behavior.
Original change's description:
> Replace the implementation of `GetContributingSources()` on the audio side.
>
> This change replaces the `ContributingSources`-implementation of `GetContributingSources()` and `GetSynchronizationSources()` on the audio side with the spec-compliant `SourceTracker`-implementation.
>
> The most noticeable impact is that the per-frame dictionaries are now updated when frames are delivered to the RTCRtpReceiver's MediaStreamTrack rather than when RTP packets are received on the network.
>
> This change is almost identical to the previous video side change at: https://webrtc-review.googlesource.com/c/src/+/143177
>
> Bug: webrtc:10545
> Change-Id: Ife7f08ee8ca1346099b7466837a3756947085fc5
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144422
> Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
> Commit-Queue: Chen Xing <chxg@google.com>
> Cr-Commit-Position: refs/heads/master@{#28459}
TBR=ossu@webrtc.org,chxg@google.com
Change-Id: I5c631d4dcfb39601055ffce9b104f45eea871fd3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10545
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144562
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28478}
This change replaces the `ContributingSources`-implementation of `GetContributingSources()` and `GetSynchronizationSources()` on the audio side with the spec-compliant `SourceTracker`-implementation.
The most noticeable impact is that the per-frame dictionaries are now updated when frames are delivered to the RTCRtpReceiver's MediaStreamTrack rather than when RTP packets are received on the network.
This change is almost identical to the previous video side change at: https://webrtc-review.googlesource.com/c/src/+/143177
Bug: webrtc:10545
Change-Id: Ife7f08ee8ca1346099b7466837a3756947085fc5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144422
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Commit-Queue: Chen Xing <chxg@google.com>
Cr-Commit-Position: refs/heads/master@{#28459}
And delete corresponding plumbing via the internal stats attribute
MediaReceiverInfo::fraction_lost. The latter attribute is not deleted
yet, since downstream projects have to be updated first.
Bug: webrtc:10744
Change-Id: Id5401aeee7e5637a406ddf2fa33fbfe336abec9f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143178
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28385}
RTP timestamp was recently added to contributing sources in the WebRTC
specification. This CL implements that change in WebRTC.
Bug: webrtc:10650
Change-Id: Ic0ccfbea7049a5b66063fa6cf60d01d5bd713132
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/137515
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28020}
Currently we pass media_transport from PeerConnection to media layers. The goal of this change is to replace media_transport with struct MediaTransportCondif, which will enable adding different transports (i.e. we plan to add DatagramTransport) as well as other media-transport related settings without changing 100s of files.
TODO: In the future we should consider also adding rtp_transport in the same config, but it will require a bit more work, so I did not include it in the same change.
Bug: webrtc:9719
Change-Id: Ie31e1faa3ed9e6beefe30a3da208130509ce00cd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/137181
Commit-Queue: Anton Sukhanov <sukhanov@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Bjorn Mellem <mellem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28016}
This reverts commit 7dd83e2bf73a7f1746c5ee976939bf52e19fa8be.
Reason for revert: This wasn't the cause of the break.
Original change's description:
> Revert "Refactor FrameDecryptorInterface::Decrypt to use new API."
>
> This reverts commit 642aa81f7d5cc55d5b99e2abc51327eed9d40195.
>
> Reason for revert: Speculative revert. The chromium roll is failing:
> https://ci.chromium.org/p/chromium/builders/try/linux-rel/64388
> But I can't figure out exactly what is failing, this looks suspecious.
>
> Original change's description:
> > Refactor FrameDecryptorInterface::Decrypt to use new API.
> >
> > This change refactors the FrameDecryptorInterface to use the new API. The new
> > API surface simply moves bytes_written to the return type and implements a
> > simple Status type.
> >
> > Bug: webrtc:10512
> > Change-Id: I622c5d344d58e618853c94c2f691cf7c8fb73a36
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131460
> > Reviewed-by: Steve Anton <steveanton@webrtc.org>
> > Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
> > Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> > Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> > Commit-Queue: Benjamin Wright <benwright@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#27497}
>
> TBR=brandtr@webrtc.org,steveanton@webrtc.org,solenberg@webrtc.org,ossu@webrtc.org,stefan@webrtc.org,benwright@webrtc.org
>
> Change-Id: Ia9ec70263762c34671af13f0d519e636eb8473cd
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: webrtc:10512
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/132013
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Commit-Queue: Henrik Boström <hbos@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27510}
TBR=brandtr@webrtc.org,steveanton@webrtc.org,solenberg@webrtc.org,hbos@webrtc.org,ossu@webrtc.org,stefan@webrtc.org,benwright@webrtc.org
Change-Id: I8e4b7965cf1d1a1554c3b46e6245f5ad0d2dcbb4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10512
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131982
Reviewed-by: Benjamin Wright <benwright@webrtc.org>
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27529}
This reverts commit 642aa81f7d5cc55d5b99e2abc51327eed9d40195.
Reason for revert: Speculative revert. The chromium roll is failing:
https://ci.chromium.org/p/chromium/builders/try/linux-rel/64388
But I can't figure out exactly what is failing, this looks suspecious.
Original change's description:
> Refactor FrameDecryptorInterface::Decrypt to use new API.
>
> This change refactors the FrameDecryptorInterface to use the new API. The new
> API surface simply moves bytes_written to the return type and implements a
> simple Status type.
>
> Bug: webrtc:10512
> Change-Id: I622c5d344d58e618853c94c2f691cf7c8fb73a36
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131460
> Reviewed-by: Steve Anton <steveanton@webrtc.org>
> Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
> Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
> Reviewed-by: Stefan Holmer <stefan@webrtc.org>
> Commit-Queue: Benjamin Wright <benwright@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27497}
TBR=brandtr@webrtc.org,steveanton@webrtc.org,solenberg@webrtc.org,ossu@webrtc.org,stefan@webrtc.org,benwright@webrtc.org
Change-Id: Ia9ec70263762c34671af13f0d519e636eb8473cd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10512
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/132013
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27510}
This change refactors the FrameDecryptorInterface to use the new API. The new
API surface simply moves bytes_written to the return type and implements a
simple Status type.
Bug: webrtc:10512
Change-Id: I622c5d344d58e618853c94c2f691cf7c8fb73a36
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131460
Reviewed-by: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27497}
to clearly signal passed ownership.
Drop support for accepting nullptr clock to avoid copying the Configuration structure.
Update all calls in webrtc to the new factory function
Bug: None
Change-Id: Ic5a78da8e59ba3988a757a9d9634fa31499ce0db
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/125901
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26994}
The following APIs on AudioCodingModule are deprecated with this CL:
static int NumberOfCodecs();
static int Codec(int, CodecInst*);
static int Codec(const char*, CodecInst*, int, size_t);
static int Codec(const char*, int, size_t);
absl::optional<CodecInst> SendCodec() const;
bool RegisterReceiveCodec(int, const SdpAudioFormat&);
int RegisterExternalReceiveCodec(int, AudioDecoder*, int, int, const std::string&);
int UnregisterReceiveCodec(uint8_t);
int32_t ReceiveCodec(CodecInst*);
absl::optional<SdpAudioFormat> ReceiveFormat();
As well as this method on RtpRtcp module:
int32_t RegisterSendPayload(const CodecInst&);
Bug: webrtc:7626
Change-Id: I1230732136f1fe9048cf74afdeab767ca57ac9ce
Reviewed-on: https://webrtc-review.googlesource.com/c/113816
Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26025}
setting max reordering recently has been fix to actually set it.
(https://webrtc-review.googlesource.com/c/src/+/111752)
Another recent change fix stats to skip counting large sequence number jumps as packet loss
(https://webrtc-review.googlesource.com/c/src/+/111962)
max reordering thresholds affects how packet loss is calculated.
Packet loss is then reported to remote sending participant in rtcp receiver reports.
Sender uses packet loss mostly for stats, but also e.g. for opus fec adjustment.
Setting threshold to zero de-facto imply all packets should be considered in order.
That bug was mitigated by two other bugs mentioned above
This change increase threshold to default 50 packets aligning it with Video receiver
and unblocks (re)landing 2nd fix
Bug: b/120482366
Change-Id: Iadda0c2148ed84dd83c01183cfe9285568db4e29
Reviewed-on: https://webrtc-review.googlesource.com/c/113064
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25905}
Note that this value will override the minimum delay that is used for audio/video sync.
Bug: webrtc:10053
Change-Id: Ia129f6c9ee9da5d00a3d955afaaa6e8f0c2bee33
Reviewed-on: https://webrtc-review.googlesource.com/c/112121
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ivo Creusen <ivoc@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25805}
Replaced by interface ChannelSendInterface, implemented by ChannelSend
and mock class.
Thread checkers are moved to ChannelSend, which is also moved into
the anonymous namespace and exposed only via a function CreateChannelSend.
Bug: webrtc:9801
Change-Id: I73b2e2bfb67c1a5077709f2379533bf315babad9
Reviewed-on: https://webrtc-review.googlesource.com/c/111240
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25684}
So far ANA was not available for media transport interface. With recent changes to media transport, we can now account for packet overhead, network route (ip/tcp/udp/turn overheads) and we can also use bandwidth estimate from the media transport.
Bug: webrtc:9719
Change-Id: I98c9a09dd418b763c339ee2ee05592e164cf9199
Reviewed-on: https://webrtc-review.googlesource.com/c/110367
Commit-Queue: Peter Slatala <psla@webrtc.org>
Reviewed-by: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Anton Sukhanov <sukhanov@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25677}
Replaced by an interface ChannelReceiveInterface, implemented
by ChannelReceive and the corresponding mock class.
Moved thread checkers to ChannelReceive. That class is moved to the
anonymous namespace in the .cc file, and exposed only via a function
CreateChannelReceive.
Bug: webrtc:9801
Change-Id: Iecacbb1858885bf86da9484f2422e53323dbe87a
Reviewed-on: https://webrtc-review.googlesource.com/c/110610
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25665}
This is a preparation for deleting ChannelReceiveProxy, Changes
signature of some methods, and demotes methods OnData and
OnReceivedPayloadData to private.
Bug: webrtc:9801
Change-Id: Ib00a80c6482ed5238f3cc8233860c70f11484df9
Reviewed-on: https://webrtc-review.googlesource.com/c/110606
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25599}
This patch adds (optional) csrc to ContributingSources.
This will be used if using virtual audio ssrc, since
the audio level is otherwise unaccessible in that configuration.
BUG=webrtc:3333
Change-Id: Ied263b8f0850553cd637fd6bead373ed4252fd1e
Reviewed-on: https://webrtc-review.googlesource.com/c/109281
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25516}
This change corrects a potential race condition when updating a FrameEncryptor
for the audio send channel. If a FrameEncryptor is set on an active audio
stream it is possible for the current FrameEncryptor attached to the audio channel to be deallocated due to
the FrameEncryptors reference count reaching zero before the new FrameEncryptor is set on the
channel.
To address this issue the ChannelSend is now holds a scoped_reftptr<FrameEncryptor>
to only allow deallocation when it is actually set on the encoder queue.
ChannelSend is unique in this respect as the Audio Receiver a long with the
Video Sender and Video Receiver streams all recreate themselves when they have
a configuration change. ChannelSend instead reconfigures itself using the
existing channel object.
Added Seth as TBR as this only introduces mocks.
TBR=shampson@webrtc.org
Bug: webrtc:9907
Change-Id: Ibf391dc9cecdbed1874e0252ff5c2cb92a5c64f4
Reviewed-on: https://webrtc-review.googlesource.com/c/107664
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Qingsi Wang <qingsi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25374}
This CL is the result of running include-what-you-use tool on part
of the code base (audio target and dependencies) plus manual fixes.
bug: webrtc:8311
Change-Id: I277d281ce943c3ecc1bd45fd8d83055931743604
Reviewed-on: https://webrtc-review.googlesource.com/c/106280
Commit-Queue: Yves Gerey <yvesg@google.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25311}
This change adds a new subcategory to the public native webrtc::CryptoOptions
structure: webrtc::CryptoOptions::Frame.
This new structure has a single off by default property:
crypto_options.frame.require_frame_encryption.
This new flag if set prevents RtpSenders from sending outgoing payloads unless
a frame_encryptor_ is attached and prevents RtpReceivers from receiving
incoming payloads unless a frame_decryptor_ is attached.
This option is important to enforce no unencrypted data can ever leave the
device or be received.
I have also attached bindings for Java and Objective-C.
I have implemented this functionality for E2EE audio but not E2EE video
since the changes are still in review.
Bug: webrtc:9681
Change-Id: Ie184711190e0cdf5ac781f69e9489ceec904736f
Reviewed-on: https://webrtc-review.googlesource.com/c/105540
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Oskar Sundbom <ossu@webrtc.org>
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25238}
Followup to cl https://webrtc-review.googlesource.com/c/src/+/103640.
Set the rtcp_send_transport at construction time, delete
RegisterTransport, and the proxying of transport methods.
In addition, delete the unused RtcpRtpStats argument from the
constructor.
Bug: webrtc:9801
Change-Id: I80f25bc08dc2130386053568ddce4ef91654caeb
Reviewed-on: https://webrtc-review.googlesource.com/c/103803
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25010}
This change integrates the FrameDecryptorInterface and the FrameEncryptorInterface into
the audio media path. If a FrameEncryptorInterface is set on an outgoing audio RTPSender
then each outgoing audio payload will first pass through the provided FrameEncryptor which
will have a chance to modify the payload contents for the purposes of encryption.
If a FrameDecryptorInterface is set on an incoming audio RtpReceiver then each incoming
audio payload will first pass through the provided FrameDecryptor which have a chance to
modify the payload contents for the purpose of decryption.
While AEAD is supported by the FrameDecryptor/FrameEncryptor interfaces this CL does not
use it and so it is left as null.
Bug: webrtc:9681
Change-Id: Ic383a9dce280528739f9d271357c2220e0a0dccf
Reviewed-on: https://webrtc-review.googlesource.com/c/101702
Commit-Queue: Benjamin Wright <benwright@webrtc.org>
Reviewed-by: Fredrik Solenberg <solenberg@webrtc.org>
Reviewed-by: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Emad Omara <emadomara@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25001}
