304 Commits

Author SHA1 Message Date
Harald Alvestrand
d3ac3b668a Remove AsSendChannel/AsReceiveChannel methods
This CL removes a couple more opportunities for client code
to interact directly with the MediaChannel implementation classes.

No-try because of infra failure.

No-Try: true
Bug: webrtc:13931
Change-Id: I658b8b04eff11de7831a1933d16d40fc59c3f0fc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/288380
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38935}
2022-12-21 07:47:32 +00:00
Harald Alvestrand
c0d44d9d63 Split audio and video channels into Send and Receive APIs.
The implementation here has a number of changes that force the callers
that called the "channel" functions into specific interfaces rather than
just letting C++ take care of it; this should go away once there stops
being a common implementation class for those interfaces.

Bug: webrtc:13931
Change-Id: Ic4e279528a341bc0a0e88d2e1e76c90bc43a1035
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/287640
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38888}
2022-12-14 11:00:17 +00:00
Florent Castelli
acabb3641b pc: Add asynchronous RtpSender::SetParameters() call
As the synchronous version only posts a task to recreate the encoder
later, it is not possible to catch errors and state changes that
could appear then.
The asynchronous version of SetParameters() aims to solve this by
providing a callback to wait for the completion of the encoder
reconfiguration, allowing any error to be propagate and subsequent
getParameters() call to have up to date information.

Bug: webrtc:11607
Change-Id: I5548e75aa14a97f8d9c0c94df1e72e9cd40887b2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/278420
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38627}
2022-11-15 15:31:40 +00:00
Henrik Boström
262d9cca08 [Stats] Fix outbound-rtp.active bug in SVC.
The code for determining outbound-rtp.active assumed, as the spec says,
that we have one RtpEncodingParameters per RTP stream. Unfortunately
SVC is currently implemented as one RtpEncodingParameters per SVC
layer. This causes a discrepency where we do correctly only have one
outbound-rtp stats object, but the lookup to check whether or not we are
"active" needs to look at more than a single encoding.

The bug is that if SVC layers are {inactive, active, active} then
stats reports outbound-rtp.active: false. With this fix, active: true is
reported if ANY of the SVC layers are active.

For singlecast or simulcast this CL has no change in behavior. In these
cases we have the same number of outbound-rtp and encodings and a simple
ssrc lookup does work.

The fix is exercised by unit tests and has also manually been confirmed:
- Singlecast tested by https://jsfiddle.net/henbos/nvd6p4j1/.
- Simulcast tested by https://crbug.com/webrtc/14628#c11.
- SVC tested by Google Meet and chrome://webrtc-internals/.

Bug: webrtc:14628
Change-Id: Ib89945caf29c8f4b85dd8a1120dcf8279296e4a3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/282222
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38569}
2022-11-07 17:20:30 +00:00
Henrik Boström
d81992197c [Stats] Update totalPacketSendDelay to only cover time in pacer queue.
This metric was always supposed to be the spec's answer to
googBucketDelay, and is defined as "The total number of seconds that
packets have spent buffered locally before being transmitted onto the
network." But our implementation measured the time between capture and
send, including encode time. This is incorrect and yields a much larger
value than expected.

This CL updated the metric to do what the spec says. Implementation-wise
we measure the time between pushing and popping each packet from the
queue (in modules/pacing/prioritized_packet_queue.cc).

The spec says to increment the delay counter at the same time as we
increment the packet counter in order for the app to be able to do
"delta totalPacketSendDelay / delta packetSent". For this reason,
`total_packet_delay` is added to RtpPacketCounter. (Previously, the
two counters were incremented on different threads and observers.)

Running Google Meet on a good network, I could observe a 2-3 ms average
send delay per packet with this implementation compared to 20-30 ms
with the old implementation. See b/137014977#comment170 for comparison
with googBucketDelay which is a little bit different by design -
totalPacketSendDelay is clearly better than googBucketDelay.

Since none of this depend on the media kind, we can wire up this metric
for audio as well in a follow-up:
https://webrtc-review.googlesource.com/c/src/+/280523

Bug: webrtc:14593
Change-Id: If8fcd82fee74030d0923ee5df2c2aea2264600d4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/280443
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38480}
2022-10-26 21:29:20 +00:00
Evan Shrubsole
09da10e24f Add powerEfficientDecoder and powerEfficientEncoder stats
The spec for these are at https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-powerefficientdecoder and https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-powerefficientdecoder

These stats are based on the is_hardware_accelerated boolean in both the
DecoderInfo and EncoderInfo structs.

Bug: webrtc:14483
Change-Id: I4610da3c6ae977f5853a3b3424d91d864fe72592
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/274409
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38441}
2022-10-19 13:15:31 +00:00
Sergio Garcia Murillo
d1196636dc Recreate default receive stream when unsignaled ssrc for rtx is received
Bug: webrtc:10297
Change-Id: Ie166b778b30e1c178c765f348e38426a6d57e29f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/278682
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38378}
2022-10-13 08:34:05 +00:00
Jonas Oreland
80c87d7151 RtpEncodingParameters::request_resolution patch 2
This cl/ implements configuring of encode resolution
in the video_stream_encoder (webrtc_video_engine) in
a way that is independent of frame resolution (i.e
not using scale_resolution_down_by).

The cl/ reuses the VideoAdapter as is, and hence
the output resolution will be the same as it is today.

Anticipated further patches
3) Hook up resource adaptation
4) Let VideoSource do adaption if possible

Bug: webrtc:14451
Change-Id: I881b031c5b23be26cacfe138730154f1cb1b66a8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/276742
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38245}
2022-09-29 14:10:44 +00:00
Johannes Kron
bb591c49e8 Change the default setting for PreStreamDecoders/LazyDecoderCreation
The experiment has been approved for a full launch. Changing the
default value so that no decoder is created before the stream starts.
All decoders are created lazily on demand when we receive payload
data of the corresponding type.

Bug: chromium:1319864
Change-Id: Ifb412bbe49a7577a45c340496d5b8572ebc1ba44
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/277120
Auto-Submit: Johannes Kron <kron@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38232}
2022-09-28 11:40:22 +00:00
Jonas Oreland
1262eb5ebc Move EncoderStreamFactory into own file
This cl/ is a NOP refactoring,
moving the EncoderStreamFactory from within webrtc_video_engine.cc
into own file in video/. simulcast.cc is collateral.

Bug: webrtc:14451
Change-Id: Ia69b9241d8cd8a12be6628d887701f2e244c07cc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/276861
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38224}
2022-09-27 17:29:11 +00:00
Erik Språng
7aaeb5a270 Create pre-allocated decoders on the decoder thread.
This way we're sure instantiation, configuration and decode calls all
happen on the decoder queue - making thread checking easier in the
actual decoder classes.

Bug: None
Change-Id: Ia98f47009f26b34eb8dad2ee0b4ddcde082d1994
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/272022
Reviewed-by: Evan Shrubsole <eshr@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37825}
2022-08-18 13:14:26 +00:00
Tommi
dfc4d81fd3 [WebRtcVideoReceiveStream] Construct flexfec stream on demand.
...for payload type changes and avoid recreating the main video stream.

Bug: webrtc:11993
Change-Id: I03e44ff25d88caeb082a2e44b2e802d3b9392feb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269244
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37666}
2022-08-02 16:33:04 +00:00
Tommi
185f10c082 [WebRtcVideoReceiveStream] Add ability to config flexfec post init.
Bug: webrtc:11993
Change-Id: I35d7e645e18b7cb4a86645ea52c8958063f13d3c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269243
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37661}
2022-08-02 11:37:26 +00:00
Erik Språng
a3f2e72008 Fix potential UAF in unit test.
In WebRtcVideoChannelBaseTest.EncoderSelectorSwitchCodec a mock encoder
selecter or stack allocated and then registered with the channel.
Since this test uses real-time clocks/threads, there is a chance that
the selector callback will be called after the mock goes out of scope,
but before the test had time to be torn down.

This CL fixes that by simply de-registering the callback before the
end of the test.

Bug: b/239855550
Change-Id: Ibb38a914933494fd04c963b9a13f2cc4aee160d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269402
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37618}
2022-07-26 09:19:34 +00:00
Erik Språng
5793ed8ff9 Remove unused field trial WebRTC-SendBufferSizeBytes
Bug: webrtc:9637
Change-Id: Ie4bf5caa3977f1824f6dc1014b1feacf6ebfe670
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269386
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37617}
2022-07-26 07:26:14 +00:00
Erik Språng
d460c31309 Make sure VideoStream target bitrate is not higher than max.
Bug: webrtc:14017
Change-Id: Ic59811f4ec84e429839440cea8a70c05d7795cc4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269388
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37614}
2022-07-25 20:34:48 +00:00
Erik Språng
b6ff84b516 Reland "When VP9 SVC is used, use SvcConfig to set max bitrate for the stream."
This is a reland of commit 3afb8e24311dc1297150d4011894b6cb00841735

Patchset 1 is the original CL. Patchset 2 contains a fix:
Depending on call site, the number of spatial layers for VP9 might be
signalled in three different ways. One of them was afaict only used in
out perf tests, and resulted in the max bitrate being incorrectly
capped.
The fix now checks that field too.

Original change's description:
> When VP9 SVC is used, use SvcConfig to set max bitrate for the stream.
>
> Currently, a default max bitrate is determined within WebRtcVideoEngine,
> which maxes out at 2.5Mbps - and that limits the max bitrate deteremined
> by SvcConfig for resolutions above 720p.
>
> This does not affect simulcast, as WebRtcVideoEngine already knows to
> trust the rate allocation in simulcast.cc instead.
>
> Bug: webrtc:14017
> Change-Id: I0c310a6fd496e9e5a10eae45838900068aa1ae2d
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267160
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#37370}

Bug: webrtc:14017, webrtc:14234
Change-Id: Idcaf4321a20c917e4049522c577336ddcfc7ffbb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267860
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37446}
2022-07-05 14:19:33 +00:00
Erik Språng
4f1af1156f Revert "When VP9 SVC is used, use SvcConfig to set max bitrate for the stream."
This reverts commit 3afb8e24311dc1297150d4011894b6cb00841735.

Reason for revert: Causes some unexpected perf regressions.

Original change's description:
> When VP9 SVC is used, use SvcConfig to set max bitrate for the stream.
>
> Currently, a default max bitrate is determined within WebRtcVideoEngine,
> which maxes out at 2.5Mbps - and that limits the max bitrate deteremined
> by SvcConfig for resolutions above 720p.
>
> This does not affect simulcast, as WebRtcVideoEngine already knows to
> trust the rate allocation in simulcast.cc instead.
>
> Bug: webrtc:14017
> Change-Id: I0c310a6fd496e9e5a10eae45838900068aa1ae2d
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267160
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#37370}

Bug: webrtc:14017
Change-Id: I1e45ee3f78deb50a9057d648146b1a6360782aa3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267800
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37438}
2022-07-05 10:25:23 +00:00
Niels Möller
c8152fe4a8 Update/delete old TODOs
Bug: webrtc:10198
Change-Id: I226768c2a6bd97ffcd0638e5bc6a1c286b71815f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267704
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37435}
2022-07-05 09:44:53 +00:00
Erik Språng
3afb8e2431 When VP9 SVC is used, use SvcConfig to set max bitrate for the stream.
Currently, a default max bitrate is determined within WebRtcVideoEngine,
which maxes out at 2.5Mbps - and that limits the max bitrate deteremined
by SvcConfig for resolutions above 720p.

This does not affect simulcast, as WebRtcVideoEngine already knows to
trust the rate allocation in simulcast.cc instead.

Bug: webrtc:14017
Change-Id: I0c310a6fd496e9e5a10eae45838900068aa1ae2d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267160
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37370}
2022-06-29 14:23:08 +00:00
Philipp Hancke
d970b0901b measure decode time in TimeDelta instead of ms
increasing precision since summing up rounded values leads to
a rounding error, in particular for small frames which take very
little time to decode.

BUG=webrtc:12526,webrtc:13756

Change-Id: I647c702808856a002c746ed9f115aa9bcaddc1f3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262810
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philipp Hancke <philipp.hancke@googlemail.com>
Cr-Commit-Position: refs/heads/main@{#37249}
2022-06-17 09:57:27 +00:00
Jonas Oreland
6545516a14 RtpSenderInterface::SetEncoderSelector
This cl/ adds a way of setting an EncoderSelector on a specific
RtpSenderInterface. This makes it possible to easily use different
EncoderSelector on different streams within the same or different PeerConnections.

The cl/ is almost identical to the impl. of RtpSenderInterface::SetFrameEncryptor.

Iff a EncoderSelector is set on the RtpSender, it will take precedence
over the VideoEncoderFactory::GetEncoderSelector.

Bug: webrtc:14122
Change-Id: Ief4f7c06df7f1ef4ce3245de304a48e9de0ad587
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/264542
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37150}
2022-06-08 11:06:52 +00:00
Tommi
a136ed4085 Add SetTransportCc to ReceiveStreamInterface.
Setting the transport cc flag was only possible post creation for
audio receive streams, while video receive streams need to be recreated.

This CL moves the setter for transport_cc() to where the getter is and
adds boiler plate implementations for the video streams. For audio
streams this splits "SetUseTransportCcAndNackHistory" into two methods,
SetTransportCc and SetNackHistory.

Bug: none
Change-Id: Idbec8217aef10ee77907cebaecdc27b4b0fb18e4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/264443
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37038}
2022-05-30 14:07:04 +00:00
Harald Alvestrand
485457f050 Delete ChannelManager class
Bug: webrtc:13931
Change-Id: I331aed0e304f89a0c53d8db20ab2c9733ebbb34c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/263120
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36970}
2022-05-23 10:06:26 +00:00
Harald Alvestrand
c3fa7c38b2 Remove remaining trampoline functions from channel_manager
This is part of the project to delete the class entirely.
The CL also adds an "use_rtx" parameter to the function for listing
video codecs, rather than filtering those away afterwards.

Bug: webrtc:13931
Change-Id: I96b9b18c694a1c0986ccf22face76ef4c704d372
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262666
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36963}
2022-05-23 08:09:06 +00:00
Tommi
f6f4543304 Rename VideoReceiveStream to VideoReceiveStreamInterface
Bug: webrtc:7484
Change-Id: I653cfe46486e0396897dd333069a894d67e3c07b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262769
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36958}
2022-05-22 10:54:38 +00:00
Niels Möller
83830f316e Delete TestListener and top-level thread wrapping.
Instead use rtc::AutoThread in tests that need that.

Bug: webrtc:9714
Change-Id: I1f33b1b2d321770d062504dd9ef86d66a345dd42
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254681
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36950}
2022-05-20 15:21:21 +00:00
Philipp Hancke
0359ba2225 stats: add frame assembly time stats
implements a total frame assembly time statistic that measures the
cumulative time between the arrival of the first packet of a frame
(the lowest reception time) and the time all packets of the frame have
been received (i.e. the highest reception time)

This is similar to totalProcessingDelay
  https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-totalprocessingdelay
in particular with respect to only being incremented for frames that are being decoded but does not include the amount of time spent decoding the frame.

This statistic is useful for evaluating mechanisms like NACK and FEC
and gives some insight into the behavior of the pacer sending the
packets.
Note that for frames with just a single packet the assembly time will be zero. In order to calculate an average assembly time an additional frames_assembled_from_multiple_packets counter for frames with more than a single packet is added.

Currently this is a nonstandard stat so will only show up in webrtc-internals and not in getStats. Formally it can be defined as

totalAssemblyTime of type double
	Only exists for video. 	The sum of the time, in seconds, each video frame takes from the time the first RTP packet is received (reception timestamp) and to the time the last RTP packet of a frame is received.
    Given the complexities involved, the time of arrival or the reception timestamp is measured as close to the network layer as possible.

    This metric is not incremented for frames that are not decoded, i.e., framesDropped, partialFramesLost or frames that fail decoding for other reasons (if any). Only incremented for frames consisting of more than one RTP packet. The average frame assembly time can be calculated by dividing the totalAssemblyTime with framesAssembledFromMultiplePacket.

framesAssembledFromMultiplePacket of type unsigned long
	Only exists for video. It represents the total number of frames correctly decoded for this RTP stream that consist of more than one RTP packet.
	For such frames the totalAssemblyTime is incremented.

BUG=webrtc:13986

Change-Id: Ie0ae431d72a57a0001c3240daba8eda35955f04e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260920
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36922}
2022-05-18 09:16:10 +00:00
Asa Persson
2698353d43 Disable quality scaling if multiple spatial layers are requested for VP9
VP9 automaticResizeOn is disabled if more than one spatial layer is configured via scalability mode.

Bug: webrtc:13960
Change-Id: I7c6351bca6d2f32bcc7391894e8dcc9e74ca2050
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261315
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36897}
2022-05-16 15:11:24 +00:00
Niels Möller
807328fec7 Move frame drop config to VideoCodec and VideoEncoderConfig.
Intend to delete corresponding codec-specific settings in a followup.

Bug: webrtc:6883
Change-Id: I78ab07729a5aee1055f80d39d8f7289beb6721e1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262244
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36882}
2022-05-13 13:40:14 +00:00
Danil Chapovalov
8f44ae4fa5 Remove dead code in cricket::FakeVideoRenderer
Remove unused accessors
move helper implementation into .cc file

Bug: None
Change-Id: Iccd877180901b278af4800f681669089b8a046ce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262254
Commit-Queue: Niels Moller <nisse@webrtc.org>
Auto-Submit: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36881}
2022-05-13 13:36:35 +00:00
Tommi
cb7c7366d0 Separate reading remote_ssrc from using the rtp_config() getter.
`remote_ssrc` can be considered const while some other state represented
by rtp_config() can not and also is tied to a specific thread.
Separating access to these variables, makes moving things around easier.

Bug: webrtc:11993
Change-Id: I70aa000daab6174a401e01dca163213174e8f284
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261316
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36818}
2022-05-09 14:55:00 +00:00
Erik Språng
536f587b98 Revert "Calculate video stream max bitrate using expression."
This reverts commit 45361f78ed18c350b3edcaef19ae4c7cf167e95b.

Reason for revert: Perf alerts galore.

Original change's description:
> Calculate video stream max bitrate using expression.
>
> This replaces the ealier table-based caps.
> Apart from the VGA cap (now 1600kbps instead of 1700kbps), or if using
> "in between" resolutions, the caps are unchanged - but now cover high
> resolutions better.
>
> Bug: webrtc:14017
> Change-Id: I8649b528495d6c917e38ea8cb1a272df6c464c03
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260940
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Per Kjellander <perkj@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#36776}

Bug: webrtc:14017
Change-Id: I18ebc81c6054713c58d49bd227e37090686958c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261309
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#36794}
2022-05-06 09:37:32 +00:00
Erik Språng
45361f78ed Calculate video stream max bitrate using expression.
This replaces the ealier table-based caps.
Apart from the VGA cap (now 1600kbps instead of 1700kbps), or if using
"in between" resolutions, the caps are unchanged - but now cover high
resolutions better.

Bug: webrtc:14017
Change-Id: I8649b528495d6c917e38ea8cb1a272df6c464c03
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260940
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36776}
2022-05-05 10:10:20 +00:00
Niels Möller
ba2de58a22 Update audio/, media/, and video/ to not use implicit conversion
from scoped_refptr<T> to T*.

Bug: webrtc:13464
Change-Id: Ia14885f359fea2bdf08a41b3ded82532a9585d34
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/259503
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36599}
2022-04-21 09:00:14 +00:00
Jonas Oreland
8ca06137dc WebRTC-DeprecateGlobalFieldTrialString/Enabled/ - part 4/inf
convert almost all of video/ (and the collateral)

Bug: webrtc:10335
Change-Id: Ic94e05937f54d11ee8a635b6b66fd146962d9f11
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254601
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36192}
2022-03-14 14:36:35 +00:00
Jonas Oreland
4476b82b35 WebRTC-DeprecateGlobalFieldTrialString/Enabled/ - part 3/inf
convert much of media/ (and the collateral)

Bug: webrtc:10335
Change-Id: I04489dfe9622efe7f89e04aba3be6b3f60e77c91
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254520
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36177}
2022-03-11 07:46:34 +00:00
Sergey Silkin
e1cd3ad4f5 Switch encoder on init failure
Currently if encoder initialization fails WebRTC doesn't send any video.
This CL adds functionality that changes encoder type in such case and
restores the video. If encoder selector is available we switch to
encoder it recommends. Otherwise, VP8 is used as the default fallback
encoder.

Bug: webrtc:13572
Change-Id: Ifcdf707a575711f5ff81f9451caf30140c9171dc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/246960
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35761}
2022-01-21 12:05:17 +00:00
Sergey Silkin
42bf2c670c Put current send codec to front of codecs list in RTP sender parameters
WebRTC can switch encoder on-fly when encoder fails or by request from
encoder selector. Putting the current send codec to the front of the
codecs list provides a simple way for apps to know what is actually
used without retrieving stats.

Bug: webrtc:13572
Change-Id: Iaaa5f7ad8667f59016dc92bff9e9a57a7425ef44
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/246500
Reviewed-by: Mirta Dvornicic <mirtad@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35723}
2022-01-18 14:36:33 +00:00
Erik Språng
5065e5b922 Don't configure frame dropping externally for screenshare.
Legacy code depended of setting VideoCodecVP8::frameDroppingOn to false
for screensharing since the reference frame management handles frame
dropping in the VP8 wrapper instead.
Now the frame dropping is instead configured based on what the
Vp8FrameBufferController instance in use signals.

This change unblocks relanding
https://webrtc-review.googlesource.com/c/src/+/242366

This CL also turns frame dropping on for H264 screenshare, which
should be desirable as it allows for quicker recovery from rate control
overshoots.

Bug: webrtc:9734
Change-Id: I34a29edcd41bb5fd07f7f9bf68660472a1570533
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242965
Reviewed-by: Markus Handell <handellm@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35592}
2021-12-29 17:02:38 +00:00
Markus Handell
68f06af6f6 WebRtcVideoChannelBaseTest.InvalidRecvBufferSize: fix UAF.
The test could cause a UAF as the test exits while the lambda is
still running. Only seems to happen on Linux for some reason.

Bug: webrtc:12854
Change-Id: Ie0c0de09b675ef93dc195a6470752a772083029e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/238425
Auto-Submit: Markus Handell <handellm@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35389}
2021-11-19 12:53:33 +00:00
Niels Möller
13d163654a Delete support for has_internal_source
Bug: webrtc:12875
Change-Id: I9683e71e1fe5b24802033ffcb32a531ca685fc6f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/179220
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35353}
2021-11-16 11:29:40 +00:00
Byoungchan Lee
efe46b6bee Change the type of RTCVideoSourceStats.framesPerSecond
Spec: https://w3c.github.io/webrtc-stats/#dom-rtcvideosourcestats-framespersecond

Bug: webrtc:12905
Change-Id: If53e2e480e2d6f687c3f8bb95a9e1d1e386fe9c2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237420
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Byoungchan Lee <daniel.l@hpcnt.com>
Cr-Commit-Position: refs/heads/main@{#35352}
2021-11-16 11:21:41 +00:00
Markus Handell
ee03431107 WebRtcVideoEngineTest: use simulated time.
Bug: chromium:1255737
Change-Id: I6036ae5af4b3f0e7bd04352b055935f501ecc52b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237341
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35316}
2021-11-05 14:06:12 +00:00
philipel
83d667925f Removed unused WebRTC-SupportVP9SVC field trial.
Instead use `parameters_.config.rtp.ssrcs.size()` directly to make decisions about the number of temporal and spatial layer used.

Bug: none
Change-Id: Icba553178ae7fea281c2c67654c510228d9ab5b6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/237080
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35299}
2021-11-02 16:05:52 +00:00
Mirko Bonadei
e5e78c4521 Fix -Wunused-but-set-variable.
Bug: None
Change-Id: I8943227108e46c4c942895e4bd8fb276947502e0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/236525
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35272}
2021-10-28 12:53:49 +00:00
Taylor Brandstetter
b9b0890541 Fix error adding receive stream after packet received from non-primary SSRC.
The non-primary SSRC being RTX, for example. Normally a default stream
wouldn't be created from RTX packets, but there is a window of time
where packets can be received before the video engine has receive
parameters/payload type mappings, so it creates one anyway.

Then in AddRecvStream, normally the default stream would be destroyed
before creating a new one, but this only happens for sp.first_ssrc().
Resulting in the error "Receive stream with SSRC 'X' already exists".

Fixed by simply iterating over all SSRCs.

Bug: webrtc:13171
Change-Id: Iaf4e4a3ceafddee3d9b2d1e24af68be56f9695de
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/231633
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Taylor Brandstetter <deadbeef@webrtc.org>
Commit-Queue: Taylor Brandstetter <deadbeef@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34971}
2021-09-10 21:28:23 +00:00
Tommi
1f38a38b6f Add ability to set rtp header extensions without recreating streams.
Setting the rtp header extensions on the packet delivery thread
(currently worker, soon to be network), is now possible without
taking the hit of deleting and recreating the receive stream (and
rtp receiver and related state).

Bug: webrtc:11993
Change-Id: I9bbe306844a25d85d79cd216092ead66eaf68960
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/223741
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#34953}
2021-09-08 13:39:36 +00:00
Åsa Persson
fb1959625d Allow setting different number of temporal layers per simulcast layer.
Setting different number of temporal layers is supported by SimulcastEncodeAdapter and LibvpxVp8Encoder will fallback to SimulcastEncoderAdapter if InitEncode fails.

Bug: none
Change-Id: I8a09ee1e6c70a0006317957c0802d019a0d28ca2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/228642
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34785}
2021-08-17 13:33:55 +00:00
Artem Titov
37f664f6d5 Use backticks not vertical bars to denote variables in comments for /media
Bug: webrtc:12338
Change-Id: Ia800a4017ede1f647b36f809ef3c5b37a2616fdd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/226949
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#34567}
2021-07-27 17:11:33 +00:00