221 Commits

Author SHA1 Message Date
Danil Chapovalov
ec2670e631 Cleanup ReportBlockData class: use Timestamp and TimeDelta
Bug: webrtc:13757
Change-Id: Ic3ddb05413f58cedd12bf0f32c852befb9bd40f4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/300940
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39841}
2023-04-13 08:51:12 +00:00
philipel
40cb0091a1 Unnest VideoEncoderFactoryTemplate in webrtc_video_engine_unittest.cc
Bug: webrtc:13573
Change-Id: I43517b6b7a130704803ff149b8a738ed4713d88a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/300361
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39767}
2023-04-05 15:13:36 +00:00
philipel
44437d35cd Replace BuiltinVideo{Encoder,Decoder}Factory with Video{Encoder,Decoder}FactoryTemplate.
Bug: webrtc:13573
Change-Id: I4eb5bb99bfde9c38c1d4072b933ef11a9ca32f53
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/299703
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39743}
2023-04-03 09:39:08 +00:00
Henrik Boström
75ea06f0fa Revert "Ship ability to opt-in to VP9/AV1 simulcast."
This reverts commit 75990b9a8f98ea2d597a31472fb778ec4d55f698.

Reason for revert: Breaks downstream, a use case of having three VP9
encodings, scalability mode only specified on the first layer
(L2T2_KEY) and the other two layers not having a scalability mode but
also being active=false appears to trigger a DCHECK in
call/rtp_video_sender.cc:501. More investigation needed

Original change's description:
> Ship ability to opt-in to VP9/AV1 simulcast.
>
> With this unflagging, an app can opt-in to simulcast when using multiple
> encodings by specifying RTCRtpEncodingParameters.scalabilityMode. This
> ensures backwards-compat with apps relying on 3 encodings to mean SVC
> who traditionally have not specified scalabilityMode.
>
> It fixes the spec/API bug of asking for simulcast and not getting
> simulcast. The field trial exists only as a kill-switch with a TODO to
> remove it.
>
> This ships initial support, however note that the VP9/AV1 simulcast uses
> SimulcastRateAllocator (just like VP8/H264 simulcast). This rate
> allocator uses more kbps than SvcRateAllocator. This should be revisited
> to avoid significant higher bitrates, for example when comparing VP9
> simulcast to VP9 SVC.
>
> Shipping the ability for apps to opt-in makes it easier to exercise
> these new code paths and allows initial feedback from developers, but
> due to the high bitrate (= same bitrate as VP8/H264 simulcast today)
> many apps may find that VP9 SVC is still more beneficial for BW reasons.
>
> Bug: webrtc:14884, webrtc:15005
> Change-Id: I748aae1adb47acc8a6b79b5852cff6aa47a46f5d
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/298046
> Commit-Queue: Henrik Boström <hbos@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#39601}

Bug: webrtc:14884, webrtc:15005
Change-Id: Ic8f77e6a2971f493d6cd8c23faecd435058a8847
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/298440
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39605}
2023-03-20 12:47:21 +00:00
Henrik Boström
75990b9a8f Ship ability to opt-in to VP9/AV1 simulcast.
With this unflagging, an app can opt-in to simulcast when using multiple
encodings by specifying RTCRtpEncodingParameters.scalabilityMode. This
ensures backwards-compat with apps relying on 3 encodings to mean SVC
who traditionally have not specified scalabilityMode.

It fixes the spec/API bug of asking for simulcast and not getting
simulcast. The field trial exists only as a kill-switch with a TODO to
remove it.

This ships initial support, however note that the VP9/AV1 simulcast uses
SimulcastRateAllocator (just like VP8/H264 simulcast). This rate
allocator uses more kbps than SvcRateAllocator. This should be revisited
to avoid significant higher bitrates, for example when comparing VP9
simulcast to VP9 SVC.

Shipping the ability for apps to opt-in makes it easier to exercise
these new code paths and allows initial feedback from developers, but
due to the high bitrate (= same bitrate as VP8/H264 simulcast today)
many apps may find that VP9 SVC is still more beneficial for BW reasons.

Bug: webrtc:14884, webrtc:15005
Change-Id: I748aae1adb47acc8a6b79b5852cff6aa47a46f5d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/298046
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39601}
2023-03-20 10:41:15 +00:00
Markus Handell
28c4986e1b WebRTCVideoChannel::OnPacketReceived: avoid PostTasks.
Under the combined network/worker thread project, tasks
are unnecessarily posted to the same thread. Avoid this
by posting only if invoked on a diffferent sequence.

TESTED=presubmit + local Meet calls.

Bug: webrtc:137439
Change-Id: Ic5dd99e5fbb843ad4c54d4466138135ae81596cf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/295867
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39471}
2023-03-03 15:27:45 +00:00
Harald Alvestrand
16579cc81d Change MediaChannel to have a Role parameter
This allows MediaChannel to know whether it's being used
for sending, receiving, or both. This is a preparatory CL
for landing the split of MediaChannel usage into sending and
receiving objects.

Bug: webrtc:13931
Change-Id: If518c8b53d5256771200a42e1b5f2b3321d26d8c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/292860
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39283}
2023-02-09 14:29:08 +00:00
Per K
c5455e7b53 Allow RTX ssrc to be updated on receive streams
This is used when an unsignaled stream with a known payload type is received and later a RTX packet is received.

Bug: webrtc:14817
Change-Id: I29f43281cec17553e1ec2483e21b8847714d2931
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291328
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39243}
2023-02-01 12:54:46 +00:00
Per K
217b384c1b Remove rtp header extension from config of Call audio and video receivers
These configurations are no longer used by call. Header extensions are identified once when demuxing packets in WebrtcVideoEngine::OnPacketReceived and WebrtcVoiceEngine::OnPacketReceived.

Change-Id: I49de9005f0aa9ab32f2c5d3abcdd8bd12343022d
Bug: webrtc:7135
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291480
Owners-Override: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39236}
2023-01-31 11:58:43 +00:00
Per K
438b5b4ca5 WebRtcVideoChannel creates default stream with dummy SSRC on received RTX packet.
This ensure transport feedback is sent for RTX packets that are received before media payload packets.

Bug: webrtc:14795, webrtc:14817
Change-Id: I6a2579b87c8863e003decb2b2559ef51a852cadb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291119
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39174}
2023-01-23 14:37:49 +00:00
Åsa Persson
e6b4cbe606 Add SVC fallback.
Fallback to a default value if the scalability mode is unset or not supported by the codec.

The fallback logic is only enabled if the scalability mode is configured for any of the encodings for now (i.e. initial default values are not set).

Bug: webrtc:11607
Change-Id: Ie632767b627a1dbbef71c59f9340573daf386c14
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/287600
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39074}
2023-01-11 16:49:49 +00:00
Per K
89ca299161 Use parsed packet from RtpTransport::DemuxPacket in engine and call
With this cl, a packet is only parsed once in RtpTransport::DemuxPacket and the metadata is reused.
Extensions are still identified twice- one for demuxing based on mid. The second time in Channel::OnReceivedPacket in order to use extensions specific to that mid.

Bug: webrtc:7135, webrtc:14795
Change-Id: I50e3814af92ca4378f148876b20a54bcfac1e146
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/290540
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39058}
2023-01-10 15:06:50 +00:00
Per K
17c4ca8fb3 Use RtpPacketReceived in media/engine/webrtc_video_engine_unittest.cc
This is done in preparation of https://webrtc-review.googlesource.com/c/src/+/290540

Bug: webrtc:7135, webrtc:14795
Change-Id: Ia9c5a34afc040a101403de52f8d22ec68531070e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/290576
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#39029}
2023-01-09 10:45:36 +00:00
Harald Alvestrand
1251c6418e Split stats generation for MediaChannel into sender and receiver APIs
This is in preparation for splitting MediaChannel into sender and
receiver channels, with independent objects.

Bug: webrtc:13931
Change-Id: I8e34b0c80b4d76132394efcda658a8face3ab873
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/288750
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38998}
2023-01-04 14:17:20 +00:00
Per K
9253240305 Reland "Remove use of ReceiveStreamRtpConfig:transport_cc"
This is a reland of commit 97ba853295578975a04fc504315cccd465f9f0bd
This cl did not cause the regression in Chrome rolls https://chromium-review.googlesource.com/c/chromium/src/+/4132644?tab=checks. Real culprit reverted in https://webrtc-review.googlesource.com/c/src/+/290502.

Original change's description:
> Remove use of ReceiveStreamRtpConfig:transport_cc
>
> With this change, webrtc will send RTCP transport feedback for all received packets that have a transport sequence number, if the header extension
> http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions is negotiated.
> I.e the SDP attribute a=rtcp-fb:96 transport-cc is ignored.
>
>
> Change-Id: I95d8d4405dc86a2f872f7883b7bafd623d5f7841
>
> Bug: webrtc:14802
> Change-Id: I95d8d4405dc86a2f872f7883b7bafd623d5f7841
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/290403
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#38980}

Bug: webrtc:14802
Change-Id: Ib98e61413161d462da60144942cdb0140e12bc42
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/290503
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38997}
2023-01-04 11:35:19 +00:00
Olga Sharonova
be5c7135f9 Revert "Remove use of ReceiveStreamRtpConfig:transport_cc"
This reverts commit 97ba853295578975a04fc504315cccd465f9f0bd.

Reason for revert: Suspected in breaking WebRTC into Chrome rolls https://chromium-review.googlesource.com/c/chromium/src/+/4132644?tab=checks

Original change's description:
> Remove use of ReceiveStreamRtpConfig:transport_cc
>
> With this change, webrtc will send RTCP transport feedback for all received packets that have a transport sequence number, if the header extension
> http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions is negotiated.
> I.e the SDP attribute a=rtcp-fb:96 transport-cc is ignored.
>
>
> Change-Id: I95d8d4405dc86a2f872f7883b7bafd623d5f7841
>
> Bug: webrtc:14802
> Change-Id: I95d8d4405dc86a2f872f7883b7bafd623d5f7841
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/290403
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#38980}

Bug: webrtc:14802
Change-Id: I2b04274466a5a81d767a48ff2e001b0a04f7f541
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/288943
Reviewed-by: Christoffer Jansson <jansson@webrtc.org>
Auto-Submit: Olga Sharonova <olka@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Christoffer Jansson <jansson@webrtc.org>
Owners-Override: Christoffer Jansson <jansson@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#38988}
2023-01-03 16:18:08 +00:00
Per K
97ba853295 Remove use of ReceiveStreamRtpConfig:transport_cc
With this change, webrtc will send RTCP transport feedback for all received packets that have a transport sequence number, if the header extension
http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions is negotiated.
I.e the SDP attribute a=rtcp-fb:96 transport-cc is ignored.


Change-Id: I95d8d4405dc86a2f872f7883b7bafd623d5f7841

Bug: webrtc:14802
Change-Id: I95d8d4405dc86a2f872f7883b7bafd623d5f7841
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/290403
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38980}
2023-01-03 09:44:26 +00:00
Harald Alvestrand
d3ac3b668a Remove AsSendChannel/AsReceiveChannel methods
This CL removes a couple more opportunities for client code
to interact directly with the MediaChannel implementation classes.

No-try because of infra failure.

No-Try: true
Bug: webrtc:13931
Change-Id: I658b8b04eff11de7831a1933d16d40fc59c3f0fc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/288380
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38935}
2022-12-21 07:47:32 +00:00
Harald Alvestrand
c0d44d9d63 Split audio and video channels into Send and Receive APIs.
The implementation here has a number of changes that force the callers
that called the "channel" functions into specific interfaces rather than
just letting C++ take care of it; this should go away once there stops
being a common implementation class for those interfaces.

Bug: webrtc:13931
Change-Id: Ic4e279528a341bc0a0e88d2e1e76c90bc43a1035
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/287640
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38888}
2022-12-14 11:00:17 +00:00
Florent Castelli
acabb3641b pc: Add asynchronous RtpSender::SetParameters() call
As the synchronous version only posts a task to recreate the encoder
later, it is not possible to catch errors and state changes that
could appear then.
The asynchronous version of SetParameters() aims to solve this by
providing a callback to wait for the completion of the encoder
reconfiguration, allowing any error to be propagate and subsequent
getParameters() call to have up to date information.

Bug: webrtc:11607
Change-Id: I5548e75aa14a97f8d9c0c94df1e72e9cd40887b2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/278420
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38627}
2022-11-15 15:31:40 +00:00
Henrik Boström
262d9cca08 [Stats] Fix outbound-rtp.active bug in SVC.
The code for determining outbound-rtp.active assumed, as the spec says,
that we have one RtpEncodingParameters per RTP stream. Unfortunately
SVC is currently implemented as one RtpEncodingParameters per SVC
layer. This causes a discrepency where we do correctly only have one
outbound-rtp stats object, but the lookup to check whether or not we are
"active" needs to look at more than a single encoding.

The bug is that if SVC layers are {inactive, active, active} then
stats reports outbound-rtp.active: false. With this fix, active: true is
reported if ANY of the SVC layers are active.

For singlecast or simulcast this CL has no change in behavior. In these
cases we have the same number of outbound-rtp and encodings and a simple
ssrc lookup does work.

The fix is exercised by unit tests and has also manually been confirmed:
- Singlecast tested by https://jsfiddle.net/henbos/nvd6p4j1/.
- Simulcast tested by https://crbug.com/webrtc/14628#c11.
- SVC tested by Google Meet and chrome://webrtc-internals/.

Bug: webrtc:14628
Change-Id: Ib89945caf29c8f4b85dd8a1120dcf8279296e4a3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/282222
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38569}
2022-11-07 17:20:30 +00:00
Henrik Boström
d81992197c [Stats] Update totalPacketSendDelay to only cover time in pacer queue.
This metric was always supposed to be the spec's answer to
googBucketDelay, and is defined as "The total number of seconds that
packets have spent buffered locally before being transmitted onto the
network." But our implementation measured the time between capture and
send, including encode time. This is incorrect and yields a much larger
value than expected.

This CL updated the metric to do what the spec says. Implementation-wise
we measure the time between pushing and popping each packet from the
queue (in modules/pacing/prioritized_packet_queue.cc).

The spec says to increment the delay counter at the same time as we
increment the packet counter in order for the app to be able to do
"delta totalPacketSendDelay / delta packetSent". For this reason,
`total_packet_delay` is added to RtpPacketCounter. (Previously, the
two counters were incremented on different threads and observers.)

Running Google Meet on a good network, I could observe a 2-3 ms average
send delay per packet with this implementation compared to 20-30 ms
with the old implementation. See b/137014977#comment170 for comparison
with googBucketDelay which is a little bit different by design -
totalPacketSendDelay is clearly better than googBucketDelay.

Since none of this depend on the media kind, we can wire up this metric
for audio as well in a follow-up:
https://webrtc-review.googlesource.com/c/src/+/280523

Bug: webrtc:14593
Change-Id: If8fcd82fee74030d0923ee5df2c2aea2264600d4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/280443
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38480}
2022-10-26 21:29:20 +00:00
Evan Shrubsole
09da10e24f Add powerEfficientDecoder and powerEfficientEncoder stats
The spec for these are at https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-powerefficientdecoder and https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-powerefficientdecoder

These stats are based on the is_hardware_accelerated boolean in both the
DecoderInfo and EncoderInfo structs.

Bug: webrtc:14483
Change-Id: I4610da3c6ae977f5853a3b3424d91d864fe72592
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/274409
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38441}
2022-10-19 13:15:31 +00:00
Sergio Garcia Murillo
d1196636dc Recreate default receive stream when unsignaled ssrc for rtx is received
Bug: webrtc:10297
Change-Id: Ie166b778b30e1c178c765f348e38426a6d57e29f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/278682
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38378}
2022-10-13 08:34:05 +00:00
Jonas Oreland
80c87d7151 RtpEncodingParameters::request_resolution patch 2
This cl/ implements configuring of encode resolution
in the video_stream_encoder (webrtc_video_engine) in
a way that is independent of frame resolution (i.e
not using scale_resolution_down_by).

The cl/ reuses the VideoAdapter as is, and hence
the output resolution will be the same as it is today.

Anticipated further patches
3) Hook up resource adaptation
4) Let VideoSource do adaption if possible

Bug: webrtc:14451
Change-Id: I881b031c5b23be26cacfe138730154f1cb1b66a8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/276742
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38245}
2022-09-29 14:10:44 +00:00
Johannes Kron
bb591c49e8 Change the default setting for PreStreamDecoders/LazyDecoderCreation
The experiment has been approved for a full launch. Changing the
default value so that no decoder is created before the stream starts.
All decoders are created lazily on demand when we receive payload
data of the corresponding type.

Bug: chromium:1319864
Change-Id: Ifb412bbe49a7577a45c340496d5b8572ebc1ba44
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/277120
Auto-Submit: Johannes Kron <kron@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38232}
2022-09-28 11:40:22 +00:00
Jonas Oreland
1262eb5ebc Move EncoderStreamFactory into own file
This cl/ is a NOP refactoring,
moving the EncoderStreamFactory from within webrtc_video_engine.cc
into own file in video/. simulcast.cc is collateral.

Bug: webrtc:14451
Change-Id: Ia69b9241d8cd8a12be6628d887701f2e244c07cc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/276861
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#38224}
2022-09-27 17:29:11 +00:00
Erik Språng
7aaeb5a270 Create pre-allocated decoders on the decoder thread.
This way we're sure instantiation, configuration and decode calls all
happen on the decoder queue - making thread checking easier in the
actual decoder classes.

Bug: None
Change-Id: Ia98f47009f26b34eb8dad2ee0b4ddcde082d1994
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/272022
Reviewed-by: Evan Shrubsole <eshr@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37825}
2022-08-18 13:14:26 +00:00
Tommi
dfc4d81fd3 [WebRtcVideoReceiveStream] Construct flexfec stream on demand.
...for payload type changes and avoid recreating the main video stream.

Bug: webrtc:11993
Change-Id: I03e44ff25d88caeb082a2e44b2e802d3b9392feb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269244
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37666}
2022-08-02 16:33:04 +00:00
Tommi
185f10c082 [WebRtcVideoReceiveStream] Add ability to config flexfec post init.
Bug: webrtc:11993
Change-Id: I35d7e645e18b7cb4a86645ea52c8958063f13d3c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269243
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37661}
2022-08-02 11:37:26 +00:00
Erik Språng
a3f2e72008 Fix potential UAF in unit test.
In WebRtcVideoChannelBaseTest.EncoderSelectorSwitchCodec a mock encoder
selecter or stack allocated and then registered with the channel.
Since this test uses real-time clocks/threads, there is a chance that
the selector callback will be called after the mock goes out of scope,
but before the test had time to be torn down.

This CL fixes that by simply de-registering the callback before the
end of the test.

Bug: b/239855550
Change-Id: Ibb38a914933494fd04c963b9a13f2cc4aee160d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269402
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37618}
2022-07-26 09:19:34 +00:00
Erik Språng
5793ed8ff9 Remove unused field trial WebRTC-SendBufferSizeBytes
Bug: webrtc:9637
Change-Id: Ie4bf5caa3977f1824f6dc1014b1feacf6ebfe670
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269386
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37617}
2022-07-26 07:26:14 +00:00
Erik Språng
d460c31309 Make sure VideoStream target bitrate is not higher than max.
Bug: webrtc:14017
Change-Id: Ic59811f4ec84e429839440cea8a70c05d7795cc4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/269388
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37614}
2022-07-25 20:34:48 +00:00
Erik Språng
b6ff84b516 Reland "When VP9 SVC is used, use SvcConfig to set max bitrate for the stream."
This is a reland of commit 3afb8e24311dc1297150d4011894b6cb00841735

Patchset 1 is the original CL. Patchset 2 contains a fix:
Depending on call site, the number of spatial layers for VP9 might be
signalled in three different ways. One of them was afaict only used in
out perf tests, and resulted in the max bitrate being incorrectly
capped.
The fix now checks that field too.

Original change's description:
> When VP9 SVC is used, use SvcConfig to set max bitrate for the stream.
>
> Currently, a default max bitrate is determined within WebRtcVideoEngine,
> which maxes out at 2.5Mbps - and that limits the max bitrate deteremined
> by SvcConfig for resolutions above 720p.
>
> This does not affect simulcast, as WebRtcVideoEngine already knows to
> trust the rate allocation in simulcast.cc instead.
>
> Bug: webrtc:14017
> Change-Id: I0c310a6fd496e9e5a10eae45838900068aa1ae2d
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267160
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#37370}

Bug: webrtc:14017, webrtc:14234
Change-Id: Idcaf4321a20c917e4049522c577336ddcfc7ffbb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267860
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37446}
2022-07-05 14:19:33 +00:00
Erik Språng
4f1af1156f Revert "When VP9 SVC is used, use SvcConfig to set max bitrate for the stream."
This reverts commit 3afb8e24311dc1297150d4011894b6cb00841735.

Reason for revert: Causes some unexpected perf regressions.

Original change's description:
> When VP9 SVC is used, use SvcConfig to set max bitrate for the stream.
>
> Currently, a default max bitrate is determined within WebRtcVideoEngine,
> which maxes out at 2.5Mbps - and that limits the max bitrate deteremined
> by SvcConfig for resolutions above 720p.
>
> This does not affect simulcast, as WebRtcVideoEngine already knows to
> trust the rate allocation in simulcast.cc instead.
>
> Bug: webrtc:14017
> Change-Id: I0c310a6fd496e9e5a10eae45838900068aa1ae2d
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267160
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#37370}

Bug: webrtc:14017
Change-Id: I1e45ee3f78deb50a9057d648146b1a6360782aa3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267800
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37438}
2022-07-05 10:25:23 +00:00
Niels Möller
c8152fe4a8 Update/delete old TODOs
Bug: webrtc:10198
Change-Id: I226768c2a6bd97ffcd0638e5bc6a1c286b71815f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267704
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37435}
2022-07-05 09:44:53 +00:00
Erik Språng
3afb8e2431 When VP9 SVC is used, use SvcConfig to set max bitrate for the stream.
Currently, a default max bitrate is determined within WebRtcVideoEngine,
which maxes out at 2.5Mbps - and that limits the max bitrate deteremined
by SvcConfig for resolutions above 720p.

This does not affect simulcast, as WebRtcVideoEngine already knows to
trust the rate allocation in simulcast.cc instead.

Bug: webrtc:14017
Change-Id: I0c310a6fd496e9e5a10eae45838900068aa1ae2d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/267160
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37370}
2022-06-29 14:23:08 +00:00
Philipp Hancke
d970b0901b measure decode time in TimeDelta instead of ms
increasing precision since summing up rounded values leads to
a rounding error, in particular for small frames which take very
little time to decode.

BUG=webrtc:12526,webrtc:13756

Change-Id: I647c702808856a002c746ed9f115aa9bcaddc1f3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262810
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philipp Hancke <philipp.hancke@googlemail.com>
Cr-Commit-Position: refs/heads/main@{#37249}
2022-06-17 09:57:27 +00:00
Jonas Oreland
6545516a14 RtpSenderInterface::SetEncoderSelector
This cl/ adds a way of setting an EncoderSelector on a specific
RtpSenderInterface. This makes it possible to easily use different
EncoderSelector on different streams within the same or different PeerConnections.

The cl/ is almost identical to the impl. of RtpSenderInterface::SetFrameEncryptor.

Iff a EncoderSelector is set on the RtpSender, it will take precedence
over the VideoEncoderFactory::GetEncoderSelector.

Bug: webrtc:14122
Change-Id: Ief4f7c06df7f1ef4ce3245de304a48e9de0ad587
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/264542
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37150}
2022-06-08 11:06:52 +00:00
Tommi
a136ed4085 Add SetTransportCc to ReceiveStreamInterface.
Setting the transport cc flag was only possible post creation for
audio receive streams, while video receive streams need to be recreated.

This CL moves the setter for transport_cc() to where the getter is and
adds boiler plate implementations for the video streams. For audio
streams this splits "SetUseTransportCcAndNackHistory" into two methods,
SetTransportCc and SetNackHistory.

Bug: none
Change-Id: Idbec8217aef10ee77907cebaecdc27b4b0fb18e4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/264443
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37038}
2022-05-30 14:07:04 +00:00
Harald Alvestrand
485457f050 Delete ChannelManager class
Bug: webrtc:13931
Change-Id: I331aed0e304f89a0c53d8db20ab2c9733ebbb34c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/263120
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36970}
2022-05-23 10:06:26 +00:00
Harald Alvestrand
c3fa7c38b2 Remove remaining trampoline functions from channel_manager
This is part of the project to delete the class entirely.
The CL also adds an "use_rtx" parameter to the function for listing
video codecs, rather than filtering those away afterwards.

Bug: webrtc:13931
Change-Id: I96b9b18c694a1c0986ccf22face76ef4c704d372
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262666
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36963}
2022-05-23 08:09:06 +00:00
Tommi
f6f4543304 Rename VideoReceiveStream to VideoReceiveStreamInterface
Bug: webrtc:7484
Change-Id: I653cfe46486e0396897dd333069a894d67e3c07b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262769
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36958}
2022-05-22 10:54:38 +00:00
Niels Möller
83830f316e Delete TestListener and top-level thread wrapping.
Instead use rtc::AutoThread in tests that need that.

Bug: webrtc:9714
Change-Id: I1f33b1b2d321770d062504dd9ef86d66a345dd42
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254681
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36950}
2022-05-20 15:21:21 +00:00
Philipp Hancke
0359ba2225 stats: add frame assembly time stats
implements a total frame assembly time statistic that measures the
cumulative time between the arrival of the first packet of a frame
(the lowest reception time) and the time all packets of the frame have
been received (i.e. the highest reception time)

This is similar to totalProcessingDelay
  https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-totalprocessingdelay
in particular with respect to only being incremented for frames that are being decoded but does not include the amount of time spent decoding the frame.

This statistic is useful for evaluating mechanisms like NACK and FEC
and gives some insight into the behavior of the pacer sending the
packets.
Note that for frames with just a single packet the assembly time will be zero. In order to calculate an average assembly time an additional frames_assembled_from_multiple_packets counter for frames with more than a single packet is added.

Currently this is a nonstandard stat so will only show up in webrtc-internals and not in getStats. Formally it can be defined as

totalAssemblyTime of type double
	Only exists for video. 	The sum of the time, in seconds, each video frame takes from the time the first RTP packet is received (reception timestamp) and to the time the last RTP packet of a frame is received.
    Given the complexities involved, the time of arrival or the reception timestamp is measured as close to the network layer as possible.

    This metric is not incremented for frames that are not decoded, i.e., framesDropped, partialFramesLost or frames that fail decoding for other reasons (if any). Only incremented for frames consisting of more than one RTP packet. The average frame assembly time can be calculated by dividing the totalAssemblyTime with framesAssembledFromMultiplePacket.

framesAssembledFromMultiplePacket of type unsigned long
	Only exists for video. It represents the total number of frames correctly decoded for this RTP stream that consist of more than one RTP packet.
	For such frames the totalAssemblyTime is incremented.

BUG=webrtc:13986

Change-Id: Ie0ae431d72a57a0001c3240daba8eda35955f04e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260920
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36922}
2022-05-18 09:16:10 +00:00
Asa Persson
2698353d43 Disable quality scaling if multiple spatial layers are requested for VP9
VP9 automaticResizeOn is disabled if more than one spatial layer is configured via scalability mode.

Bug: webrtc:13960
Change-Id: I7c6351bca6d2f32bcc7391894e8dcc9e74ca2050
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261315
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36897}
2022-05-16 15:11:24 +00:00
Niels Möller
807328fec7 Move frame drop config to VideoCodec and VideoEncoderConfig.
Intend to delete corresponding codec-specific settings in a followup.

Bug: webrtc:6883
Change-Id: I78ab07729a5aee1055f80d39d8f7289beb6721e1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262244
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36882}
2022-05-13 13:40:14 +00:00
Danil Chapovalov
8f44ae4fa5 Remove dead code in cricket::FakeVideoRenderer
Remove unused accessors
move helper implementation into .cc file

Bug: None
Change-Id: Iccd877180901b278af4800f681669089b8a046ce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262254
Commit-Queue: Niels Moller <nisse@webrtc.org>
Auto-Submit: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36881}
2022-05-13 13:36:35 +00:00
Tommi
cb7c7366d0 Separate reading remote_ssrc from using the rtp_config() getter.
`remote_ssrc` can be considered const while some other state represented
by rtp_config() can not and also is tied to a specific thread.
Separating access to these variables, makes moving things around easier.

Bug: webrtc:11993
Change-Id: I70aa000daab6174a401e01dca163213174e8f284
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261316
Commit-Queue: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36818}
2022-05-09 14:55:00 +00:00
Erik Språng
536f587b98 Revert "Calculate video stream max bitrate using expression."
This reverts commit 45361f78ed18c350b3edcaef19ae4c7cf167e95b.

Reason for revert: Perf alerts galore.

Original change's description:
> Calculate video stream max bitrate using expression.
>
> This replaces the ealier table-based caps.
> Apart from the VGA cap (now 1600kbps instead of 1700kbps), or if using
> "in between" resolutions, the caps are unchanged - but now cover high
> resolutions better.
>
> Bug: webrtc:14017
> Change-Id: I8649b528495d6c917e38ea8cb1a272df6c464c03
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260940
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Per Kjellander <perkj@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#36776}

Bug: webrtc:14017
Change-Id: I18ebc81c6054713c58d49bd227e37090686958c9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261309
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Auto-Submit: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#36794}
2022-05-06 09:37:32 +00:00