Start to save local capture clock offset for video. This is part of a effort to add End 2 End metric on Android.
Bug: None
Change-Id: Icd6e567faf66f1dc200d8661344708356bda470b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/320300
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Olov Brändström <brandstrom@google.com>
Cr-Commit-Position: refs/heads/main@{#40764}
Change-Id: I118817fe9fc4d4e674268743ac7d6d2773d366de
Bug: webrtc:15496
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/320260
Reviewed-by: Michael Horowitz <mhoro@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Dan Tan <dwtan@google.com>
Cr-Commit-Position: refs/heads/main@{#40756}
The FrameCadenceAdapter starts a delayed task to request a
new refresh frame on receiving frame drop. However, the
resulting RepeatingTaskHandle was not Stop()ed on destruction,
leading to UAF.
Fixed: chromium:1478944
Change-Id: Iba441420953e989cfc7fcfd2f358b5b30f375786
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/320200
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40747}
logging the configuration, in particular the content type which
together with RTP configuration information like the ssrcs helps differentiating between encoders.
BUG=None
Change-Id: I1b4b2ec2bffea338cc73c3a9c6a3f775d8f1c26b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/319560
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#40744}
With an intent to use it instead of the SendSideDelayUpdated
Bug: None
Change-Id: Ifa2b76af6882b36b2ccca13d8038aa4fbb1a67fb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317801
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40725}
Reduce redundant map lookups,
On the way update one the time variable to Timestamp type
Bug: None
Change-Id: I0224bae866942a8d404e465bd2226befc9ce6763
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/319480
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40723}
The main goal of this change is to disable the quality scaler when multiple spatial layers are used.
Bug: b/295129711
Change-Id: I25e0b7440a8c2adee3e97720a1e0ee5e0a914334
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/319181
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40709}
The code currently issues frames for encode before scheduling
a new repeat. Swap this order to account for time taken by for
slow encodes.
Bug: webrtc:15456
Change-Id: I74177069e30c1bf65268231ffba033411a0f7b9a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/318580
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40690}
The synthetic delay added in ZeroHzAdapterMode::OnFrame does not
account for delay with respect to the initial frame post from
FrameCadenceAdapter::OnFrame. Fix this to account for time spent
in contention on the encode sequence.
Bug: webrtc:15456
Change-Id: I63446e8dfe8f62b09d972434a705e912f8a73d69
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/318420
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40675}
Remove EncodedFrame::MissingFrame, as it was always false in actual
in-use code anyway, and remove usages of the Decode missing_frames param
within WebRTC. Uses/overrides in other projects will be cleaned up
shortly, allowing that variant to be removed from the interface.
Bug: webrtc:15444
Change-Id: Id299d82e441a351deff81c0f2812707a985d23d8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317802
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Auto-Submit: Tony Herre <herre@google.com>
Commit-Queue: Tony Herre <herre@google.com>
Cr-Commit-Position: refs/heads/main@{#40662}
Remove VCMEncodedFrame from the inheritance chain of EncodedFrames by
- moving getters for EncodedImage fields up to EncodedImage
- copying other non-deprecated fields & Methods from VCMEncodedFrame over to EncodedFrame
- Removing EncodedFrame's inheritance of VCMEncodedFrame
We leave VCMEncodedFrame as part of the (near) deprecated
VideoCodingModule code. The only place which needs to accept either is
in the generic decoder.
Bug: webrtc:9378, b:296992877
Change-Id: I60706aebbb6eacc7fd4b35ec90cc903efdbe14c8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317160
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Auto-Submit: Tony Herre <herre@google.com>
Commit-Queue: Tony Herre <herre@google.com>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40639}
Remove redundant ssrc_ set, instead construct send_delay_stats_ early
Remove extra lookup when packet is sent out, instead memorize pointer to needed object
minor style improvments using syntax new in c++17
Bug: None
Change-Id: I0f0e28f5a01de0380502d4bee64cdf76e44a59cd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317760
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40630}
In fucntion EncoderStreamFactory::CreateSimulcastOrConferenceModeScreenshareStreams, the follow code allows TL for H264.
const bool temporal_layers_supported =
absl::EqualsIgnoreCase(codec_name_, kVp8CodecName) ||
absl::EqualsIgnoreCase(codec_name_, kH264CodecName);
However, the helper function IsTemporalLayersSupported does not allow TL for H264. The diff unifies the logic by using the helper function
Bug: webrtc:15442
Change-Id: I1497ccc1cd5d3715310e0485f9179bd8e6948f1a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317542
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Philipp Hancke <phancke@microsoft.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40629}
The wrapped encoders may sometimes shift the callback
threads, so SequenceChecker is not legits for this case.
Replaced with a Mutex.
Bug: b/296242528
Change-Id: I7b2e6e630563246d5214ff4f18c6855ba7869a92
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317460
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40609}
Expose new function MaybeCreateFrameDumpingEncoderWrapper that
wraps another passed encoder and dumps its encoded frames out
into a unique IVF file into the directory specified by the
"WebRTC-EncoderDataDumpDirectory" field trial. If the passed
encoder is nullptr, or the field trial is not setup, the function
just returns the passed encoder. The directory specified by the
field trial parameter should be delimited by ';'.
The new function is wired up in VideoStreamEncoder.
Bug: b/296242528
Change-Id: I6143adf899f78fcc03d4239a86c68dcbab483f1c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317200
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40600}
Remove code where integer -1 as delay is used to represent unset value.
Bug: webrtc:13756
Change-Id: I16a01e12c25a09ce21a971c9edabf47af5936662
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/316923
Auto-Submit: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40592}
which makes it possible to understand which error occured.
BUG=chromium:1366910
Change-Id: Ided288ea7aa7c6cb283f7d46692c67efb15764d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/316863
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40573}
Remove support for setting one limit without another limit
because related rtp header extension doesn't support such values.
Start morphing VideoPlayouDelay into a class and stricter type: add accessors returning TimeDelta
Bug: webrtc:13756
Change-Id: If0dd02620528dc870b015beeff3a8103e04022ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/315921
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40570}
This is a reland of commit 0e37f5ebd44183d9fe5318d844235aae28fda86a
with backward compability added to allow downstream tests to migrate to the new signature.
Original change's description:
> Fix definition of keyframes decoded statistics
>
> which are defined to be measured after decoding, not before:
> https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-keyframesdecoded
>
> BUG=webrtc:14728
>
> Change-Id: I0a83dde278e1ebe8acf787bdac729af369a1ecf8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/315520
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Philipp Hancke <phancke@microsoft.com>
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#40545}
BUG=webrtc:14728
Change-Id: I4cf52bb22ba8244155b4fa8c367b9c0306a77590
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/316120
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#40553}
It used input frame resolution before this change which caused unnecessary resolution adaptations when resolution scaling is used.
Found that initial frame dropping was always enabled for AV1 SVC. After fixing DropDueToSize the AV1 SVC tests [1] started to fail ("number of encoded temporal layers is less than expected") on bots. The tests encode 1850x1110 in L3T3 for 5s using the default 300kbps start bitrate. Before the fix the initial frame dropping kicked in and reduced the resolution to a level that let encoder to generate all temporal layers. After the fix the resolution stayed at 1850x1110 and encoder dropped all T1 and T2 layer frames. Mitigated this by increasing test duration from 5 to 10s. This gives enough time for BWE to ramp up and for encoder to generate (stop dropping) all temporal layers.
[1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/pc/test/svc_e2e_tests.cc;l=460;bpv=1
Bug: chromium:1466809
Change-Id: I16802689e234f8fc16f891f024d5f644985de01c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/315142
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40536}
This CL adds [[deprecated]] to the old signatures, and uses the new
signatures throughout.
Bug: webrtc:14870
Change-Id: Ic9a8198ac0a2f954e1b2e7d05a55dbe04342f958
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/314962
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40517}
EncoderStreamFactory has two code paths for creating a stream: the
"simulcast path" and the "default path". Only the former cares about
encoding paramter's maxBitrate. The latter assumes that
`encoder_config.max_bitrate_bps` already encompasses the maxBitrate of
the first encoding, but this is not always the case.
As of M113, when scalability mode is specified, {active,inactive} does
not count as simulcast stream but as a default stream represented by
encoding[0].
The problem is that `encoder_config.max_bitrate_bps` only includes
`encodings[0].max_bitrate_bps` when `encodings.size() == 1` which isn't
the case here.
This CL fixes the problem by making the "create default stream" code
path look at the first encoding's maxBitrate and remove existing
assumptions that `encoder_config.max_bitrate_bps` encompasses
`encodings[0].max_bitrate_bps`. This is a step in the right direction
since we're trying to remove all special cases and have encodings map
1:1 with SSRCs, so the "max bps of entire stream" should indeed be a
separate limit than the per-encoding limits and it was confusing that
sometimes it included and sometimes it excluded encoding[0]'s limit.
This issue did not happen in {inactive,active} since that code path
counts as "simulcast stream", so "default stream" is only ever
applicable for index 0.
TESTED=Simulcast Playground, see https://crbug.com/1455962.
Bug: chromium:1455962
Change-Id: I7c44925b780623b5979751e8959e972293648a3d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/313282
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40482}
It was discovered that if libvpx reported a scalability mode in getStats
(e.g. L3T3_KEY) and we then changed encoder implementation to an
RTCVideoEncoder (such as MediaFoundationVideoEncodeAccelerator),
getStats continued to report the old scalability mode value.
This CL makes sure to clear the scalability mode on encoder
implementation change or if the `codec_info` is missing.
We should update MediaFoundation to report L1T1 as well, but in the
meantime we should clear any old scalability modes values when the
implementation changes (if the scalability mode is not known it is
better to report nothing than to report an old misleading value).
Bug: chromium:1426440
Change-Id: I1b5f324c4d29a00a6c73404cbee0faa2ae9cd843
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/312900
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40467}
This extension is documented to carry one bit: Screenshare.
It's been used for carrying simulcast layers and experiment IDs.
This CL removes that usage.
Bug: webrtc:15383
Change-Id: I048b283cde59bf1f607d8abdd53ced07a7add6f8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/312420
Commit-Queue: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40457}
Without this, 'Sender' frames inserted into the writer of an encoded
transform have an invalid receive time (0), which breaks all later
heuristics which build on the receive time, eg the VCMTiming estimators
used for controlling the playback delay.
Bug: chromium:1463451
Change-Id: I413c884e08986148d4a854cd275212b21d093ceb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/311544
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Palak Agarwal <agpalak@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Tony Herre <herre@google.com>
Cr-Commit-Position: refs/heads/main@{#40416}
allowing for better correlation with MaybeEncodeVideoFrame
which also logs the ntp timestamp.
BUG=None
Change-Id: I00fc99e69cd703f6da3f25043361d68b3cb3f3fa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/311542
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40415}
It is partial reland, which adds call to Start() to all relevant places,
but doesn't actually switches frame generator to not produce frames from
the moment it was created.
Bug: b/272350185
Change-Id: I6e3bd7af6f5cd8d9baff79c2aada7b2ddfae1c8d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/310782
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40379}
which means this will not show up in getStats inbound-rtp/outbound-rtp
until the encoder/decoder is known. This has implications in particular
for inbound-rtp where the value is currently "unknown" until video
frames have been received.
This is safe to change as the previous change to gate
decoderImplementation behind getUserMedia access already broke
the assumption that the field is always string.
BUG=webrtc:14906
Change-Id: Ie6040ada3656e80f792c0c32c1b86ad1d6609d3c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/293600
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Philipp Hancke <phancke@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#40334}
There are now multiple ways to configure VP9 L1Tx:
- Legacy API: configure legacy SVC and disable encodings, this gets
interpreted as disabling spatial layers (non-standard API hack).
- Standard API: configure scalability_mode. This can be done either
with a single encoding or multiple encodings. As long as only one
encoding is active we get a single L1Tx ssrc, same as legacy API.
Due to a bug, the ApplySpatialLayerBitrateLimits() logic which tweaks
bitrates was only applied in the legacy API code path, not the standard
API code path, despite both code paths configuring L1Tx.
The issue is that IsSimulcastOrMultipleSpatialLayers() was checking if
`number_of_streams == 1`. This is true in legacy code path but not
standard code path. The fix is to look at
`numberOfSimulcastStreams == 1` instead, which is set to the correct
value regardless of code path used.
This CL adds comments documenting the difference between
`number_of_streams` and `numberOfSimulcastStreams` to reduce the risk
of more mistakes like this in the future.
Bug: chromium:1455039, b:279161263
Change-Id: I69789b68cc5d45ef1b3becd310687c8dec8e7c87
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/308722
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40287}
Most of the usage of the H264Encoder::Create(codec) method passes a
simple codec with just the H264 codec name. This simplified the call
sites in many places and removes references to the codec types.
Bug: webrtc:15214
Change-Id: I4039c0be4ce6e3147c14c7853df4635f344b7d70
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/307222
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40214}
