After landing this change, we can change the corresponding usage in
blink to start using presentation_timestamp as well and then delete
the remaining usage of capture_time_identifier.
Bug: webrtc:373365537
Change-Id: I0c4f2b6b3822df42d6e3387df2c243c3684d8a41
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/365640
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Commit-Queue: Palak Agarwal <agpalak@google.com>
Cr-Commit-Position: refs/heads/main@{#43317}
More test coverage for previously fixed bug
https://crbug.com/webrtc/369654168.
Two tests are added:
1. LibvpxVp9Encoder unit test that 4:2:1 720p can be reconfigured to
singlecast (which is what happens for encodings[0] in the bug).
2. Integration test that 4:2:1 720p can change to 180p,360p,540p.
This is the exact same test as was added in [1] but using
requested_resolution instead of scale_resolution_down_by.
[1] https://webrtc-review.googlesource.com/c/src/+/363941
Bug: webrtc:369654168
Change-Id: I83456b9254c1c6f647586d340d0fe5864b5515c6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/364200
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Auto-Submit: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#43185}
This allows to utilize libvpx optimizations considerably improving performance.
The change happens inside libvpx_vp9_encoder and is invisible to other parts of webrtc.
This CL includes unit tests, an E2E test already exists: StandardPath/PeerConnectionEncodingsIntegrationParameterizedTest.Simulcast/VP9 in peerconnection_unittests.
Bug: webrtc:347737882
Change-Id: I03bc27c920787a7305a9775e6341e26904592fb8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/360280
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42931}
Based on the results of the experiment (b/335129329).
Bug: webrtc:15827, b/320629637, b/335129329, chromium:329396373
Change-Id: I1599f4c1be79ee3385aac1ff345168982c8278f8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/360960
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Auto-Submit: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42895}
Libvpx was adjusted to support scenarios test verifies, but WebRTC tests were forgotten.
Bug: webrtc:42223649
Change-Id: I19a10c939d844d00dd564bc0a16fe21844cc7cfb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/357680
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42665}
This reverts commit 86ff48adaea08fd4e7044595e1c25a22fcceac34.
Reason for revert: Speculative revert due to failing downstream tests
Original change's description:
> Rewrite simulcast config to equivalent SVC for vp9 simulcast
>
> This allows to utilize libvpx optimizations considerably improving performance.
> The change happens inside libvpx_vp9_encoder and is invisible to other parts of webrtc.
>
> This CL includes unit tests, an E2E test already exists: StandardPath/PeerConnectionEncodingsIntegrationParameterizedTest.Simulcast/VP9 in peerconnection_unittests.
>
> Bug: webrtc:347737882
> Change-Id: Ic48316ad597700ed07e594d592413cf84b6b20d4
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355003
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#42554}
Bug: webrtc:347737882
Change-Id: Ib84c9c0e20763348abfae838f2fb1aff31581a55
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355943
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Jeremy Leconte <jleconte@google.com>
Reviewed-by: Jeremy Leconte <jleconte@google.com>
Commit-Queue: Jeremy Leconte <jleconte@google.com>
Cr-Commit-Position: refs/heads/main@{#42564}
This allows to utilize libvpx optimizations considerably improving performance.
The change happens inside libvpx_vp9_encoder and is invisible to other parts of webrtc.
This CL includes unit tests, an E2E test already exists: StandardPath/PeerConnectionEncodingsIntegrationParameterizedTest.Simulcast/VP9 in peerconnection_unittests.
Bug: webrtc:347737882
Change-Id: Ic48316ad597700ed07e594d592413cf84b6b20d4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/355003
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42554}
This is a cleanup in VP9 encoder wrapper. The removed code paths were only used in tests. In prod layers are configured explicitly via VideoCodec::spatialLayers[].
Bug: webrtc:42225151
Change-Id: I1de90039488b36e3c88e788c78e675bf2ee68f9b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/349222
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Auto-Submit: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#42250}
Example: "WebRTC-LibvpxVp9Encoder-SvcFrameDropConfig/Enabled,layer_drop_mode:1,max_consec_drop:7/"
It is only possible to enable LAYER_DROP (layer_drop_mode=1) for now. All other modes are ignored. Max consecutive frame drops (max_consec_drop) value from the field is always applied if the field trial is enabled.
LAYER_DROP requires flexible mode (is_flexible_mode_=true) which can be enabled by means of WebRTC-Vp9InterLayerPred: https://source.chromium.org/chromium/chromium/src/+/main:third_party/webrtc/media/engine/webrtc_video_engine.cc;l=976
Bug: webrtc:15827, b/320629637
Change-Id: I9c4d4838b11547e608d863198b109cb1485902d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/335041
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#41755}
Remove EncodedFrame::MissingFrame, as it was always false in actual
in-use code anyway, and remove usages of the Decode missing_frames param
within WebRTC. Uses/overrides in other projects will be cleaned up
shortly, allowing that variant to be removed from the interface.
Bug: webrtc:15444
Change-Id: Id299d82e441a351deff81c0f2812707a985d23d8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/317802
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Auto-Submit: Tony Herre <herre@google.com>
Commit-Queue: Tony Herre <herre@google.com>
Cr-Commit-Position: refs/heads/main@{#40662}
The preferred method to create codecs is to use the function
cricket::CreateAudioCodec or cricketCreateVideoCodec.
Empty codec objects are deprecated and should be replaced
with alternatives such as methods returning an
absl::optional object instead.
Bug: webrtc:15214
Change-Id: I7fe40f64673cd407830dbbb0e541b85a3aee93aa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/307521
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#40226}
This CL propagates capture_time_identifier introduced in
webrtc::VideoFrame and propagates it to EncodedImage. For use cases
involving EncodedTransforms, this identifier is further propagated to
TransformableVideoSenderFrame.
VideoEncoder::Encode function is overriden by each encoder. Each of
these overriden functions needs to be changed so that they can handle
this new identifier and propagate its value in the created EncodedImage.
Change-Id: I5bea4c5a3fe714f1198e497a4bcb5fd059afe516
Bug: webrtc:14878
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/291800
Reviewed-by: Tony Herre <herre@google.com>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Palak Agarwal <agpalak@google.com>
Cr-Commit-Position: refs/heads/main@{#39374}
This is a refactor to simplify a follow-up CL of adding
SdpVideoFormat::IsSameCodec.
The original files media/base/h264_profile_level_id.* and
media/base/vp9_profile.h must be kept until downstream projects
stop using them.
Bug: chroimium:1187565
Change-Id: Ib39eca095a3d61939a914d9bffaf4b891ddd222f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/215236
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Johannes Kron <kron@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33782}
This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
- To achieve this, WebRTC encoders are updated to map kNative video
buffers so that in a follow-up CL VideoStreamEncoder can stop mapping
intermediate buffer sizes.
In this CL LibvpxVp9Encoder is updated to map kNative buffers of pixel
formats it supports and convert ToI420() if the kNative buffer is
something else. A fake native buffer that keeps track of which
resolutions were mapped, MappableNativeBuffer, is added.
Because VP9 is currently an SVC encoder and not a simulcast encoder, it
does not need to invoke CropAndScale.
This CL also fixes MultiplexEncoderAdapter, but because it simply
forwards frames it only cares about the pixel format when
|supports_augmented_data_| is true so this is the only time we map it.
Because this encoder is not used with kNative in practise, we don't care
to make this path optimal.
Bug: webrtc:12469, chromium:1157072
Change-Id: I74edf85b18eccd0d250776bbade7a6444478efce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33526}
Since for such frame SvcController haven't setup how buffer should be
referenced and updated, the frame would likely have unexpected configuration.
Log an error to note resource have been wasted produce it and drop such frame.
Bug: webrtc:11999
Change-Id: I1784403e67b7207092d46016510460738994404e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205140
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33148}
Using WebRTC-VP9-PerformanceFlags and settings a multi-layer config,
and then configuring the codec in non-svc mode would cause us to not
set the cpu speed in libvpx. For some reason, that could trigger a
crash in the encoder.
This CL fixes that, and adds new test coverage for the code affected
byt the trial.
Bug: chromium:1167353, webrtc:11551
Change-Id: Iddb92fe03fc12bac37717908a8b5df4f3d411bf2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202761
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33051}
In particular move end_of_picture flag out of vp9 specific information
since VP9 is not the only codec that can use spatial scalability and
thus need to distinguish layer frame and picture (aka temporal unit).
Bug: webrtc:12167
Change-Id: I0d046d8785fbea55281209ad099738c03ea7db96
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/192542
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32588}
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.
This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.
Testing: Tested in Chrome with media::VideoFrame adapters.
Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
NV12 frames can be encoded by libvpx now, and this change allows for
encoding of them with VP9.
VP9 encode/decode tests now run with NV12 as well as I420.
Manually tested using video loopback with VP9 and NV12 generated frames.
out/Default/video_loopback.app/Contents/MacOS/video_loopback --clip=GeneratorNV12 --codec="VP9"
Bug: webrtc:11635, webrtc:11974
Change-Id: Ifc5cbf77d2a27821cd5560c253d5d447c7a7cf53
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185123
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32220}
This is a reland of d5925756980f6e82a55f57532c8d855e954459fb
Patchset 2 is a reland of
https://webrtc-review.googlesource.com/c/src/+/177012
Patchset 3 is a fix for a potential crash when InitDecode()is called from
VideoStreamDecoderImpl::GetDecoder(), where the decoder_settings
parameter is a but surprisingly set to nullptr.
Original change's description:
> VP9 decoder: Sets thread count based on resolution, reinit on change.
>
> Previously, number of decoder threads for VP9 were always set to 8 but
> with a cap at number of cores. This was done since we "can't know" the
> resolution that will be used.
>
> With this change, we now intialize the number of threads based on
> resolution given in InitDecode(). If a resolution change happens in
> flight, it requires a keyframe. We therefore parse the header from
> any key frame and if it has a new resolution, we re-initialize the
> decoder.
>
> The number of threads used is based on pixel count. We set one thread
> as target for 1280x720, and scale up lineraly from there. The 8-thread
> cap is gone, but still limit it core count.
>
> This means for instance: 1 <= 720p, 2 for 1080p, 4 for 1440p, 9 for 4K.
>
> Bug: webrtc:11551
> Change-Id: I14c169a6c651c50bd1b870c4b22bc4495c8448fd
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174460
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31507}
Bug: webrtc:11551
Change-Id: I2b4b146d0b8319f07ce1660202d6aa4b374eb015
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177246
Reviewed-by: Johannes Kron <kron@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31527}
Show it can make vp9 tests cleaner too.
Bug: None
Change-Id: I8333a61dec1ef90ade9faffea94e1555ccbfcfaa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177013
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31523}
This is more logical way to remove inactive lower layers.
Current way is to notify the encoder that the layer is inactive,
then renumber layers at the packatization level.
This Cl will allow to simplify libvpx vp9 encoder, svcRateAllocator and
vp9 packetizer.
Bug: webrtc:11319
Change-Id: Idf0bb30b729f5ecc97e31454b32934546b681aa2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173182
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31058}
This CL fixes a few issues where the reported fraction of frames
allocated to various temporal layers could be incorrect:
* In LibvpxVp8Encoder, calling GetEncoderInfo() while not initialized,
or when first configuring with temporal layers and then without,
could trigger incorrect fps allocations.
* In VP9 when different spatial layers have different max framerates,
the layer fps should be compared to the layer with the highest
configured fps, not codec_.maxFramerate which is updated to the
current input fps on SetRates().
* In EncoderBitrateAdjuster, just warn and ignore if a layer has
non-zero bps but zero fps, rather than passing down the chain and
risk weird behavior or divide by zero.
Bug: b/152040235
Change-Id: I548fb3e099b1ec9f536a7b93313fb40c4d32e596
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171516
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30880}
If e.g. CPU adaptation reduces input video size too much, video pipeline would
reduce the number of used simulcast streams/spatial layers. This may result in
disabled video if some streams are disabled by Rtp encoding parameters API.
Bug: webrtc:11319
Change-Id: Id7f157255599dcb6f494129b83477cda4bea982a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168480
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30498}
