309 Commits

Author SHA1 Message Date
Artem Titov
d15a575ec3 Use SequenceChecker from public API
Bug: webrtc:12419
Change-Id: I00cca16a0ec70246156ba00b97aa7ae5ccbf5364
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205323
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33220}
2021-02-10 15:04:55 +00:00
Åsa Persson
7f354f8606 Use bandwidth allocation in DropDueToSize when incoming resolution increases.
Use bandwidth allocation instead of encoder target bitrate in DropDueToSize when incoming resolution increases to avoid downgrades due to target bitrate being limited by the max bitrate at low resolutions.

Bug: none
Change-Id: Ic41b31c1a86911d4e97b61b0cbc41ce0da739bd4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205622
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33168}
2021-02-04 17:26:21 +00:00
Åsa Persson
c91c4233e3 LibvpxVp9Encoder: add option to configure resolution_bitrate_limits.
Bug: none
Change-Id: Icdd7333296d652b1e0c159226df702084303475c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/204701
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33121}
2021-02-01 11:48:00 +00:00
Per Kjellander
1aa1d6463a Ensure VideoLayersAllocation.frame_rate_fps can not overflow
Bug: webrtc:12000
Change-Id: I14d5f0f987fb20bd74e0428b3791bf370476296e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/204220
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33089}
2021-01-28 12:39:30 +00:00
Jakob Ivarsson
461b1d903f Restart CPU overuse detection when encoder settings has changed.
This could potentially lead to unnecessary restarts since it is also
started after the encoder is created. However, it is needed since the
hardware acceleration support can change even though the encoder has
not been recreated.

Bug: b/145730598
Change-Id: Iad1330e7c7bdf769a68c4ecf7abb6abbf3a4fe71
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203140
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33060}
2021-01-22 17:10:12 +00:00
Åsa Persson
a7e34d33fe Add resolution_bitrate_limits to EncoderInfo field trial.
Added class EncoderInfoSettings for parsing settings.
Added use of class to SimulcastEncoderAdapter.

Bug: none
Change-Id: I8182b2ab43f0c330ebdf077e9f7cbc79247da90e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202246
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33050}
2021-01-21 07:53:57 +00:00
Åsa Persson
a24d35eb09 AlignmentAdjuster: take reduced layers into account for default downscaling.
Bug: none
Change-Id: Id70f7763d2e1b11c24ad98774f1bf6a661728437
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202257
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33038}
2021-01-19 16:59:11 +00:00
Per Kjellander
b03b6c8a94 Move setting of encoder bitrate allocation callback type to VideoSendStream
It turned out that the negotiated rtp header extensions are not fully known in WebRtcVideoChannel::AddSendStream.

The cl also remove the unnecessary factory for creating VideoStreamEncoder.


Bug: webrtc:12000
Change-Id: If994c8deb69f3ce4212896d3ad757dac94c6e09f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198840
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32916}
2021-01-07 09:29:05 +00:00
Per Kjellander
f86cf4c2de Add support for VideoLayersAllocation for Vp9 scv/ksvc and none scalable
VideoCodecInitializer::VideoEncoderConfigToVideoCodec is modified to always set correct frame rate, width and height on spatial layer 0 so the rest of the code does not need to differentiate between scalable/none scalable codecs.


Bug: webrtc:12000
Change-Id: I5a068b98ca2038621205f55e4024f949ab51587a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198540
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32890}
2020-12-30 16:45:03 +00:00
Henrik Boström
ad70609509 Implement fake PixelLimitResource for TestBed.
This CL implements a Resource that aggressively reports overuse or
underuse until the encoded stream has the max pixels specified. The
pixel limit is controlled with a field trial, e.g:

--force-fieldtrials="WebRTC-PixelLimitResource/Enabled-307200/"

This caps the resolution to 307200 (=640x480). This can be used by the
TestBed to simulate being CPU limited. Note that the resource doesn't
care about degradation preference at the moment, so if the degradation
preference would be set to "maintain-resolution" the PixelLimitResource
would never stop reporting overuse and we would quickly get a low-FPS
stream.

PixelLimitResource runs a repeating task and reports overuse, underuse
or neither every 5 seconds. This ensures we quickly reach the desired
resolution.

Unit tests are added. I did not add any integration tests (I think
that's overkill for a testing-only resource) but I have manually
verified that this works as intended.

This CL also moves the FakeVideoStreamInputStateProvider into a test/
folder and exposes video_stream_adapter.cc's GetLowerResolutionThan().

Bug: webrtc:12261
Change-Id: Ifbf7c4c05e9dd2843543589bebef3f49b18c38c0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195600
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32771}
2020-12-04 10:35:53 +00:00
Ilya Nikolaevskiy
cde4a9f669 Enable initial frame drop for SVC 'singlecast'
Bug: none
Change-Id: Ideda726f4f7df5e92556048a199cda06261e76b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195542
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32714}
2020-11-27 14:08:45 +00:00
Mirko Bonadei
20e4c80fbe Reland "Introduce RTC_NO_UNIQUE_ADDRESS."
This is a reland of f5e261aaf65cdf2eb903cdf40d651846be44f447

This CL disables RTC_NO_UNIQUE_ADDRESS on MSan builds since
there have been some issues.

Original change's description:
> Introduce RTC_NO_UNIQUE_ADDRESS.
>
> This macro introduces the possibility to suggest the compiler that a
> data member doesn't need an address different from other non static
> data members.
>
> The usage of a macro is to maintain portability since at the moment
> the attribute [[no_unique_address]] is only supported by clang
> with at least -std=c++11 but it should be supported by all the
> compilers starting from C++20.
>
> Bug: webrtc:11495
> Change-Id: I9f12b67b4422a2749649eaa6b004a67d5fd572d8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173331
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32246}

Bug: webrtc:11495, webrtc:12218
Change-Id: I4e6c7cc37d3daffad2407c9a2acfa897fa5b426a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/189968
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32668}
2020-11-23 11:29:36 +00:00
Åsa Persson
f46723c8aa Enable initial frame drop for one active simulcast layer.
Bug: webrtc:12216
Change-Id: Ib2ac2fab45e560ba3eae30a926ce72667a257b07
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193840
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32652}
2020-11-20 16:21:05 +00:00
philipel
87e99095a7 Make video scalability mode configurable from peerconnection level.
This CL does not aim at cleaning up simulcast/SVC configuration, just to make it possible to set the scalability mode for AV1. Implementing a codec agnostic SVC/simulcast API is a (big) project on its own.

Change-Id: Ia88df31eb1111713e5f8832e95c8db44f92887ca

BUG: webrtc:11607
Change-Id: Ia88df31eb1111713e5f8832e95c8db44f92887ca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/192541
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32631}
2020-11-18 12:06:03 +00:00
Per Kjellander
c4ad6062fd Ensure EncoderSink::OnBitrateAllocationUpdated not called after Stop
Bug: chromium:1143311, webrtc:12000
Change-Id: I149e960a4999442b289f4b3c576206cc6baf6f24
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193063
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32605}
2020-11-13 16:26:41 +00:00
Erik Språng
9d69cbeabf Changes default pacing factor to 1.1x
This changes the default behavior to use pacing factor of 1.1x instead
of 2.5x, it also sets libvpx rate controler as trusted, turns on the
encoder pushback mechanism and sets spatial hysteresis to 1.2.
The unused "dynamic rate" settings in libvpx is removed.

The new settings matches what has been used in chromium since 2019.
If needed, the legacy behavior can be enabled using the field trial
WebRTC-VideoRateControl.

Bug: webrtc:10155
Change-Id: I8186b491aa5bef61e8f568e96c980ca68f0c208f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186661
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32477}
2020-10-23 13:43:32 +00:00
Per Kjellander
a94348440b VideoStreamEncoder report VideoLayersAllocation for simulcast
Adds support for Vp8 simulcast.

Bug: webrtc:12000
Change-Id: Ib24fd0542642b023ec35f7a7bdc4880d72365edf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187341
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32416}
2020-10-15 17:39:06 +00:00
Evan Shrubsole
9577be3693 Use VideoFrameBuffer::CropAndScale in VideoStreamEncoder
This avoids a conversion to I420 for frames that support crop
and scale.

Bug: webrtc:11976
Change-Id: I6517a016403cff3ea7ebce1f3de9f9af8b569933
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187357
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32366}
2020-10-09 13:02:46 +00:00
Evan Shrubsole
b556b08668 Allow encoders to receive preferred pixel formats from native buffers
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.

This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.

Testing: Tested in Chrome with media::VideoFrame adapters.

Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
2020-10-08 13:47:50 +00:00
Per Kjellander
dcef6410b3 Stop using VideoBitrateAllocationObserver in VideoStreamEncoder.
VideoBitrateAllocation is instead reported through the EncoderSink.
Enable VideoBitrateAllocation reporting from WebRtcVideoChannel::AddSendStream in preparation for
using the extension RtpVideoLayersAllocationExtension instead of RTCP XR.

Bug: webrtc:12000
Change-Id: I5ea8e4f237a1c4e84a89cbfd97ac4353d4c2984f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186940
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32347}
2020-10-07 18:01:13 +00:00
Per Kjellander
d0a8f51ef7 Reland Refactor reporting of VideoBitrateAllocation
Original description
 Move reporting of target bitrate to just after the encoder has been
 updated. Originall submitted as refs/heads/master@{#32275}

Patch 1 contains the original cl
,patch 2 the fix to send rtcp even if BWE does not change.

Bug: webrtc:12000
Change-Id: I16766e08229fe1f6f65f449e0e074bed03338693
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186948
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32340}
2020-10-07 11:06:56 +00:00
Per Kjellander
b8781796d4 Revert "Refactor reporting of VideoBitrateAllocation"
This reverts commit 39a31afb77e3ce5c4ff53b8bab06364712cae7ce.

Reason for revert: Will cause RTCP Target bitrate messages to not be sent unless BWE changes. 

Original change's description:
> Refactor reporting of VideoBitrateAllocation
>
> Move reporting of target bitrate to just after the encoder has been
> updated.
>
> Bug: webrtc:12000
> Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32275}

TBR=sprang@webrtc.org,perkj@webrtc.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:12000
Change-Id: Icf21e6ae28dc17c61b9243c037ffef9b623894ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186945
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32337}
2020-10-07 08:14:21 +00:00
Mirko Bonadei
0abd518abd Revert "Introduce RTC_NO_UNIQUE_ADDRESS."
This reverts commit f5e261aaf65cdf2eb903cdf40d651846be44f447.

Reason for revert: Breaks downstream projects.

Original change's description:
> Introduce RTC_NO_UNIQUE_ADDRESS.
>
> This macro introduces the possibility to suggest the compiler that a
> data member doesn't need an address different from other non static
> data members.
>
> The usage of a macro is to maintain portability since at the moment
> the attribute [[no_unique_address]] is only supported by clang
> with at least -std=c++11 but it should be supported by all the
> compilers starting from C++20.
>
> Bug: webrtc:11495
> Change-Id: I9f12b67b4422a2749649eaa6b004a67d5fd572d8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173331
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32246}

TBR=mbonadei@webrtc.org,kwiberg@webrtc.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:11495
Change-Id: Ice318d1b11ca3dff09c190187a0b0a32ca945fe3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186944
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32335}
2020-10-07 07:37:01 +00:00
Evan Shrubsole
895556e19c Avoid converting frames to I420 in VideoStreamEncoder
This needs to be done still for kNative frames, but all other frame types
can be passed in.

I have checked all VideoEncoder implementations in Chromium and confirmed they either convert the frame to their preferred pixel format, or just
forward the frame to a delegate encoder.

Tested:
- video_loopback with NV12 generated frames for VP9, the only
codec supporting NV12, as well as VP8 which only accepts I420 frames.
- internal_tests tryrun

Bug: webrtc:11976,webrtc:11635
Change-Id: If39a815fb0c5636fceb1040c8946c3db2fb350a1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185803
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32306}
2020-10-05 08:33:46 +00:00
Per Kjellander
39a31afb77 Refactor reporting of VideoBitrateAllocation
Move reporting of target bitrate to just after the encoder has been
updated.

Bug: webrtc:12000
Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32275}
2020-10-01 14:10:09 +00:00
Mirko Bonadei
f5e261aaf6 Introduce RTC_NO_UNIQUE_ADDRESS.
This macro introduces the possibility to suggest the compiler that a
data member doesn't need an address different from other non static
data members.

The usage of a macro is to maintain portability since at the moment
the attribute [[no_unique_address]] is only supported by clang
with at least -std=c++11 but it should be supported by all the
compilers starting from C++20.

Bug: webrtc:11495
Change-Id: I9f12b67b4422a2749649eaa6b004a67d5fd572d8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173331
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32246}
2020-09-30 09:52:49 +00:00
Yun Zhang
1e4d4fdf88 Don't trigger key frame when encoder is not reset during reconfigure
Currently, key frames are scheduled even when the encoder is not reset
during reconfigeration. This means whenever new parameters like max
bitrate or min bitrate are updated through SetRtpParameters(), the
triggered encoder reconfigeration will always schedule key frames even
they are not necessary. Since parameters' changes like bitrate doesn't
require encoder instance reset.
This causes flood of key frames in our app since we do regularly max
bitrate update according to server control message.


Bug: None
Change-Id: I15d953b24c30e6026c0e97b30f44495d845f293f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185380
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32245}
2020-09-30 08:53:15 +00:00
Tomas Gunnarsson
d41c2a6b8a Remove AsyncInvoker from WebRtcVideoChannel.
RequestEncoderFallback, RequestEncoderSwitch and
SetVideoCodecSwitchingEnabledRequest are now all called on the
worker thread. Before, the work already happened on that thread but
WebRtcVideoChannel adapted internally when needed.

With this CL, there are thread checks to make sure that these calls are
always made the same way, we don't need the async invoker and there
are fewer calls out from the encoder thread in VideoStreamEncoder
(reducing the chance of unintentional blocking).

Bug: webrtc:11908
Change-Id: If8738bc2a708a0fefc6fe850b32655f049f30bdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184603
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32151}
2020-09-21 15:04:43 +00:00
Tomas Gunnarsson
612445ea60 Remove use of asyncinvoker from WebRtcVideoSendStream.
This turned out to be a bit complicated, mostly
related to the tests, but here's what's changed:

* No AsyncInvoker (and avoid ClearInternal) in
  WebRtcVideoSendStream (WVSS)
* The reason it was there is due to a "design leak" from
  VideoSourceSinkController/VideoStreamEncoder where the former uses
  locks in all methods and is unaware of a threading model. That design
  affected downstream objects, pushed the need for an async hop into
  WVSS and added a lock.
  A suggestion was made to address this in a follow-up change, here:
  https://webrtc-review.googlesource.com/c/src/+/165684
* All methods in VideoSourceSinkController are now called on a known
  and checked sequence and this CL removes the lock. This also makes
  checking state consistent (i.e. calling a getter twice in a row on the
  same sequence, will always return the same value, avoiding race with
  other threads).
* Handling of reporting state changes from the encoder queue to the
  VSSC, is done by VideoStreamEncoder.
* VideoSendStreamImpl is still instantiated on the incorrect thread [1]
  but has two initialization steps [2]. The second one already runs on
  the right thread. Addressing that TODO [1] is something we should do
  but it has side effects to consider. For the purposes of this CL
  the steps relating to the encoder (setting the sink pointer) have
  been moved to [2].

[1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94
[2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115

Bug: webrtc:11222, webrtc:11908
Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 13:29:53 +00:00
Åsa Persson
c5a74ffba4 Add support so requested resolution alignment also apply to scaled layers.
Bug: webrtc:11872
Change-Id: I7f904e2765330ee93270b66b0102ce57f336f9a0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181883
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32146}
2020-09-21 09:23:22 +00:00
Evan Shrubsole
99b0f8d26c Reland "[Adaptation] Remove QualityScalerResource when disabled."
This is a reland of ba8abbb630cdd9d05e22c830d0845e920762850d

This can be relanded as the queuing issues that were causing a
crash in the WebRTC roll in Chromium have been resolved. I have
added the Chromium failing targets to the CQ for this commit and
they have succeeded.

Original change's description:
> [Adaptation] Remove QualityScalerResource when disabled.
>
> Bug: webrtc:11843
> Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31924}

Bug: webrtc:11843
Change-Id: I228331293060ef996f1dd7f8e18d52b0818f526b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182080
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31996}
2020-08-26 06:33:43 +00:00
Evan Shrubsole
8572841131 [Adaptation] Remove resource adaptation queue
Resource adaptation needs refactoring for async adaptations. For now
the resource adaptation processor can work on the encoder thread, until
it is refactored to support async adaptation.

Bug: webrtc:11867
Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 12:16:20 +00:00
Danil Chapovalov
2549f174b5 Remove RTPFragmentationHeader creation and propagation through webrtc
Bug: webrtc:6471
Change-Id: I5cb1e10088aaecb5981888082b87ae9957bbaaef
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181541
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31955}
2020-08-17 16:37:33 +00:00
Guido Urdaneta
61b2993aaf Revert "[Adaptation] Remove QualityScalerResource when disabled."
This reverts commit ba8abbb630cdd9d05e22c830d0845e920762850d.

Reason for revert: Suspect of causing Chormium trybots to fail, preventing rolls. Will reland if the revert does not fix it.

Original change's description:
> [Adaptation] Remove QualityScalerResource when disabled.
> 
> Bug: webrtc:11843
> Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31924}

TBR=ilnik@webrtc.org,eshr@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:11843
Change-Id: Idc3950be209c6edce0dbe72d98c9b4becae0049f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181880
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Commit-Queue: Guido Urdaneta <guidou@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31952}
2020-08-17 13:46:30 +00:00
Evan Shrubsole
d73d421565 [Adaptation] Move min pixel limit logic out of adaptation processor
This is in preperation for eventual multi-stream and multi-mitigation
adaptation. This logic only applied to a single stream and thus is
better fit in the VideoStreamAdapter.

Bug: webrtc:11754
Change-Id: Icc5c7920038c82b574f4b5f7efbc92698691076f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181585
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31944}
2020-08-17 10:33:44 +00:00
Evan Shrubsole
ba8abbb630 [Adaptation] Remove QualityScalerResource when disabled.
Bug: webrtc:11843
Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31924}
2020-08-13 08:42:27 +00:00
Evan Shrubsole
552eed534c [Adaptation] Don't toggle EncoderUsageResource on/off
Instead it should be ensured that it is started with the correct config.
This removes confusion regarding a resource state. If the resource
check is stopped then the adaptations for that resource should be
removed, and there is no way to determine that if we have one method for
stop for both reconfigure and shutdown.

Bug: webrtc:11843
Change-Id: I491f2fd1f4f803a4610124c7b0026ad75ab4a9cc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181368
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31913}
2020-08-11 15:05:10 +00:00
Evan Shrubsole
a1c77f6d0d [Adaptation] Move Balanced MinFpsDiff logic to VideoStreamAdapter
This way can double adapt right away instead of relying
on the qp scaler checking soon into the future.

Bug: webrtc:11830
Change-Id: I8e878168303cf6a4c3edcf3997dd8ac2413a4479
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181060
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31895}
2020-08-10 15:56:07 +00:00
Florent Castelli
d3511010d9 Reland "Only enable conference mode simulcast allocations with flag enabled"
This is a reland of 32ca95145c4636374266f5b5d4d1ac43658bc758

Fix includes not enabling the screenshare conference behavior on non
screenshare sources even if the flag is enabled.

Original change's description:
> Only enable conference mode simulcast allocations with flag enabled
>
> Non-conference mode simulcast screenshares were mistakenly using the
> conference mode semantics in the simulcast rate allocator, which broke
> spec compliant usage in some situation.
>
> This behavior should only be used when explicitly using the SDP entry
> "a=x-google-flag:conference" in both offer and answer.
>
> Bug: webrtc:11310, chromium:1093819
> Change-Id: Ibcba75c88a8405d60467546b33977a782e04e469
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/179081
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Florent Castelli <orphis@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31828}

Bug: webrtc:11310
Bug: chromium:1093819
Change-Id: Ic933f93a5c4bad20583354fe821f8a1170e911cd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180802
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31847}
2020-08-04 10:30:08 +00:00
Florent Castelli
834dc9cfa1 Revert "Only enable conference mode simulcast allocations with flag enabled"
This reverts commit 32ca95145c4636374266f5b5d4d1ac43658bc758.

Reason for revert: Internal test failure

Original change's description:
> Only enable conference mode simulcast allocations with flag enabled
> 
> Non-conference mode simulcast screenshares were mistakenly using the
> conference mode semantics in the simulcast rate allocator, which broke
> spec compliant usage in some situation.
> 
> This behavior should only be used when explicitly using the SDP entry
> "a=x-google-flag:conference" in both offer and answer.
> 
> Bug: webrtc:11310, chromium:1093819
> Change-Id: Ibcba75c88a8405d60467546b33977a782e04e469
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/179081
> Reviewed-by: Harald Alvestrand <hta@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Florent Castelli <orphis@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31828}

TBR=ilnik@webrtc.org,hta@webrtc.org,orphis@webrtc.org

Change-Id: I5ccb6e87594f491ba09fe6b837ee24d63db878ca
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:11310
Bug: chromium:1093819
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/180801
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31829}
2020-08-03 10:31:21 +00:00
Florent Castelli
32ca95145c Only enable conference mode simulcast allocations with flag enabled
Non-conference mode simulcast screenshares were mistakenly using the
conference mode semantics in the simulcast rate allocator, which broke
spec compliant usage in some situation.

This behavior should only be used when explicitly using the SDP entry
"a=x-google-flag:conference" in both offer and answer.

Bug: webrtc:11310, chromium:1093819
Change-Id: Ibcba75c88a8405d60467546b33977a782e04e469
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/179081
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31828}
2020-08-03 10:09:46 +00:00
Danil Chapovalov
6d008a8ab7 Ignore RTPFragmentationHeader when rewriting H264 SPS
RTPFragmentationHeader is already ignored by H264 packetizer
and thus doesn't need to be provided and calculated.

Bug: webrtc:6471
Change-Id: I45bc22827f0dc811457e3ebe477a16293501c2fb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/179843
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Marco Paniconi <marpan@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31791}
2020-07-27 09:42:25 +00:00
Evan Shrubsole
73ecede5ad [Adaptation] Move AdaptationConstraints to VideoStreamAdapter
Bug: webrtc:11754
Change-Id: Ic51059834ea61c08e977a3a6b5bba48abc86f873
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178906
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31688}
2020-07-09 13:06:56 +00:00
Evan Shrubsole
517f81e0a4 [Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe
This is one less dependency on the task queue, and will make
things like removing resources and cleanup much easier in the future.

Bug: webrtc:11754
Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 12:37:26 +00:00
Evan Shrubsole
dc4d422738 [Adaptation] Move AdaptationListeners to VideoStreamAdapter
This is a step needed for multi-stream and new mitigations. It also
cleans up needing to signal adaptation changes in mutiple places
from ResourceAdaptationProcessor.

R=hbos@webrtc.org

Bug: webrtc:11754
Change-Id: Ib185dc9f66fbb4a087eb9e970c68c3f47eafb17f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178874
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31684}
2020-07-09 11:10:36 +00:00
Markus Handell
a376518817 Migrate video/ except video/end_to_end_tests and video/adaptation to webrtc::Mutex.
Also migrates test/ partly.

Bug: webrtc:11567
Change-Id: If5b2eae65c5f297f364b6e3c67f94946a09b4a96
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178862
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31672}
2020-07-08 12:21:08 +00:00
Markus Handell
a827a30bb7 Revert "Migrate video/ except video/end_to_end_tests and video/adaptation to webrtc::Mutex."
This reverts commit 0eba415fb40cc4e3958546a8ee53c698940df0a1.

Reason for revert: previously unknown lock recursion occurring downstream.

Original change's description:
> Migrate video/ except video/end_to_end_tests and video/adaptation to webrtc::Mutex.
> 
> Also migrates test/ partly.
> 
> Bug: webrtc:11567
> Change-Id: I4203919615c087e5faca3b2fa1d54cba9f171e07
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178813
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Markus Handell <handellm@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31653}

TBR=sprang@webrtc.org,handellm@webrtc.org

Change-Id: I13f337e0de5b8f0eb19deb57cb5623444460ec4d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:11567
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178842
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31656}
2020-07-07 20:46:48 +00:00
Markus Handell
0eba415fb4 Migrate video/ except video/end_to_end_tests and video/adaptation to webrtc::Mutex.
Also migrates test/ partly.

Bug: webrtc:11567
Change-Id: I4203919615c087e5faca3b2fa1d54cba9f171e07
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178813
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31653}
2020-07-07 18:01:44 +00:00
Evan Shrubsole
3444a49d77 [Adaptation] Remove deg.prefs from ResourceAdaptationProcessor
Changes in effective degradation preference is now done directly
on the VideoStreamAdapter. This means that the
DegradationPreferenceListener class can be removed.

Bug: webrtc:11700
Change-Id: I9ce4b627de0279df5ef2e5a38435a6d352fbb135
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178609
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31642}
2020-07-07 08:37:05 +00:00
Evan Shrubsole
9492d500dd [Adaptation] Move deg.pref. out of ResourceAdaptationProcessor
This patch creates a new class which provides the DegradationPreference
thread safe to all classes that need if (BalancedConstraint and
QpScalerResource). It also broadcasts to all listeners when the
preferences are updated, so the ResourceAdaptationProcessor can update
the video if needed.

In future work, this could be used to remove the need for two task
queues for the VideoStreamEncoder resources.

Bug: webrtc:11700
Change-Id: I05480db8b7157b5643f6f86ec9c64850839b3e76
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177522
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31623}
2020-07-03 13:10:27 +00:00