202 Commits

Author SHA1 Message Date
Evan Shrubsole
6c13fd9071 Move bandwidth overuse detection out of VideoStreamEncoder
Bug: webrtc:11222
Change-Id: I12ccd008c848a0146fb22292f8dac46d1f7be9b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166531
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30329}
2020-01-21 10:24:31 +00:00
Ilya Nikolaevskiy
8b1338bf8e Propagate is_bw_limited flag with bw allocation everywhere it's copied
Bug: webrtc:11015
Change-Id: Ie971d29d8a8d140ba120a51dd3920291034a4d48
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30325}
2020-01-20 16:12:03 +00:00
Henrik Boström
fae6f0e87b [Overuse] MaybeUpdateTargetFrameRate() & ResetVideoSourceRestrictions()
This CL does two things for the sake of getting us closer to adaptation
modules being injectable and usable without knowing implementation
details.

Firstly, RefreshTargetFramerate() is removed. The target frame rate is
dependent on two things: 1) the codec max frame rate, and 2) the video
source restrictions. If either of these two changes, the target frame
rate is updated - there is no need to trigger this externally; the
module already knows if either of these factors change.
The private method MaybeUpdateTargetFrameRate() is added to ensure
overuse_detector->OnTargetFramerateUpdated() happens when necessary.

In doing this, the frame rates are updated to use
absl::optional<double>. This documents its optionality and avoids
magical values (previously -1 was not a bug but meaning "missing"). It
also matches VideoSourceRestrictions::max_frame_rate()'s type.

Secondly, ResetAdaptationCounters() is renamed
ResetVideoSourceRestrictions(). This more accurately describes what it
is doing; it is resetting the restrictions (the adaptation counters
getting reset is merely an implementation specific side-effect of
this). This method is added to the generic interface.

The usefulness of being able to ResetVideoSourceRestrictions() is
questioned in a TODO - current usage of this is when "quality rampup"
finishes. Nevertheless, any module could implement this functionality
so it belongs to the interface for now.

Bug: webrtc:11222
Change-Id: I079785df55fc9894e85087ec98be3e4ebd0713c5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30320}
2020-01-20 12:58:38 +00:00
Henrik Boström
a3d4252a01 ResourceAdaptationModule HasInputVideo+DegradationPreference setters.
The "has_input_video_ logic" is simplified to abort AdaptUp() and
AdaptDown() directly rather than in each calling place of the
VideoSourceRestrictor. The intent is no change in behavior.
The degradation_preference_ is removed from the VideoSourceRestrictor
as its only usage was DCHECKing (not worth it).

ResourceAdaptationModuleInterface gets SetHasInputVideo() and
SetDegradationPreference(), making these things controllable without
knowing implementation details.

StartCheckForOveruse() and StopCheckForOveruse() are renamed to
StartResourceAdaptation() and StopResourceAdaptation().

Bug: webrtc:11222
Change-Id: Id2d7f34d427dfb3ecd4831b1a245d07becae6520
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166173
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30296}
2020-01-17 10:43:02 +00:00
Danil Chapovalov
64f1f3f04e Replace RTC_FALLTHROUGH with ABSL_FALLTHROUGH_INTENTED
Bug: None
Change-Id: I7287403f3fb13b8e30f92ca3cf1882b03bb53a6e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166176
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30283}
2020-01-16 15:20:35 +00:00
Henrik Boström
07b17df771 Move DegradationPreference logic to the encoder queue.
This moves SetHasInputVideoAndDegradationPreference() to the encoder
queue. OveruseFrameDetectorResourceAdaptationModule is now entirely
single-threaded, including its inner class VideoSourceRestrictor.

VideoStreamEncoder now protects the module with RTC_GUARDED_BY. This
ensures it is safely used, even without a SequenceChecker inside of the
module. The module's |encoder_queue_| is removed.

The one task queue reference that is needed - passing down the current
task queue to StartCheckForOveruse() - is replaced by a TaskQueueBase*
(instead of rtc::TaskQueue*), enabling obtaining the current queue with
TaskQueueBase::Current(). (There is no rtc::TaskQueue::Current().)

Furthermore, the only uses of VideoSourceSinkController that isn't on
the encoder queue are documented, with a TODO saying if these are moved
the VideoSourceSinkController could also be made single-threaded.
However since this requires introducing a delay to
VideoStreamEncoder::SetSource() and VideoStreamEncoder::Stop(),
arguably a more risky change, if this is to be attempted that should be
in a separate CL.

Bug: webrtc:11222
Change-Id: I448ca5125708d5f66b95b0b180d6d24cc356dfa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165783
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30263}
2020-01-15 11:58:04 +00:00
Jonas Olsson
b2b2031457 Concatenate string literals at compile time.
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format

After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.

This primary benefit of this change is a small reduction in binary size.

Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
2020-01-14 14:47:48 +00:00
Henrik Boström
8234b92ba3 Move DegradationPreference logic out of VideoSourceSinkController.
The DegradationPreference logic is moved into
OveruseFrameDetectorResourceAdaptationModule. This makes the adaptation
module solely responsible for degradation preference, and the
VideoStreamEncoder the only bridge between the adaptation module and the
VideoSourceSinkController.

The adaptation module is now unaware of the existence of a controller.
It only "speaks" VideoSourceRestrictions, which is a big milestone in
making adaptation modules injectable.

A follow-up CL will explore the possibility of reconfiguring the
controller's source and which degradation preference to use to the
encoder queue. This would allow us to make several classes
single-threaded, but it is a change in behavior and should be done in a
separate CL.

Bug: webrtc:11222
Change-Id: Ib7f640e12789da5f801177926c2072a51818f261
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165684
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30237}
2020-01-13 17:24:48 +00:00
Henrik Boström
ce0ea49001 VideoStreamEncoder configuring source/sink with VideoSourceController.
This is part of the work for making VideoStreamEncoder responsible for
configuring its source/sink and limiting the responsibility of
OveruseFrameDetectorResourceAdaptationModule to only output relevant
VideoSourceRestrictions.

BEFORE THIS CL

Prior to this CL, OveruseFrameDetector was responsible for performing
AddOrUpdateSink() on the source, which it did using its nested class
VideoSourceProxy.

AddOrUpdateSink() could happen for both adaptation and non-adaptation
related reasons. For example:
- Adaptation related: AdaptUp() or AdaptDown() happens, causing updated
  VideoSourceRestrictions.
- Non-adaptation related: VideoStreamEncoder asks the module to
  reconfigure the source/sink for it, such as with
  SetMaxFramerateAndAlignment() or SetWantsRotationApplied().

AFTER THIS CL

AddOrUpdateSink() is performed by VideoSourceController, which is owned
by VideoStreamEncoder. Any reconfiguration has to go through the
VideoStreamEncoder. This means that:
- Non-adaptation related settings happen between VideoStreamEncoder and
  VideoSourceController directly (without going through the adaptation
  module).
- Adaptation related changes can be expressed in terms of
  VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule
  only has to output the restrictions and not know or care about other
  source/sink settings.

For now, VideoSourceController has to know about DegradationPreference.
In a future CL, the DegradationPreference logic should move back to
the adaptation module. The VideoSourceRestrictions are fully capable of
expressing all possible source/sink values without the "modifier" that
is the degradation preference.

Bug: webrtc:11222
Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:14:04 +00:00
Henrik Boström
d238200882 Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions.
The VideoSourceRestrictions describe the maximum pixels per frame and
max frame rate of a video source.

This CL makes the ResourceAdaptationModuleInterface responsible for the
reconfiguration of video sources. The VideoSourceRestrictions is the
output of an adaptation module, and the ResourceAdaptationModuleListener
handles the callback for when the source restrictions change.

The OveruseFrameDetectorResourceAdaptationModule is updated to output
its changes using these interfaces, and VideoStreamEncoder - now a
listener - is made responsible for triggering the reconfiguring the
video source.

Performing the reconfiguration still requires interacting with the
VideoSourceProxy - it is still partially responsible for keeping track
of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For
now this may look a bit weird: the VideoSourceProxy tells the
VideoStreamEncoder about the new restrictions, and then the
VideoStreamEncoder tells the VideoSourceProxy to apply these
restrictions on the source/sink. This exercises the listener though, and
unblocks the next CL.

The next CL should move all "configuring the source" logic to the
VideoStreamEncoder instead, so that the only information that is tracked
by OveruseFrameDetectorResourceAdaptationModule is what it actually
outputs to the listener. See the next CL
(https://webrtc-review.googlesource.com/c/src/+/162802) where a
VideoSourceController is introduced, to be owned by the
VideoStreamEncoder rather than the adaptation module.

Bug: webrtc:11222
Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-13 10:57:00 +00:00
Henrik Boström
382cc6d8a6 Add incomplete ResourceAdaptationModuleInterface.
This interface will be improved upon iteratively to aid reviewability.
The initial version only handles starting and stopping the module; input
and output of the module is still implementation-specific.

TBR=sprang@webrtc.org

Bug: webrtc:11222
Change-Id: Ie307cfe3d3211c84346c035f2c0e9a632f58221b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30167}
2020-01-07 13:24:42 +00:00
Henrik Boström
b08882b625 Refactor out VideoStreamEncoder's overuse logic to separate module.
This CL puts the VideoStreamEncoder's current adaptation logic inside
the new class OveruseFrameDetectorResourceAdaptationModule. The
intention is not to change any behavior, only to move code.

Future CLs should step by step decrease the coupling between
OveruseFrameDetectorResourceAdaptationModule, VideoStreamEncoder and
the VideoStreamEncoder's QualityScaler by introducing more abstract
interfaces. This is not done in this CL because it is large enough as
it is, but the long term goal is to make it possible to replace the
existing overuse module with a different implementation.

This CL relies on existing tests exercising the VideoStreamEncoder, but
part of making overuse logic modular should include testing each module
separately as well as continued integration testing of the
VideoStreamEncoder.

Bug: webrtc:11222
Change-Id: I316a174adfd00d60cdd224a23a5f616efd235d13
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161953
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30163}
2020-01-07 09:55:54 +00:00
Rasmus Brandt
5cad55b240 Signal requested resolution alignment requirements from sinks to sources.
Bug: webrtc:11218
Change-Id: I593b0515ea389bece472234a3c4082ccc5321ea5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162400
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30113}
2019-12-19 10:39:04 +00:00
Florent Castelli
907dc806c7 Reland "Add support for RtpEncodingParameters::max_framerate"
Perf test failure was fixed separately.

TBR=steveanton@webrtc.org,sprang@webrtc.org,asapersson@webrtc.org

Original change's description:
> This adds the framework support for the max_framerate parameter.
> It doesn't implement it in any encoder yet.
>
> Bug: webrtc:11117
> Change-Id: I329624cc0205c828498d3623a2e13dd3f97e1629
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160184
> Reviewed-by: Steve Anton <steveanton@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Commit-Queue: Florent Castelli <orphis@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#29907}

Bug: webrtc:11117
Change-Id: I9c1daf7887c2024c6669dc79bff89d737417458c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161445
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30030}
2019-12-06 15:11:54 +00:00
Ilya Nikolaevskiy
648b9d77c7 Implement automatic animation detection in VideoStreamEncoder
If WebRTC-AutomaticAnimationDetectionScreenshare experiment is enabled,
content type is screenshare and degradation preference is BALANCED,
then input resolution is restricted if update_rect of the incoming frames
is the same for considerable amount of time and is big enough.

This entails treating BALANCED degradation preference for screenshare as
MAINTAIN_RESOLUTION in adaptation logic.

Bug: webrtc:11058
Change-Id: I903dddf53fcbd7c8eac6c5b1447225b15fd8fe5f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161097
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30002}
2019-12-04 11:24:31 +00:00
Erik Språng
7968530418 Removes caching SimulcastEncoderAdapter::GetEncoderInfo()
There are edge cases where the caching of encoder info will cause
issues. For instance if a sub-encoder fails en Encode call and falls
back to some other implementation, or if the fps targets shift due to
SetRates() triggering new layers to be enabled.

This CL forces a complete rebuild on every call to GetEncoderInfo().

It also adds new logging of when the info changes, as debugging issues
can be very time consuming if we can't tell that happened.

Bug: webrtc:11000
Change-Id: I7ec7962a589ccba0e188e60a11f851c9de874fab
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160960
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29938}
2019-11-27 17:06:54 +00:00
Florent Castelli
5cef9c3581 Revert "Add support for RtpEncodingParameters::max_framerate"
This reverts commit 15be5282e91ba38894e6ad51fe9a35a38a6b7f29.

Reason for revert: crbug.com/1028937

Original change's description:
> Add support for RtpEncodingParameters::max_framerate
> 
> This adds the framework support for the max_framerate parameter.
> It doesn't implement it in any encoder yet.
> 
> Bug: webrtc:11117
> Change-Id: I329624cc0205c828498d3623a2e13dd3f97e1629
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160184
> Reviewed-by: Steve Anton <steveanton@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Commit-Queue: Florent Castelli <orphis@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#29907}

TBR=steveanton@webrtc.org,asapersson@webrtc.org,sprang@webrtc.org,orphis@webrtc.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:11117
Change-Id: Ic44dd36bea66561f0c46e73db89d451cb3e22773
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160941
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29935}
2019-11-27 14:01:53 +00:00
Florent Castelli
15be5282e9 Add support for RtpEncodingParameters::max_framerate
This adds the framework support for the max_framerate parameter.
It doesn't implement it in any encoder yet.

Bug: webrtc:11117
Change-Id: I329624cc0205c828498d3623a2e13dd3f97e1629
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160184
Reviewed-by: Steve Anton <steveanton@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29907}
2019-11-25 16:43:59 +00:00
Sergey Silkin
7f44505461 Allow min start bitrate to be lower than min bitrate.
Bug: none
Change-Id: Ic5825a642bfe28a7cd7fe4c6d9beaba3552167f2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160406
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29905}
2019-11-25 15:12:05 +00:00
philipel
c7a46c49a0 Fix VideoStreamEncoder to not reference encoded data from the RunPostEncode task.
Bug: webrtc:9378
Change-Id: I1ada7018507d0c78fee51523f8cd4fab76c35432
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160306
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29903}
2019-11-25 14:41:55 +00:00
Åsa Persson
e644a03195 Add field trial for rampup in quality based on available bandwidth.
Bug: none
Change-Id: I32e1ea6fb2f2e20fc631e09b02c8f3a11b6c9fac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/158888
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29751}
2019-11-11 10:13:28 +00:00
Jakob Ivarsson
159b417c98 Keep the video send stream alive if the encoder drop frames.
The encoder can drop all frames for extended periods if it has produced over budget. After 2 seconds without any encoded frames, the video send stream times out and deallocates the stream.

Ideally the send stream should keep track if frames are captured instead of encoded, but keeping the stream alive using OnDroppedFrame can work as a proxy for that.

Bug: webrtc:11062
Change-Id: Id7ec1ff333427643453c4a36d1db03ca826cd9ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/158700
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29662}
2019-10-31 11:30:47 +00:00
Ilya Nikolaevskiy
9560d7dc58 Make update_rect optional in VideoFrame
For the automatic content type detection we need to know if the update
rect is trusted or just not available.

Currently we only care if it's not empty, so in case of no update rect
available, full frame resolution was set as a changed region.

This CL makes the update_rect field optional but should be a no-op in the
current code, as absence of update_rect is treated as a full update via
a new getter method |update_rect_or_full_frame()|.

Bug: webrtc:11058
Change-Id: I913545b71ac2fc845861549ac34eb1b630012109
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/158673
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29654}
2019-10-30 11:27:54 +00:00
philipel
e5d0fe0dff Updated VideoStreamEncoder to destroy encoder_queue_ before encoder_switch_experiment_.
Bug: none
Change-Id: I0d72fd0b851bd3f9b5021bc9b51af5da882483dd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/157044
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29484}
2019-10-15 10:04:38 +00:00
Sergey Silkin
41c650bea2 Use bitrate limits provided by encoder.
- Use minimum start bitrate to drop frame and adapt resolution in the
beginning of call.

- Use minimum bitrate to decide whether or not resolution should be
increased based on quality in MAINTAIN_FRAMERATE and BALANCED modes.
In BALANCED mode bitrate limits provided by the corresponding field
trial are prioritized over the limits provided by encoder.

Bug: webrtc:10853
Change-Id: I8257eb64565bcafa6ae9887a1af18e90f8400cac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156302
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29461}
2019-10-14 12:57:24 +00:00
Evan Shrubsole
7c079f650d Reland "Fix minor regression caused by a8336d3"
This is a reland of 809198edfff416fce8d75b574a43afab5e67b1cd

A fix was made in https://webrtc-review.googlesource.com/c/src/+/154343
which fixed the regression issues caused by the original patch.

Original change's description:
> Fix minor regression caused by a8336d3
>
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
>
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}

Bug: webrtc:10126
Change-Id: Iecc3ab6a5cd1193a1fa8e824dcf4f0b8165f9bf8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154359
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29356}
2019-10-01 11:49:38 +00:00
Åsa Persson
45b176fc22 Downgrade fps in same step as resolution in initial drop due to size.
Bug: none
Change-Id: If0943ee291a029fa81035c72607873995ba8ab8b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154742
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29342}
2019-09-30 12:28:26 +00:00
Ilya Nikolaevskiy
fbf75a7891 Video: Log scalability configuration on encoder reconfigure
Lately there were 2 separate bugs, where seeing this information in the
log could help immediately figuring out the problem.

Bug: none
Change-Id: I3f2b2d5864106cdb231715e1702edee3b9b05caa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154566
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29338}
2019-09-30 09:08:20 +00:00
Ilya Nikolaevskiy
abd9e0f5c2 VideoStreamEncoder: Don't crop and scales frames for external encoders
Bug: none
Change-Id: I1edc0314450793c0ae3fb471aa8936a102773a96
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154424
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29319}
2019-09-26 11:49:09 +00:00
Evan Shrubsole
e32ae4f8fb Invalidate encoder rates on VideoStreamEncoder::ReconfigureEncoder
Without invalidation the codec configuration may not get updated. This
is prone to happen when the bandwidth is below the min or above the max
in which case the last_encoder_rate_settings_ may have not changed.

This led to a regression in b6a45dd, where last_encoder_rate_settings_
would not change the allocation or frame rate, but the frame rate would
have been set by the video adapter. Thus the frame rates were set
incorrectly, leading to lower values in the regression tests.

I have re-run this scenario against some of the metric drops and the
regression appears to be fixed.


Bug: webrtc:10126
Change-Id: I0fa6c9b71e7aff5dd80e53119db109d97eed98b6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154343
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29301}
2019-09-25 11:37:52 +00:00
Evan Shrubsole
b6a45dda4c Revert "Fix minor regression caused by a8336d3"
This reverts commit 809198edfff416fce8d75b574a43afab5e67b1cd.

Reason for revert: Performance regressions that need to be addressed.

Original change's description:
> Fix minor regression caused by a8336d3
> 
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
> 
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}

TBR=sprang@webrtc.org,eshr@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:10126
Change-Id: I133cbe5d8cb894ed944ae8a2d0f63a78bbed72ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153484
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29221}
2019-09-18 14:38:15 +00:00
Sebastian Jansson
86314cfb5d Cleaning up C++14 move into lambda TODOs.
Bug: webrtc:10945
Change-Id: I4d2f358b0e33b37e4b4f7bfcf3f6cd55e8d46bf9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153240
Commit-Queue: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29212}
2019-09-17 19:18:26 +00:00
Mirko Bonadei
317a1f09ed Use std::make_unique instead of absl::make_unique.
WebRTC is now using C++14 so there is no need to use the Abseil version
of std::make_unique.

This CL has been created with the following steps:

git grep -l absl::make_unique | sort | uniq > /tmp/make_unique.txt
git grep -l absl::WrapUnique | sort | uniq > /tmp/wrap_unique.txt
git grep -l "#include <memory>" | sort | uniq > /tmp/memory.txt

diff --new-line-format="" --unchanged-line-format="" \
  /tmp/make_unique.txt /tmp/wrap_unique.txt | sort | \
  uniq > /tmp/only_make_unique.txt
diff --new-line-format="" --unchanged-line-format="" \
  /tmp/only_make_unique.txt /tmp/memory.txt | \
  xargs grep -l "absl/memory" > /tmp/add-memory.txt

git grep -l "\babsl::make_unique\b" | \
  xargs sed -i "s/\babsl::make_unique\b/std::make_unique/g"

git checkout PRESUBMIT.py abseil-in-webrtc.md

cat /tmp/add-memory.txt | \
  xargs sed -i \
  's/#include "absl\/memory\/memory.h"/#include <memory>/g'
git cl format
# Manual fix order of the new inserted #include <memory>

cat /tmp/only_make_unique | xargs grep -l "#include <memory>" | \
  xargs sed -i '/#include "absl\/memory\/memory.h"/d'

git ls-files | grep BUILD.gn | \
  xargs sed -i '/\/\/third_party\/abseil-cpp\/absl\/memory/d'

python tools_webrtc/gn_check_autofix.py \
  -m tryserver.webrtc -b linux_rel

# Repead the gn_check_autofix step for other platforms

git ls-files | grep BUILD.gn | \
  xargs sed -i 's/absl\/memory:memory/absl\/memory/g'
git cl format

Bug: webrtc:10945
Change-Id: I3fe28ea80f4dd3ba3cf28effd151d5e1f19aff89
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153221
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29209}
2019-09-17 15:47:29 +00:00
Evan Shrubsole
809198edff Fix minor regression caused by a8336d3
VideoEncoder::SetRates was being called unnessesarily when the fields
appended to RateControlParameters were changed. Since SetRates only
cares about RateControlParameters, it should have only been called if
the RateControlParameters themselves were actually changed.

Bug: webrtc:10126
Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#29208}
2019-09-17 13:34:18 +00:00
philipel
d9cc8c08dc Encoder switching based on network and/or resolution conditions.
In this CL:
 - Renamed EncoderFailureCallback to EncoderSwitchRequestCallback. An encoder
   switch request can now also be made with a configuration that specifies which
   codec/implementation to switch to.
 - Added "WebRTC-NetworkCondition-EncoderSwitch" field trial that specifies
   switching conditions and desired codec to switch to.
 - Added checks to trigger the switch based on these conditions.

Bug: webrtc:10795
Change-Id: I9d3a9a39a7c4827915a40bdceed10b581d70b90a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151900
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29196}
2019-09-16 13:43:29 +00:00
Niels Möller
fe407b7a1d Move code related to VideoCodingModule to its own build target
The new target, modules/video_coding:video_coding_legacy, is not
depended upon by any webrtc non-test code.

Bug: webrtc:7408
Change-Id: I94127e2b8b3b8f15917bfa38e602f8face91fcdb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152163
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29133}
2019-09-10 12:34:38 +00:00
Evan Shrubsole
cc62b16658 Add qualityLimitationResolutionChanges stat
Implements the stat qualityLimitationResolutionChanges [1].

[1] https://w3c.github.io/webrtc-stats/#dom-rtcoutboundrtpstreamstats-qualitylimitationresolutionchanges

Bug: webrtc:10935
Change-Id: I391f2be5958a96b442e32c40ab7043752f3f71dd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150882
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#29113}
2019-09-09 15:22:57 +00:00
Florent Castelli
a8336d3cf4 Connect the stable target rate to the video encoders
The stable target rate is used to make smarter choices in the rate
to chose which layers to enable in SVC or simulcast modes.
the addition of hysteresis, we can improve a call quality by reducing
the amount of resolution switch.


Bug: webrtc:10126
Change-Id: I04d0df9e6bbe247e2f2a668207ff74d475e2464c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150642
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29112}
2019-09-09 15:06:51 +00:00
Erik Språng
cf9cbf5edb Add support for stable bitrate target in SvcRateAllocator
Bug: webrtc:10126
Change-Id: I1362d183bb91510db4e2763a779bcdf681d855ef
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/149069
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29066}
2019-09-04 14:22:43 +00:00
Erik Språng
5056af0678 Make sure link allocation is at least as large as bitrate sum.
The VideoBitrateAllocator subclasses may actually allocate more than the
target, in order to satisfy the min bitrate constraint. In this case,
make sure the bandwidth allocation we signal to the encoder is at least
this large.

Bug: chromium:995462
Change-Id: I08b89a7c54392330d773e13c1b0a3eff42f81672
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151125
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29040}
2019-09-02 15:46:10 +00:00
Åsa Persson
30ab015fc9 BalancedDegradationSettings: add min bitrate configuration for resolution.
Add separate setting for configuring min bitrate that only applies when
adapting up in resolution.

Bug: none
Change-Id: I83d33ac3110a22602065b8d83130e3f619cb1eba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150329
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Mirta Dvornicic <mirtad@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28970}
2019-08-27 11:05:10 +00:00
Åsa Persson
4869bd6309 Add method CanAdaptUp based on bitrate to BalancedDegradationSettings.
Bug: none
Change-Id: Ibeded1f7193384a8ae5bd3f2ce4ccaa4c7db7290
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150333
Reviewed-by: Mirta Dvornicic <mirtad@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28957}
2019-08-26 12:48:35 +00:00
Åsa Persson
f5e5d250bc BalancedDegradationSettings: add option to configure a min framerate diff.
If a framerate reduction (input fps - restricted fps) is less than the
configured diff, shorten interval to next qp check.

Bug: none
Change-Id: Ia0b9e0638e5ba75cdc20a1bb45bfcb7d858c5f89
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/149040
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28880}
2019-08-16 16:13:46 +00:00
Åsa Persson
1b247f1e6e BalancedDegradationSettings: add option to configure min bitrate.
Add possibility to configure min bitrate based on resolution.
Only adapt up if bw estimate is above the min bitrate for next higher resolution.

Bug: none
Change-Id: Ie38faae07d23336675ec33697ace6f6fed322efa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148598
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28863}
2019-08-15 07:16:33 +00:00
Sergey Silkin
6b2cec17c2 Use recommended min bitrate limit provided by encoder.
Also:
- DCHECK that bitrate limits recommended by encoder are reasonable.
- Restrict target bitrate such that it never exceed the max bitrate.

Bug: webrtc:10853
Change-Id: Ie43d30a7acfc8fa115deffd94165844248ce7945
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148442
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28822}
2019-08-09 14:54:09 +00:00
Åsa Persson
139f4dc7ac QualityScaler: Add option to try fast adapt down at start up based on initial bw estimates.
optional<int> initial_bitrate_interval_ms: time interval since start of call
where fast adapt down is allowed.
optional<double> initial_bitrate_factor: try fast adapt down if bw estimate is
below initial bitrate * factor.

Bug: none
Change-Id: I63e1fdaac6556d8e9a961a42e11c925f9ecb9771
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147725
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28753}
2019-08-05 09:43:19 +00:00
Florent Castelli
8bbdb5b9bd Update VideoBitrateAllocator allocate to take a struct with more fields
We want to evaluate more data in order to make better choices in the
bitrate allocators.
In order to freely update the parameter list without
breaking the API many times for projects customizing them, we'll use a
struct instead.

Bug: webrtc:10126
Change-Id: I443f86781c5134950294cdd1e3197a47447cf973
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141418
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28748}
2019-08-02 13:52:54 +00:00
Sergey Silkin
6456e352ac Use max bitrate limit recommended by encoder.
If VideoEncoderConfig::max_bitrate_bps is unset then max bitrate of
video stream is set equal to max bitrate value recommended by encoder
for given resolution via encoder capabilities (if available).

Bug: webrtc:10796
Change-Id: I7fce9afc476b794a16956e694e891faee110048e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28515}
2019-07-09 13:56:56 +00:00
Åsa Persson
48284b86d4 BalancedDegradationSettings: Add option to configure fps based on codec type.
Bug: none
Change-Id: I43b3d976b9400a0552fee80a6a65c215c71049ac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144543
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28503}
2019-07-08 11:46:25 +00:00
philipel
e8ed83003d WebRtcVideoChannel encoder fallback.
In this CL:
 - Added WEBRTC_VIDEO_CODEC_ENCODER_FAILURE return code that can
   be returned by the encoder wrapper in case of a broken encoder.
 - Added EncoderFailureCallback interface that can be called
   to request encoder fallback to be performed. Implemented by
   WebRtcVideoChannel and called from the VideoStreamEncoder.
 - Updated SelectSendVideoCodec to select all compatible codecs instead
   of just one.

Bug: webrtc:10795
Change-Id: I87a83fd02e48c40493c930471c06c3d0941031ab
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140888
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28462}
2019-07-03 12:31:42 +00:00