1180 Commits

Author SHA1 Message Date
Evan Shrubsole
f2be3eff26 Move initial frame drop to overuse module
It would be nice for this to stay in video stream encoder,
but this feature is mostly related to quality scaling. Perhaps
something easier to understand is possible in the future.

Bug: webrtc:11222
Change-Id: I71705f33ff94bbcf2fb9b5c94226c8e76dcba94c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168051
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30446}
2020-02-03 11:56:31 +00:00
Evan Shrubsole
c809e8bd62 Move quality scaling frame drop logic to adaptation module
Bug: webrtc:11222
Change-Id: I43db57fa128924ccaa3e44cd58098e7938e5ff5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168050
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30445}
2020-02-03 10:55:11 +00:00
Evan Shrubsole
bfe3ef8feb Report frame qp to quality scaler via overuse module
Bug: webrtc:11222
Change-Id: I63938adf5f623429eab1bcd668cde8fa5a1a083a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167924
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30435}
2020-01-31 09:18:28 +00:00
Evan Shrubsole
cf0595234c Move quality scaler into adaptation module
This allows for further refactoring, eventually moving
all of quality scaler out of video stream encoder.

Bug: webrtc:11222
Change-Id: Id121608da56f57549a616ccc5f141bb598668b40
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167728
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30417}
2020-01-30 09:05:19 +00:00
Evan Shrubsole
73a5e916a9 Remove task_queue dependency for QualityScaler
This allows for the possiblity to move the QualityScaler
out of the VideoStreamEncoder in the future.


Bug: webrtc:11222
Change-Id: I1d563cf08791e27ff5065ce90bcb150a7974d868
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167534
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30406}
2020-01-29 12:14:10 +00:00
Danil Chapovalov
97ffbefdab Pass and store PacketBuffer::Packet by unique_ptr
to avoid expensive move of the Packet and prepare PacketBuffer
to return list of packets as a frame.

Bug: None
Change-Id: I19f0452c52238228bbe28284ebb197491eb2bf4e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167063
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30404}
2020-01-29 11:48:55 +00:00
Henrik Boström
ad515a255b [Overuse] Move GetCpuOveruseOptions() to adaption module.
This removes the last remaining explicit reference from
OveruseFrameDetectorResourceAdaptationModule to
VideoStreamEncoder.

VideoStreamEncoder's call to SetEncoderSettings() inside
ReconfigureEncoder() is moved a few lines down - it was discovered that
during these lines the EncoderInfo config could get modified in
response to InitEncode() - so this fixes a potential bug.

Bug: webrtc:11222
Change-Id: I9746f28a4df8e631e297669c10636bf17b39acec
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167363
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30381}
2020-01-27 13:51:40 +00:00
Danil Chapovalov
159c414ff8 Detach LossNotificationController from RtpGenericFrameDescriptor
To allow to use the LossNotificationController with
an updated version of the frame descriptor extension

Bug: webrtc:10342
Change-Id: I5ac44dc5549dfcfc73bf81ad1e8eab8bd5dd136e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166166
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30369}
2020-01-24 11:53:28 +00:00
Artem Titov
cdd73e095c Migrate PC level tests on new video codec configuration API
Bug: webrtc:10138
Change-Id: I7129857724aafbfae71f36554ef79be78c442cb6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167066
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30362}
2020-01-23 13:45:33 +00:00
Evan Shrubsole
02d51f9fdc Remove unused field trial WebRTC-InitialFramedrop
Bug: webrtc:9176, webrtc:6086
Change-Id: Ie02800963f790f07b4c60ff01a04ecd6b5e1113d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167181
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30361}
2020-01-23 13:42:03 +00:00
Henrik Boström
d4578ae962 [Overuse] Encoding pipeline as input signals in the abstract interface.
This defines the following methods:
- OnFrame(), replaces SetLastFramePixelCount().
- OnFrameDroppedDueToSize(), a rename of FrameDroppedDueToSize() to
  match the other methods.
- OnEncodeStarted(), a rename of the incorrectly named FrameCaptured().
- OnEncodeCompleted(), a rename of the poorly named FrameSent().

In order to get rid of SetLastFramePixelCount(), the "we don't know the
frame size" use case - which was previously implicitly avoided by
invoking SetLastFramePixelCount() with a made-up value for
last_frame_info_ - is now avoided using ".value_or()" in
LastInputFrameSizeOrDefault(). This does mean that a constant 144p
resolution value is referenced in two places, but the fact that this is
a magic value is at least made explicit. This may help future
improvements.

Bug: webrtc:11222
Change-Id: I3b28daa8c5ecf57c6537957d4759f15e24bb2234
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166961
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30352}
2020-01-22 17:11:20 +00:00
Evan Shrubsole
2bc91e8c6a Avoid extra EncodedFrame copy in RunPostEncode
All uses of encoded_image are const, except for the copy for running on
the encoder_queue_.

Bug: None
Change-Id: I7fc8cb46f6afb42a2d27961d3d3ff8d9e63fe1b5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166442
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30351}
2020-01-22 15:41:38 +00:00
Henrik Boström
ede69c0fbe [Overuse] Setting the target bitrate through the interface.
The poorly named SetEncoderStartBitrate() is renamed
SetEncoderTargetBitrate() and added to the abstract resource adaptation
module interface.

The so-called "start bitrate" was updated to match the target bitrate,
so this was only ever a "start bitrate" until we had any estimates. The
variable is renamed in VideoStreamEncoder as well, and usage of optional
types are introduced to avoid magical values in a few places in the
existing code.

Bug: webrtc:11222
Change-Id: Idde92f68f34616aa3c34ab77a791fdbe7ea7af26
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166880
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30347}
2020-01-22 13:38:38 +00:00
Ilya Nikolaevskiy
33aaa35d54 Fix video_replay to build and actually work
Add it to default build target, so it won't get broken accidentally
again. Fix configuration issue with field trials (new parameter was
added recently, but wasn't set by video_replay)

Bug: webrtc:11287
Change-Id: I9c18746d899acd7ac68c1b9b3a646b862c41897a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166900
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30345}
2020-01-22 13:16:28 +00:00
Sergey Silkin
cd02ebaea0 Use intersection of app and encoder bitrate limits.
Before this change, if both app and encoder provided bitrate limits,
WebRTC ignored the limits provided by encoder. Now intersection of
these sets is used.

Also changed DCHECKs in GetEncoderBitrateLimits to allow zero values
of min_bitrate_bps and min_start_bitrate_bps.

Bug: none
Change-Id: Ib8be965ea43f51013b0a0f82fd4256a372432dda
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166600
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30338}
2020-01-21 15:16:29 +00:00
Rasmus Brandt
43bfe0b8a6 Enforce VideoEncoderConfig.num_temporal_layers >= 1.
This change clarifies the semantics of this field:
  unset: Depends on context.
  == 0: Invalid.
  == 1: No temporal layering.
  >= 2: Temporal layering.

We should try to remove the wrapping optional later.

Bug: webrtc:11297
Change-Id: Id765f2dc1d31a4ba3cd424978ac6054cd60152ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166528
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30336}
2020-01-21 13:38:08 +00:00
Henrik Boström
4bab2fcf6b [Overuse] Setting encoder configurations through the interface.
This squashes together several input signals that were spread out
through several calls into a single method and calling place:
SetEncoderSettings(), invoked from ReconfigureEncoder(). This is added
to the abstract interface.

This makes the following methods obsolete which are removed:
- SetEncoder(): The VideoEncoder was only used for GetEncoderInfo();
  the VideoEncoder::EncoderInfo is now part of the EncoderSettings.
- SetEncoderConfig(): The VideoEncoderConfig is part of
  EncoderSettings. The config is used for its codec_type and
  content_type enums.
- SetCodecMaxFrameRate(): The max frame rate was the same as
  VideoCodec::maxFramerate. VideoCodec is now part of EncoderSettings.

There may be some overlap in information between EncoderConfig and
VideoCodec, but that is outside the scope of this CL, which only makes
sure to bundle encoder settings-like information into one input signal.

Bug: webrtc:11222
Change-Id: I67c49c49c0a859cb7d5051939a461593c695a789
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166602
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30332}
2020-01-21 11:48:11 +00:00
Evan Shrubsole
6c13fd9071 Move bandwidth overuse detection out of VideoStreamEncoder
Bug: webrtc:11222
Change-Id: I12ccd008c848a0146fb22292f8dac46d1f7be9b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166531
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30329}
2020-01-21 10:24:31 +00:00
Ilya Nikolaevskiy
8b1338bf8e Propagate is_bw_limited flag with bw allocation everywhere it's copied
Bug: webrtc:11015
Change-Id: Ie971d29d8a8d140ba120a51dd3920291034a4d48
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30325}
2020-01-20 16:12:03 +00:00
Henrik Boström
fae6f0e87b [Overuse] MaybeUpdateTargetFrameRate() & ResetVideoSourceRestrictions()
This CL does two things for the sake of getting us closer to adaptation
modules being injectable and usable without knowing implementation
details.

Firstly, RefreshTargetFramerate() is removed. The target frame rate is
dependent on two things: 1) the codec max frame rate, and 2) the video
source restrictions. If either of these two changes, the target frame
rate is updated - there is no need to trigger this externally; the
module already knows if either of these factors change.
The private method MaybeUpdateTargetFrameRate() is added to ensure
overuse_detector->OnTargetFramerateUpdated() happens when necessary.

In doing this, the frame rates are updated to use
absl::optional<double>. This documents its optionality and avoids
magical values (previously -1 was not a bug but meaning "missing"). It
also matches VideoSourceRestrictions::max_frame_rate()'s type.

Secondly, ResetAdaptationCounters() is renamed
ResetVideoSourceRestrictions(). This more accurately describes what it
is doing; it is resetting the restrictions (the adaptation counters
getting reset is merely an implementation specific side-effect of
this). This method is added to the generic interface.

The usefulness of being able to ResetVideoSourceRestrictions() is
questioned in a TODO - current usage of this is when "quality rampup"
finishes. Nevertheless, any module could implement this functionality
so it belongs to the interface for now.

Bug: webrtc:11222
Change-Id: I079785df55fc9894e85087ec98be3e4ebd0713c5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30320}
2020-01-20 12:58:38 +00:00
Henrik Boström
a3d4252a01 ResourceAdaptationModule HasInputVideo+DegradationPreference setters.
The "has_input_video_ logic" is simplified to abort AdaptUp() and
AdaptDown() directly rather than in each calling place of the
VideoSourceRestrictor. The intent is no change in behavior.
The degradation_preference_ is removed from the VideoSourceRestrictor
as its only usage was DCHECKing (not worth it).

ResourceAdaptationModuleInterface gets SetHasInputVideo() and
SetDegradationPreference(), making these things controllable without
knowing implementation details.

StartCheckForOveruse() and StopCheckForOveruse() are renamed to
StartResourceAdaptation() and StopResourceAdaptation().

Bug: webrtc:11222
Change-Id: Id2d7f34d427dfb3ecd4831b1a245d07becae6520
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166173
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30296}
2020-01-17 10:43:02 +00:00
Danil Chapovalov
64f1f3f04e Replace RTC_FALLTHROUGH with ABSL_FALLTHROUGH_INTENTED
Bug: None
Change-Id: I7287403f3fb13b8e30f92ca3cf1882b03bb53a6e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166176
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30283}
2020-01-16 15:20:35 +00:00
Henrik Boström
07b17df771 Move DegradationPreference logic to the encoder queue.
This moves SetHasInputVideoAndDegradationPreference() to the encoder
queue. OveruseFrameDetectorResourceAdaptationModule is now entirely
single-threaded, including its inner class VideoSourceRestrictor.

VideoStreamEncoder now protects the module with RTC_GUARDED_BY. This
ensures it is safely used, even without a SequenceChecker inside of the
module. The module's |encoder_queue_| is removed.

The one task queue reference that is needed - passing down the current
task queue to StartCheckForOveruse() - is replaced by a TaskQueueBase*
(instead of rtc::TaskQueue*), enabling obtaining the current queue with
TaskQueueBase::Current(). (There is no rtc::TaskQueue::Current().)

Furthermore, the only uses of VideoSourceSinkController that isn't on
the encoder queue are documented, with a TODO saying if these are moved
the VideoSourceSinkController could also be made single-threaded.
However since this requires introducing a delay to
VideoStreamEncoder::SetSource() and VideoStreamEncoder::Stop(),
arguably a more risky change, if this is to be attempted that should be
in a separate CL.

Bug: webrtc:11222
Change-Id: I448ca5125708d5f66b95b0b180d6d24cc356dfa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165783
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30263}
2020-01-15 11:58:04 +00:00
Jonas Olsson
b2b2031457 Concatenate string literals at compile time.
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format

After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.

This primary benefit of this change is a small reduction in binary size.

Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
2020-01-14 14:47:48 +00:00
Henrik Boström
8234b92ba3 Move DegradationPreference logic out of VideoSourceSinkController.
The DegradationPreference logic is moved into
OveruseFrameDetectorResourceAdaptationModule. This makes the adaptation
module solely responsible for degradation preference, and the
VideoStreamEncoder the only bridge between the adaptation module and the
VideoSourceSinkController.

The adaptation module is now unaware of the existence of a controller.
It only "speaks" VideoSourceRestrictions, which is a big milestone in
making adaptation modules injectable.

A follow-up CL will explore the possibility of reconfiguring the
controller's source and which degradation preference to use to the
encoder queue. This would allow us to make several classes
single-threaded, but it is a change in behavior and should be done in a
separate CL.

Bug: webrtc:11222
Change-Id: Ib7f640e12789da5f801177926c2072a51818f261
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165684
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30237}
2020-01-13 17:24:48 +00:00
Henrik Boström
ce0ea49001 VideoStreamEncoder configuring source/sink with VideoSourceController.
This is part of the work for making VideoStreamEncoder responsible for
configuring its source/sink and limiting the responsibility of
OveruseFrameDetectorResourceAdaptationModule to only output relevant
VideoSourceRestrictions.

BEFORE THIS CL

Prior to this CL, OveruseFrameDetector was responsible for performing
AddOrUpdateSink() on the source, which it did using its nested class
VideoSourceProxy.

AddOrUpdateSink() could happen for both adaptation and non-adaptation
related reasons. For example:
- Adaptation related: AdaptUp() or AdaptDown() happens, causing updated
  VideoSourceRestrictions.
- Non-adaptation related: VideoStreamEncoder asks the module to
  reconfigure the source/sink for it, such as with
  SetMaxFramerateAndAlignment() or SetWantsRotationApplied().

AFTER THIS CL

AddOrUpdateSink() is performed by VideoSourceController, which is owned
by VideoStreamEncoder. Any reconfiguration has to go through the
VideoStreamEncoder. This means that:
- Non-adaptation related settings happen between VideoStreamEncoder and
  VideoSourceController directly (without going through the adaptation
  module).
- Adaptation related changes can be expressed in terms of
  VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule
  only has to output the restrictions and not know or care about other
  source/sink settings.

For now, VideoSourceController has to know about DegradationPreference.
In a future CL, the DegradationPreference logic should move back to
the adaptation module. The VideoSourceRestrictions are fully capable of
expressing all possible source/sink values without the "modifier" that
is the degradation preference.

Bug: webrtc:11222
Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:14:04 +00:00
Henrik Boström
d238200882 Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions.
The VideoSourceRestrictions describe the maximum pixels per frame and
max frame rate of a video source.

This CL makes the ResourceAdaptationModuleInterface responsible for the
reconfiguration of video sources. The VideoSourceRestrictions is the
output of an adaptation module, and the ResourceAdaptationModuleListener
handles the callback for when the source restrictions change.

The OveruseFrameDetectorResourceAdaptationModule is updated to output
its changes using these interfaces, and VideoStreamEncoder - now a
listener - is made responsible for triggering the reconfiguring the
video source.

Performing the reconfiguration still requires interacting with the
VideoSourceProxy - it is still partially responsible for keeping track
of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For
now this may look a bit weird: the VideoSourceProxy tells the
VideoStreamEncoder about the new restrictions, and then the
VideoStreamEncoder tells the VideoSourceProxy to apply these
restrictions on the source/sink. This exercises the listener though, and
unblocks the next CL.

The next CL should move all "configuring the source" logic to the
VideoStreamEncoder instead, so that the only information that is tracked
by OveruseFrameDetectorResourceAdaptationModule is what it actually
outputs to the listener. See the next CL
(https://webrtc-review.googlesource.com/c/src/+/162802) where a
VideoSourceController is introduced, to be owned by the
VideoStreamEncoder rather than the adaptation module.

Bug: webrtc:11222
Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-13 10:57:00 +00:00
philipel
539f9b376e Use a TaskQueue for decoding in VideoStreamDecoderImpl.
Long term goal is to use the VideoStreamDecoder in the VideoReceiveStream so
that we can stop using legacy VideoCodingModule components and classes. This CL is
one of several in preparation for that.

Bug: webrtc:7408, webrtc:9378
Change-Id: Ifd7e4c3c7d38dbb7c4b0636aaad318c571a29158
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/164525
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30211}
2020-01-10 14:31:22 +00:00
Danil Chapovalov
5c35f2fb1b Delete RtpDepacketizerVp9 in favor of VideoRtpDepacketizerVp9
Bug: webrtc:11152
Change-Id: Ic50f2dc49ca420b3406d4dea11ed20328aa59136
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165382
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30195}
2020-01-09 13:07:44 +00:00
Åsa Persson
f5e71e46dc BalancedDegradationSettings: add codec specific min bitrate settings.
Bug: none
Change-Id: I7875a3c8499c883ffecad519063c0eb82a8a7073
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165341
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30192}
2020-01-09 12:45:33 +00:00
Henrik Boström
382cc6d8a6 Add incomplete ResourceAdaptationModuleInterface.
This interface will be improved upon iteratively to aid reviewability.
The initial version only handles starting and stopping the module; input
and output of the module is still implementation-specific.

TBR=sprang@webrtc.org

Bug: webrtc:11222
Change-Id: Ie307cfe3d3211c84346c035f2c0e9a632f58221b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30167}
2020-01-07 13:24:42 +00:00
Henrik Boström
b08882b625 Refactor out VideoStreamEncoder's overuse logic to separate module.
This CL puts the VideoStreamEncoder's current adaptation logic inside
the new class OveruseFrameDetectorResourceAdaptationModule. The
intention is not to change any behavior, only to move code.

Future CLs should step by step decrease the coupling between
OveruseFrameDetectorResourceAdaptationModule, VideoStreamEncoder and
the VideoStreamEncoder's QualityScaler by introducing more abstract
interfaces. This is not done in this CL because it is large enough as
it is, but the long term goal is to make it possible to replace the
existing overuse module with a different implementation.

This CL relies on existing tests exercising the VideoStreamEncoder, but
part of making overuse logic modular should include testing each module
separately as well as continued integration testing of the
VideoStreamEncoder.

Bug: webrtc:11222
Change-Id: I316a174adfd00d60cdd224a23a5f616efd235d13
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161953
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30163}
2020-01-07 09:55:54 +00:00
Yves Gerey
499de2d395 Fix tiny race condition when Vp9NonFlexMode_* tests were failing.
TSAN complained at each failure, which only added confusion.

Bug: webrtc:11243
Change-Id: Ic36ec13562699478be23050a7b723e1b9bb47f0e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/164528
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Yves Gerey <yvesg@google.com>
Cr-Commit-Position: refs/heads/master@{#30161}
2020-01-07 09:29:14 +00:00
Yves Gerey
c33e4910c5 Prevent 'use after free' by waiting for all queued tasks to be processed.
Sometimes, a task bound to VideoSendStreamTest was called
after the underlying object had been destructed:
 1. |test| goes out of scope.
 2. There might still have been a task in fixture's queue,
    setup by OnSendRtp (capturing [this]) and invoked before
    the destruction of the fixture.

This CL uses the same workaround than BandwidthStatsTest:
block until all posted tasks are processed.

This fixes the following flaky tests:
 * VideoSendStreamTest.ChangingNetworkRoute
 * VideoSendStreamTest.RespectsMinTransmitBitrate*

Bug: webrtc:11156
Change-Id: I229c96d2abbbb60b43e9d9f62ad112507a21fe48
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/163984
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Yves Gerey <yvesg@google.com>
Cr-Commit-Position: refs/heads/master@{#30152}
2020-01-07 04:18:51 +00:00
Rasmus Brandt
5cad55b240 Signal requested resolution alignment requirements from sinks to sources.
Bug: webrtc:11218
Change-Id: I593b0515ea389bece472234a3c4082ccc5321ea5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162400
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30113}
2019-12-19 10:39:04 +00:00
Ilya Nikolaevskiy
9b540cb553 Correctly process disabled streams in FrameEncodeMetadataWriter
If the first simulcast stream is disabled, but the second one is enabled,
FrameEncodeMetadataWriter would fail to store frame metadata for all
streams and later fail to restore it for encoded frames.

Bug: none
Change-Id: Ib0d257abb863716ea94e56730f7caabef6ebeb64
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162480
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30106}
2019-12-17 12:28:04 +00:00
Danil Chapovalov
cebdbf650d switch RtpVideoStreamReceiver to use VideoRtpDepacketizer interface
instead of creating each time an object with RtpDepacketizer interface

this moves packet payload memcpy from RtpVideoStreamReceiver into
the depacketizers with possibility to remove it from there in follow ups.

Bug: webrtc:11152
Change-Id: If474207eb84d7e9d0207075bd395e60895f0d842
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162185
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30095}
2019-12-16 12:08:11 +00:00
Danil Chapovalov
0f6bf75ab4 Make video engine tests aware of padding packets
Specifically do not try to parse them as rtx packets.

Bug: webrtc:11213, webrtc:11188
Change-Id: I3aa5929af433b1ada9fb26516618d11207f075a5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162204
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Yves Gerey <yvesg@google.com>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30094}
2019-12-16 11:43:11 +00:00
Danil Chapovalov
b57fe17e7c Migrate video tests and tool to VideoRtpDepacketizer interface
Bug: webrtc:11152
Change-Id: I1e7868ca88b162db8615cb4903bd89d3daac4827
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161452
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30085}
2019-12-13 11:41:04 +00:00
Åsa Persson
4fc52c8329 Make struct SynchronizationDelays more general.
Bug: none
Change-Id: Iab263789cc8b51917acb3db2803fa71a927bc62a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161640
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30060}
2019-12-11 13:55:16 +00:00
Ying Wang
ef3998ffd1 Add directive to make webrtc metrics optional.
Bug: webrtc:11144
Change-Id: I4e75e6aec033784685de3670e880bb9f2b6ee8d2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161043
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Ying Wang <yinwa@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30040}
2019-12-09 13:55:50 +00:00
Artem Titov
33f9d2b383 Migrate WebRTC on FrameGeneratorInterface and remove FrameGenerator class
Bug: webrtc:10138
Change-Id: If85290581a72f81cf60181de7a7134cc9db7716e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161327
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30033}
2019-12-07 00:54:26 +00:00
Florent Castelli
907dc806c7 Reland "Add support for RtpEncodingParameters::max_framerate"
Perf test failure was fixed separately.

TBR=steveanton@webrtc.org,sprang@webrtc.org,asapersson@webrtc.org

Original change's description:
> This adds the framework support for the max_framerate parameter.
> It doesn't implement it in any encoder yet.
>
> Bug: webrtc:11117
> Change-Id: I329624cc0205c828498d3623a2e13dd3f97e1629
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160184
> Reviewed-by: Steve Anton <steveanton@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Commit-Queue: Florent Castelli <orphis@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#29907}

Bug: webrtc:11117
Change-Id: I9c1daf7887c2024c6669dc79bff89d737417458c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161445
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30030}
2019-12-06 15:11:54 +00:00
Danil Chapovalov
c347585927 Use RtpPacket instead of legacy RtpHeaderParser in video/ tests
Bug: None
Change-Id: Ia35daa58aae51becef40910187006398d825c5b9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161331
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30026}
2019-12-06 10:54:39 +00:00
Sebastian Jansson
340af975e9 Always enter yield policy scope using simulated TimeControllers.
This makes the class easier to use at a minor cost of making it slightly
more magic.

Bug: webrtc:9883
Change-Id: If807cfbf046615333c3bcd3b58a001813102a9f9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161231
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30008}
2019-12-04 17:16:32 +00:00
Ilya Nikolaevskiy
648b9d77c7 Implement automatic animation detection in VideoStreamEncoder
If WebRTC-AutomaticAnimationDetectionScreenshare experiment is enabled,
content type is screenshare and degradation preference is BALANCED,
then input resolution is restricted if update_rect of the incoming frames
is the same for considerable amount of time and is big enough.

This entails treating BALANCED degradation preference for screenshare as
MAINTAIN_RESOLUTION in adaptation logic.

Bug: webrtc:11058
Change-Id: I903dddf53fcbd7c8eac6c5b1447225b15fd8fe5f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161097
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30002}
2019-12-04 11:24:31 +00:00
Markus Handell
486cc55a02 TimeController: Rename Sleep to AdvanceTime.
This change renames TimeController's Sleep method to AdvanceTime, unifying
the same name with the same semantic as for downstream projects.

Bug: webrtc:11154
Change-Id: Id79bcf0eafcd0b47a76407ba220479d84df5a736
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161092
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29989}
2019-12-03 16:08:54 +00:00
Markus Handell
269ac81a86 VideoReceiveStream: Enable encoded frame sink.
This change ultimately enables wiring up VideoRtpReceiver::OnGenerateKeyFrame and
OnEncodedSinkEnabled into internal::VideoReceiveStream so that encoded frames
can flow to sinks installed in VideoTrackSourceInterface.

Bug: chromium:1013590
Change-Id: I0779932c251a2159880a39b2d42d5ce439cc88e6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161090
Commit-Queue: Markus Handell <handellm@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29988}
2019-12-03 15:55:04 +00:00
Danil Chapovalov
e3c4884b76 in PacketBuffer::Packet pass payload using smart buffer
Together with RtpDepacketizer refactoring that would reduce
number of memcpy while handling an rtp packet

Bug: webrtc:11152
Change-Id: I6f4e09c93af5e2a9314967a15eac8ced57ec712e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161087
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29985}
2019-12-03 14:55:54 +00:00
Erik Språng
014dd3c9f7 Trials should always be populated in call config.
The trials are always set when a Call instead is created by a
CallFactory, but a lot of unit tests creates instances directly.
This CL updates those call site. There is still a fallback in place
in RtpTransportControllerSend, since there are down-stream usages that
need to be clean up. After that, we'll remove the fallback.

Bug: webrtc:10809
Change-Id: I0aacf0473317bcd64252dd43d93c42de730e2ffa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/160408
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29978}
2019-12-03 10:34:55 +00:00