1357 Commits

Author SHA1 Message Date
Mirko Bonadei
2e161c4dd6 Revert "Remove ResourceAdaptationModule::OnMaybeEncodeFrame"
This reverts commit 93d9ae8a17f2e7b90641cbac28e740afc67d383a.

Reason for revert: Perf regression.

Original change's description:
> Remove ResourceAdaptationModule::OnMaybeEncodeFrame
>
> We can react just as well at OnEncodeVideoFrame, which is the same
> behaviour except after checking if the Encoder is paused and the frame
> dropper.
>
> For the initial frame drop, the frame dropper is irrelevant as the frame
> can not be dropped until we are accepting frames. If we didn't drop the
> frame, the encoder can't be paused as the data rate
> is over 0.
>
> For the quality rampup experiment, similar for encoder paused - we can't
> rampup if we are paused anyways since the data rate needs to be non-zero.
> If we are dropping frames we likely don't want to do quality rampup
> anyways.
>
> Bug: webrtc:11222
> Change-Id: Ie3e09d9d8d509dc17ba7a1443cf4747f61c04f6a
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168601
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#30539}

TBR=ilnik@webrtc.org,hbos@webrtc.org,eshr@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

No-Try: True
Bug: webrtc:11222
Change-Id: Ifb2fc74eb7572568fb0ee1b53a09e4180f87b30c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168880
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30568}
2020-02-20 11:03:25 +00:00
Danil Chapovalov
e8f4e09be9 Parse DependencyDescriptor rtp header extension
Bug: webrtc:10342
Change-Id: I1b5914232f73803774523fae215cf719c92da305
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168481
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30563}
2020-02-20 09:09:27 +00:00
Mirko Bonadei
4a14f4997c Remove wildcard ownership for build files.
No-Try: True
Bug: webrtc:10381
Change-Id: I852d9a2da7e0c5c12f508a1c788b0b5753503aba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168769
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30558}
2020-02-19 14:05:46 +00:00
Danil Chapovalov
b42c54f949 Refactor parsing generic descriptor extension into own function
Before making it even more complicated that it is right now.

Bug: webrtc:10342
Change-Id: I54f67309b8832cd85b6c5213f9b090908814ebd7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168766
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30557}
2020-02-19 13:50:36 +00:00
Danil Chapovalov
cad3e0e2fa Replace DataSize and DataRate factories with newer versions
This is search and replace change:
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataSize::Bytes<\(.*\)>()/DataSize::Bytes(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataSize::bytes/DataSize::Bytes/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataRate::BitsPerSec<\(.*\)>()/DataRate::BitsPerSec(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataRate::BytesPerSec<\(.*\)>()/DataRate::BytesPerSec(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataRate::KilobitsPerSec<\(.*\)>()/DataRate::KilobitsPerSec(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataRate::bps/DataRate::BitsPerSec/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/DataRate::kbps/DataRate::KilobitsPerSec/g"
git cl format

Bug: webrtc:9709
Change-Id: I65aaca69474ba038c1fe2dd8dc30d3f8e7b94c29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168647
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30545}
2020-02-18 16:09:50 +00:00
Markus Handell
c1cbf6be7e Ship GenericDescriptor00 by default.
The change ships GenericDescriptor00 and authentication by default,
but doesn't expose it by default, and makes WebRTC respond to
offers carrying it.

The change adds a unit test for the new semantics.

Tests well in munge-sdp. Frame marking replaced by
http://www.webrtc.org/experiments/rtp-hdrext/generic-frame-descriptor-00
in the offer results in an answer containing the
extension as first entry.

Bug: webrtc:11367
Change-Id: I0ef91b7d4096d949c3d547ece7d6c4d39aa241da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168661
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30542}
2020-02-18 11:11:48 +00:00
Evan Shrubsole
93d9ae8a17 Remove ResourceAdaptationModule::OnMaybeEncodeFrame
We can react just as well at OnEncodeVideoFrame, which is the same
behaviour except after checking if the Encoder is paused and the frame
dropper.

For the initial frame drop, the frame dropper is irrelevant as the frame
can not be dropped until we are accepting frames. If we didn't drop the
frame, the encoder can't be paused as the data rate
is over 0.

For the quality rampup experiment, similar for encoder paused - we can't
rampup if we are paused anyways since the data rate needs to be non-zero.
If we are dropping frames we likely don't want to do quality rampup
anyways.

Bug: webrtc:11222
Change-Id: Ie3e09d9d8d509dc17ba7a1443cf4747f61c04f6a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168601
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30539}
2020-02-18 10:50:08 +00:00
Åsa Persson
0e57858fa9 StreamSynchronizationTest: rename and make some variables const.
Bug: none
Change-Id: I5c452b0d2f58b2821db31b19506de2ba73480748
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168125
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30537}
2020-02-18 10:25:47 +00:00
Danil Chapovalov
e209fe6c68 Do not propagate generic descriptor on receiving frame
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.

This relands commit abf73de8eae90e9ac7e88ce1d52728e8102e824f.
with adjustments.

Change-Id: I935977179bef31d8e1023964b967658e9a7db92d
Bug: webrtc:10342
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168489
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30532}
2020-02-17 14:52:03 +00:00
Mirta Dvornicic
6799d732d5 Delete DefaultVideoBitrateAllocator.
It was removed from tests in https://webrtc-review.googlesource.com/c/src/+/123540.

If simulcast is not used, SimulcastRateAllocator returns the
same allocation as DefaultVideoBitrateAllocator.

Bug: webrtc:10164
Change-Id: I3d3e1aefe2fcc2bf853cd63c75e008b86eff9241
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168496
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Mirta Dvornicic <mirtad@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30509}
2020-02-12 21:29:09 +00:00
Danil Chapovalov
bc1750d52b Revert "Do not propagate generic descriptor on receiving frame"
This reverts commit abf73de8eae90e9ac7e88ce1d52728e8102e824f.

Reason for revert: breaks downstream tests

Original change's description:
> Do not propagate generic descriptor on receiving frame
> 
> It was used only for the frame decryptor.
> Decryptor needs only raw representation that it can recreate
> in a way compatible with the new version of the descriptor.
> 
> Bug: webrtc:10342
> Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
> Reviewed-by: Philip Eliasson <philipel@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#30501}

TBR=danilchap@webrtc.org,sprang@webrtc.org,philipel@webrtc.org

Change-Id: I6634df06ee75aa8cdfda614994ab11f7a5845c70
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10342
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168488
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30502}
2020-02-11 16:54:07 +00:00
Danil Chapovalov
abf73de8ea Do not propagate generic descriptor on receiving frame
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.

Bug: webrtc:10342
Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30501}
2020-02-11 16:12:16 +00:00
Henrik Boström
8cfecac6e8 [Overuse] Move initial framedrop logic into private inner class.
This is a subset of the module's behavior and accounts for 6 of the
member variables of the OveruseFrameDetectorResourceAdaptationModule.

Isolating this behavior to an inner class makes the module slightly less
convoluted.

Bug: webrtc:11222
Change-Id: Ibb5442afb03a1ee850da590b83cd5afbbb14783d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168309
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30500}
2020-02-11 16:11:11 +00:00
Evan Shrubsole
e67c6bcd06 Remove unused fields and includes from VideoStreamEncoder
Bug: webrtc:11222
Change-Id: Iec496d0955c1a30c61da147f0407fd76534129b7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168184
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30496}
2020-02-11 13:58:33 +00:00
Åsa Persson
74d2b1ded5 Add periodic logging of sync delays.
Bug: none
Change-Id: Ib2371651c7a912231c93742410a8aa1b01cc9896
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168344
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30495}
2020-02-11 09:43:49 +00:00
Danil Chapovalov
0c626afcf3 Use newer version of TimeDelta and TimeStamp factories in webrtc
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/TimeDelta::Micros<\(.*\)>()/TimeDelta::Micros(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/TimeDelta::Millis<\(.*\)>()/TimeDelta::Millis(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/TimeDelta::Seconds<\(.*\)>()/TimeDelta::Seconds(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/TimeDelta::us/TimeDelta::Micros/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/TimeDelta::ms/TimeDelta::Millis/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/TimeDelta::seconds/TimeDelta::Seconds/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/Timestamp::Micros<\(.*\)>()/Timestamp::Micros(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/Timestamp::Millis<\(.*\)>()/Timestamp::Millis(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/Timestamp::Seconds<\(.*\)>()/Timestamp::Seconds(\1)/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/Timestamp::us/Timestamp::Micros/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/Timestamp::ms/Timestamp::Millis/g"
find . -type f \( -name "*.h" -o -name "*.cc" \) | xargs sed -i -e "s/Timestamp::seconds/Timestamp::Seconds/g"
git cl format

Bug: None
Change-Id: I87469d2e4a38369654da839ab7c838215a7911e7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168402
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30491}
2020-02-10 12:21:17 +00:00
philipel
9b05803e19 Implement injectable EncoderSelectorInterface and wire it up in the VideoStreamEncoder.
The EncoderSelectorInterface is meant to replace the "WebRTC-NetworkCondition-EncoderSwitch" field trial, so the field trial will be ignored if an EncoderSelectorInterface object has been injected.

Bug: webrtc:11341
Change-Id: I5371fac9c9ad8e38223a81dd1e7bfefb2bb458cb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168193
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30490}
2020-02-10 12:12:47 +00:00
Mirko Bonadei
d4c3c3a454 Move video_replay under rtc_tools/.
As pointed out in [1], RTC public tools should live in rtc_tools.

[1] - https://webrtc-review.googlesource.com/c/src/+/168320/2#message-1f40103105ecb077aeec153c5270575138349a50

Bug: chromium:942546
Change-Id: Ic827d9b31ade9a32bf4ef24d020ef8c81d2c9a5b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168308
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30486}
2020-02-07 17:57:30 +00:00
Ying Wang
9b881abea9 Enable congestion window pushback to reduce bitrate by only drop video frames.
With current congestion window pushback, when congestion window is filling up, it will reduce bitrate directly and encoder may reduce encode quality, resolution, or framerate to adapt to the allocated bitrate, the behavior is depending on the degradation preference.
This change enable congestion window to only drop frames to reduce bitrate (when needed) instead of reduce general bitrate allocation.

Bug: webrtc:11334
Change-Id: I9cf5c20a0858c4d07d006942abe72aa5e1f7cb38
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168059
Commit-Queue: Ying Wang <yinwa@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30483}
2020-02-07 14:14:47 +00:00
Max Moroz
f12231d742 Add wildcard visibility to video_replay to make it buildable in Chromium.
Bug: chromium:942546
Change-Id: Ib798b58e854a2471ab1bb94725cb0ee2b04b84da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168320
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#30477}
2020-02-06 21:41:31 +00:00
Henrik Boström
065348503c [Overuse] Move EncodeUsageResource/QualityScalerResource to own files.
This CL changes EncodeUsageResource and QualityScalerResource from
private inner classes of OveruseFrameDetectorResourceAdaptationModule to
standalone classes, moving them into separate files.

This CL does not intend to change any lines of code, only move them.
Except for removing an unused method quality_scaler().

Bug: webrtc:11222
Change-Id: I86bf7eb78c80031888c403ac43c2bdf9b24eaea6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168198
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30472}
2020-02-06 14:08:39 +00:00
Henrik Boström
48258acabf [Overuse] Implement Resource and ResourceUsageListener.
The Resource interface (previously a skeleton not used outside of
testing) is updated to inform listeners of changes to resource
usage. Debugging methods are removed (Name, UsageUnitsOfMeasurements,
CurrentUsage). The interface is implemented by
OveruseFrameDetectorResourceAdaptationModule's inner classes
EncodeUsageResource and QualityScalerResource.

The new ResourceUsageListener interface is implemented by
OveruseFrameDetectorResourceAdaptationModule. In order to avoid adding
AdaptationObserverInterface::AdaptReason to the ResourceUsageListener
interface, the module figures out if the reason is "kCpu" or "kQuality"
by looking which Resource object triggered
OnResourceUsageStateMeasured(). These resources no longer need an
explicit reference to OveruseFrameDetectorResourceAdaptationModule and
could potentially be used by a different module.

In this CL, AdaptationObserverInterface::AdaptDown()'s return value is
still needed by QualityScaler. This is mirrored in the return value of
ResourceUsageListener::OnResourceUsageStateMeasured(). A TODO is added
to remove it and a comment explains how the current implementation
seems to break the contract of the method (as was the case prior to
this CL).

Follow-up work include:
- Move EncodeUsageResource and QualityScalerResource to separate files.
- Make resources injectable, allowing fake resources in testing and
  removing OnResourceOveruseForTesting() methods.
  (Investigate adding the necessary input signals to the Resource
  interface or relevant sub-interfaces so that the module does not need
  to know which Resource implementation is used.)
- And more! See whiteboard :)

Bug: webrtc:11222
Change-Id: I0a46ace4a2e617874e3ee97e67e3a199fef420a2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168180
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30469}
2020-02-06 12:45:14 +00:00
Henrik Boström
7875c99e82 [Overuse] Add EncodeUsageResource and QualityScalerResource.
This refactors the usage of OveruseFrameDetector in
OveruseFrameDetectorResourceAdaptationModule into an inner class of the
module, making the interaction between the detector and the module the
responsibility of this helper class instead.

Similarly, QualityScaler usage is moved into QualityScalerResource.

This takes us one step closer to separate the act of detecting
overuse/underuse of a resource and the logic of what to do when
overuse/underuse happens.

Follow-up CLs should build on this in order to materialize the concept
of having resources, streams and a central decision-maker deciding how
to reconfigure the streams based on resource usage state.

Bug: webrtc:11222
Change-Id: I99a08a42218a871db8f477f31447a6379433ad05
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168057
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30468}
2020-02-06 11:29:02 +00:00
Mirko Bonadei
a9e1026304 Make video_replay buildable from Chromium.
Bug: chromium:942546
Change-Id: Ic127e74b75ccb1fa65b317711d20344d0caee5fb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168280
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30467}
2020-02-06 10:55:22 +00:00
Evan Shrubsole
e331a122aa Move quality rampup experiment to overuse module
Bug: webrtc:11222
Change-Id: I8d0860bfe8bdfe0a051f5a6165cdcfa0cc25cfb5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168181
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30465}
2020-02-06 08:38:39 +00:00
Evan Shrubsole
7c3a1fc082 Move initial quality experiment to adaptation module
Bug: webrtc:11222
Change-Id: Iaa33bd6369a11f91e677b015eb2db412d0fbff23
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168053
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30456}
2020-02-05 10:10:22 +00:00
Evan Shrubsole
c81798b0c4 Configure QP scaler in adaptation module
Bug: webrtc:11222
Change-Id: Ia50ba3d024d0cbbaeddf8bf67ee652be602c5df9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168052
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30453}
2020-02-04 14:46:06 +00:00
Evan Shrubsole
f2be3eff26 Move initial frame drop to overuse module
It would be nice for this to stay in video stream encoder,
but this feature is mostly related to quality scaling. Perhaps
something easier to understand is possible in the future.

Bug: webrtc:11222
Change-Id: I71705f33ff94bbcf2fb9b5c94226c8e76dcba94c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168051
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30446}
2020-02-03 11:56:31 +00:00
Evan Shrubsole
c809e8bd62 Move quality scaling frame drop logic to adaptation module
Bug: webrtc:11222
Change-Id: I43db57fa128924ccaa3e44cd58098e7938e5ff5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168050
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30445}
2020-02-03 10:55:11 +00:00
Evan Shrubsole
bfe3ef8feb Report frame qp to quality scaler via overuse module
Bug: webrtc:11222
Change-Id: I63938adf5f623429eab1bcd668cde8fa5a1a083a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167924
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30435}
2020-01-31 09:18:28 +00:00
Evan Shrubsole
cf0595234c Move quality scaler into adaptation module
This allows for further refactoring, eventually moving
all of quality scaler out of video stream encoder.

Bug: webrtc:11222
Change-Id: Id121608da56f57549a616ccc5f141bb598668b40
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167728
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30417}
2020-01-30 09:05:19 +00:00
Evan Shrubsole
73a5e916a9 Remove task_queue dependency for QualityScaler
This allows for the possiblity to move the QualityScaler
out of the VideoStreamEncoder in the future.


Bug: webrtc:11222
Change-Id: I1d563cf08791e27ff5065ce90bcb150a7974d868
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167534
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30406}
2020-01-29 12:14:10 +00:00
Danil Chapovalov
97ffbefdab Pass and store PacketBuffer::Packet by unique_ptr
to avoid expensive move of the Packet and prepare PacketBuffer
to return list of packets as a frame.

Bug: None
Change-Id: I19f0452c52238228bbe28284ebb197491eb2bf4e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167063
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30404}
2020-01-29 11:48:55 +00:00
Henrik Boström
ad515a255b [Overuse] Move GetCpuOveruseOptions() to adaption module.
This removes the last remaining explicit reference from
OveruseFrameDetectorResourceAdaptationModule to
VideoStreamEncoder.

VideoStreamEncoder's call to SetEncoderSettings() inside
ReconfigureEncoder() is moved a few lines down - it was discovered that
during these lines the EncoderInfo config could get modified in
response to InitEncode() - so this fixes a potential bug.

Bug: webrtc:11222
Change-Id: I9746f28a4df8e631e297669c10636bf17b39acec
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167363
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30381}
2020-01-27 13:51:40 +00:00
Danil Chapovalov
159c414ff8 Detach LossNotificationController from RtpGenericFrameDescriptor
To allow to use the LossNotificationController with
an updated version of the frame descriptor extension

Bug: webrtc:10342
Change-Id: I5ac44dc5549dfcfc73bf81ad1e8eab8bd5dd136e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166166
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30369}
2020-01-24 11:53:28 +00:00
Artem Titov
cdd73e095c Migrate PC level tests on new video codec configuration API
Bug: webrtc:10138
Change-Id: I7129857724aafbfae71f36554ef79be78c442cb6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167066
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30362}
2020-01-23 13:45:33 +00:00
Evan Shrubsole
02d51f9fdc Remove unused field trial WebRTC-InitialFramedrop
Bug: webrtc:9176, webrtc:6086
Change-Id: Ie02800963f790f07b4c60ff01a04ecd6b5e1113d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167181
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30361}
2020-01-23 13:42:03 +00:00
Henrik Boström
d4578ae962 [Overuse] Encoding pipeline as input signals in the abstract interface.
This defines the following methods:
- OnFrame(), replaces SetLastFramePixelCount().
- OnFrameDroppedDueToSize(), a rename of FrameDroppedDueToSize() to
  match the other methods.
- OnEncodeStarted(), a rename of the incorrectly named FrameCaptured().
- OnEncodeCompleted(), a rename of the poorly named FrameSent().

In order to get rid of SetLastFramePixelCount(), the "we don't know the
frame size" use case - which was previously implicitly avoided by
invoking SetLastFramePixelCount() with a made-up value for
last_frame_info_ - is now avoided using ".value_or()" in
LastInputFrameSizeOrDefault(). This does mean that a constant 144p
resolution value is referenced in two places, but the fact that this is
a magic value is at least made explicit. This may help future
improvements.

Bug: webrtc:11222
Change-Id: I3b28daa8c5ecf57c6537957d4759f15e24bb2234
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166961
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30352}
2020-01-22 17:11:20 +00:00
Evan Shrubsole
2bc91e8c6a Avoid extra EncodedFrame copy in RunPostEncode
All uses of encoded_image are const, except for the copy for running on
the encoder_queue_.

Bug: None
Change-Id: I7fc8cb46f6afb42a2d27961d3d3ff8d9e63fe1b5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166442
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30351}
2020-01-22 15:41:38 +00:00
Henrik Boström
ede69c0fbe [Overuse] Setting the target bitrate through the interface.
The poorly named SetEncoderStartBitrate() is renamed
SetEncoderTargetBitrate() and added to the abstract resource adaptation
module interface.

The so-called "start bitrate" was updated to match the target bitrate,
so this was only ever a "start bitrate" until we had any estimates. The
variable is renamed in VideoStreamEncoder as well, and usage of optional
types are introduced to avoid magical values in a few places in the
existing code.

Bug: webrtc:11222
Change-Id: Idde92f68f34616aa3c34ab77a791fdbe7ea7af26
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166880
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30347}
2020-01-22 13:38:38 +00:00
Ilya Nikolaevskiy
33aaa35d54 Fix video_replay to build and actually work
Add it to default build target, so it won't get broken accidentally
again. Fix configuration issue with field trials (new parameter was
added recently, but wasn't set by video_replay)

Bug: webrtc:11287
Change-Id: I9c18746d899acd7ac68c1b9b3a646b862c41897a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166900
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30345}
2020-01-22 13:16:28 +00:00
Sergey Silkin
cd02ebaea0 Use intersection of app and encoder bitrate limits.
Before this change, if both app and encoder provided bitrate limits,
WebRTC ignored the limits provided by encoder. Now intersection of
these sets is used.

Also changed DCHECKs in GetEncoderBitrateLimits to allow zero values
of min_bitrate_bps and min_start_bitrate_bps.

Bug: none
Change-Id: Ib8be965ea43f51013b0a0f82fd4256a372432dda
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166600
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30338}
2020-01-21 15:16:29 +00:00
Rasmus Brandt
43bfe0b8a6 Enforce VideoEncoderConfig.num_temporal_layers >= 1.
This change clarifies the semantics of this field:
  unset: Depends on context.
  == 0: Invalid.
  == 1: No temporal layering.
  >= 2: Temporal layering.

We should try to remove the wrapping optional later.

Bug: webrtc:11297
Change-Id: Id765f2dc1d31a4ba3cd424978ac6054cd60152ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166528
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30336}
2020-01-21 13:38:08 +00:00
Henrik Boström
4bab2fcf6b [Overuse] Setting encoder configurations through the interface.
This squashes together several input signals that were spread out
through several calls into a single method and calling place:
SetEncoderSettings(), invoked from ReconfigureEncoder(). This is added
to the abstract interface.

This makes the following methods obsolete which are removed:
- SetEncoder(): The VideoEncoder was only used for GetEncoderInfo();
  the VideoEncoder::EncoderInfo is now part of the EncoderSettings.
- SetEncoderConfig(): The VideoEncoderConfig is part of
  EncoderSettings. The config is used for its codec_type and
  content_type enums.
- SetCodecMaxFrameRate(): The max frame rate was the same as
  VideoCodec::maxFramerate. VideoCodec is now part of EncoderSettings.

There may be some overlap in information between EncoderConfig and
VideoCodec, but that is outside the scope of this CL, which only makes
sure to bundle encoder settings-like information into one input signal.

Bug: webrtc:11222
Change-Id: I67c49c49c0a859cb7d5051939a461593c695a789
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166602
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30332}
2020-01-21 11:48:11 +00:00
Evan Shrubsole
6c13fd9071 Move bandwidth overuse detection out of VideoStreamEncoder
Bug: webrtc:11222
Change-Id: I12ccd008c848a0146fb22292f8dac46d1f7be9b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166531
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30329}
2020-01-21 10:24:31 +00:00
Ilya Nikolaevskiy
8b1338bf8e Propagate is_bw_limited flag with bw allocation everywhere it's copied
Bug: webrtc:11015
Change-Id: Ie971d29d8a8d140ba120a51dd3920291034a4d48
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30325}
2020-01-20 16:12:03 +00:00
Henrik Boström
fae6f0e87b [Overuse] MaybeUpdateTargetFrameRate() & ResetVideoSourceRestrictions()
This CL does two things for the sake of getting us closer to adaptation
modules being injectable and usable without knowing implementation
details.

Firstly, RefreshTargetFramerate() is removed. The target frame rate is
dependent on two things: 1) the codec max frame rate, and 2) the video
source restrictions. If either of these two changes, the target frame
rate is updated - there is no need to trigger this externally; the
module already knows if either of these factors change.
The private method MaybeUpdateTargetFrameRate() is added to ensure
overuse_detector->OnTargetFramerateUpdated() happens when necessary.

In doing this, the frame rates are updated to use
absl::optional<double>. This documents its optionality and avoids
magical values (previously -1 was not a bug but meaning "missing"). It
also matches VideoSourceRestrictions::max_frame_rate()'s type.

Secondly, ResetAdaptationCounters() is renamed
ResetVideoSourceRestrictions(). This more accurately describes what it
is doing; it is resetting the restrictions (the adaptation counters
getting reset is merely an implementation specific side-effect of
this). This method is added to the generic interface.

The usefulness of being able to ResetVideoSourceRestrictions() is
questioned in a TODO - current usage of this is when "quality rampup"
finishes. Nevertheless, any module could implement this functionality
so it belongs to the interface for now.

Bug: webrtc:11222
Change-Id: I079785df55fc9894e85087ec98be3e4ebd0713c5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30320}
2020-01-20 12:58:38 +00:00
Henrik Boström
a3d4252a01 ResourceAdaptationModule HasInputVideo+DegradationPreference setters.
The "has_input_video_ logic" is simplified to abort AdaptUp() and
AdaptDown() directly rather than in each calling place of the
VideoSourceRestrictor. The intent is no change in behavior.
The degradation_preference_ is removed from the VideoSourceRestrictor
as its only usage was DCHECKing (not worth it).

ResourceAdaptationModuleInterface gets SetHasInputVideo() and
SetDegradationPreference(), making these things controllable without
knowing implementation details.

StartCheckForOveruse() and StopCheckForOveruse() are renamed to
StartResourceAdaptation() and StopResourceAdaptation().

Bug: webrtc:11222
Change-Id: Id2d7f34d427dfb3ecd4831b1a245d07becae6520
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166173
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30296}
2020-01-17 10:43:02 +00:00
Danil Chapovalov
64f1f3f04e Replace RTC_FALLTHROUGH with ABSL_FALLTHROUGH_INTENTED
Bug: None
Change-Id: I7287403f3fb13b8e30f92ca3cf1882b03bb53a6e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166176
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30283}
2020-01-16 15:20:35 +00:00
Henrik Boström
07b17df771 Move DegradationPreference logic to the encoder queue.
This moves SetHasInputVideoAndDegradationPreference() to the encoder
queue. OveruseFrameDetectorResourceAdaptationModule is now entirely
single-threaded, including its inner class VideoSourceRestrictor.

VideoStreamEncoder now protects the module with RTC_GUARDED_BY. This
ensures it is safely used, even without a SequenceChecker inside of the
module. The module's |encoder_queue_| is removed.

The one task queue reference that is needed - passing down the current
task queue to StartCheckForOveruse() - is replaced by a TaskQueueBase*
(instead of rtc::TaskQueue*), enabling obtaining the current queue with
TaskQueueBase::Current(). (There is no rtc::TaskQueue::Current().)

Furthermore, the only uses of VideoSourceSinkController that isn't on
the encoder queue are documented, with a TODO saying if these are moved
the VideoSourceSinkController could also be made single-threaded.
However since this requires introducing a delay to
VideoStreamEncoder::SetSource() and VideoStreamEncoder::Stop(),
arguably a more risky change, if this is to be attempted that should be
in a separate CL.

Bug: webrtc:11222
Change-Id: I448ca5125708d5f66b95b0b180d6d24cc356dfa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165783
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30263}
2020-01-15 11:58:04 +00:00