Use TaskQueue in IncomingVideoStream instead of the PlatformThread + event timer approach.
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:7219, webrtc:7253
Reland of
686aa37382 (revert)
e2d1d64295 (original)
Review-Url: https://codereview.webrtc.org/2720773002
Cr-Commit-Position: refs/heads/master@{#16872}
Reason for revert:
Reverting while fixing build issue in Chromium.
Original issue's description:
> Use TaskQueue in IncomingVideoStream instead of the PlatformThread + event timer approach.
>
> BUG=webrtc:7219, webrtc:7253
>
> Review-Url: https://codereview.webrtc.org/2716473002
> Cr-Commit-Position: refs/heads/master@{#16860}
> Committed: e2d1d64295TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:7219, webrtc:7253
Review-Url: https://codereview.webrtc.org/2714393003
Cr-Commit-Position: refs/heads/master@{#16863}
The new constructor introduces two new changes:
* Support specifying thread priority at construction time.
- Moving forward, the SetPriority() method will be removed.
* New thread function type.
- The new type has 'void' as a return type and a polling loop
inside PlatformThread, is not used.
The old function type is still supported until all places have been moved over.
In this CL, the first steps towards deprecating the old mechanism are taken
by moving parts of the code that were simple to move, over to the new callback
type.
BUG=webrtc:7187
Review-Url: https://codereview.webrtc.org/2708723003
Cr-Commit-Position: refs/heads/master@{#16779}
Reason for revert:
Speculative revert due to regression in perf tests.
Original issue's description:
> Added VP8 simulcast tests. Fixed analyzer to correctly infer timestamps.
>
>
> BUG=webrtc:7095
>
> Review-Url: https://codereview.webrtc.org/2668763004
> Cr-Commit-Position: refs/heads/master@{#16428}
> Committed: 5f47126865TBR=sprang@webrtc.org,nisse@webrtc.org,mflodman@webrtc.org,magjed@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:7095
Review-Url: https://codereview.webrtc.org/2687073002
Cr-Commit-Position: refs/heads/master@{#16510}
Moves webrtc/common_video/rotation.h and parts of
webrtc/common_video/include/video_frame_buffer.h and
webrtc/video_frame.h, and adds to a new GN target api:video_frame_api.
BUG=webrtc:5880
Review-Url: https://codereview.webrtc.org/2517173004
Cr-Commit-Position: refs/heads/master@{#15993}
This is yet another reland of https://codereview.webrtc.org/2434073003/
including two fixes:
1. SimulcastRateAllocator did not handle the screenshare settings properly for numSimulcastStreams = 1. Additional test case was added for that.
2. In VideoSender, when rate allocation is updated after setting a new VideoCodec config, only update the state of the EncoderParameters, but don't actually run SetRateAllocation on the encoder itself. This caused some problems upstreams.
Please review only the changes after patch set 1.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
R=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2510583002 .
Cr-Commit-Position: refs/heads/master@{#15105}
Chrome coding standard now discourages use of references to smart
pointers. This cl updates some recent methods to the new conventions.
BUG=webrtc:6672
Review-Url: https://codereview.webrtc.org/2477233004
Cr-Commit-Position: refs/heads/master@{#15028}
Reason for revert:
Seems to be causing flakiness in perf test:
FullStackTest.ScreenshareSlidesVP8_2TL_LossyNet
Original issue's description:
> Reland of Issue 2434073003: Extract bitrate allocation ...
>
> This is a reland of https://codereview.webrtc.org/2434073003/ including
> some fixes for failing test cases.
>
> Original description:
>
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/647bf43dcb2fd16fccf276bd94dc4400728bb405
> Cr-Commit-Position: refs/heads/master@{#15023}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2491393002
Cr-Commit-Position: refs/heads/master@{#15026}
This is a reland of https://codereview.webrtc.org/2434073003/ including
some fixes for failing test cases.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2488833004
Cr-Commit-Position: refs/heads/master@{#15023}
Reason for revert:
Breaks perf tests.
Original issue's description:
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/8f46c679d24a05b3f08e02c6d91ec9637f34e24f
> Cr-Commit-Position: refs/heads/master@{#14998}
TBR=stefan@webrtc.org,perkj@webrtc.org,mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2489843002
Cr-Commit-Position: refs/heads/master@{#15001}
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2434073003
Cr-Commit-Position: refs/heads/master@{#14998}
Also improve denoiser to not assume identical stride of all involved
frames, and delete the no longer needed function I420Buffer::CopyKeepStride.
BUG=None
Review-Url: https://codereview.webrtc.org/2469763002
Cr-Commit-Position: refs/heads/master@{#14940}
This CL makes scaling and cropping lazy in AVFoundationVideoCapturer and
provides optimized paths for SW and HW encoding. For SW encoding, an
efficient NV12 -> I420 cropping and scaling is implemented in
CoreVideoFrameBuffer::NativeToI420. For HW encoding, an efficient NV12 ->
NV12 cropping and scaling is implemented in
CoreVideoFrameBuffer::CropAndScaleTo. The performance improvement over
the existing cropping and scaling is that it is now done in one step
instead of making an intermediary copy of the Y plane.
There might still be room for improvement in the HW path using some HW
support. That will be explored in a future CL.
BUG=b/30939444
Review-Url: https://codereview.webrtc.org/2394483005
Cr-Commit-Position: refs/heads/master@{#14701}
If more than 60 frames are created and not returned, the implementation will crash.
I420BufferPool are currently used by the VP8 decoder, Quality scaler and VideoFrameFactory.
BUG=b/31390397
NOTRY=true // Because of failing gclient runhooks on some bots
Review-Url: https://codereview.webrtc.org/2370653003
Cr-Commit-Position: refs/heads/master@{#14395}
Deleted from the VideoFrameBuffer base class.
BUG=webrtc:5921
Review-Url: https://codereview.webrtc.org/2278883002
Cr-Commit-Position: refs/heads/master@{#14317}
GetCopyWithRotationApplied is not yet deleted; downstream projects
must be updated first.
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/2285693002
Cr-Commit-Position: refs/heads/master@{#13973}
This cl is in preparation for https://codereview.webrtc.org/2060403002/ Add task queue to Call.
In the coming cl the video_sender, and i420_buffer_pool will be used on a task queue and therefore SequencedTaskChecker is needed instead of a ThreadChecker.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2149553002
Cr-Commit-Position: refs/heads/master@{#13474}
Permits CHECKing/DCHECKing that methods are being accessed in a
thread-safe manner, even if they are not used by one single thread
(thread pools such as VideoToolbox OK).
BUG=
R=danilchap@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/2097403002 .
Cr-Commit-Position: refs/heads/master@{#13358}
Reason for revert:
Reverting the revert. This change is not related to the failure on the Windows FYI bots. The cause of the failure has been reverted in Chromium:
https://codereview.chromium.org/2081653004/
Original issue's description:
> Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #5 id:340001 of https://codereview.webrtc.org/2078873002/ )
>
> Reason for revert:
> Breaks chromium.webrtc.fyi
>
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120
>
> Original issue's description:
> > Reland of IncomingVideoStream refactoring.
> > This reland does not contain the non-smoothing part of the original implementation. Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before. This code path is used in Chrome. As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks). Other than that, no change for Chrome.
> >
> > Original issue's description (with non-smoothing references removed):
> >
> > Split IncomingVideoStream into two implementations, with smoothing and without.
> >
> > * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
> >
> > * Removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
> >
> > * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
> >
> > * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
> >
> > * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
> >
> > * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
> >
> > * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
> >
> > * Made the render delay value in VideoRenderFrames, const.
> >
> > BUG=chromium:620232
> > R=mflodman@webrtc.org, nisse@webrtc.org
> >
> > Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> > Cr-Commit-Position: refs/heads/master@{#13219}
>
> TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:620232
>
> Committed: https://crrev.com/a536bfe70de38fe877245317a7f0b00bcf69cbd0
> Cr-Commit-Position: refs/heads/master@{#13229}
TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,sakal@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2089613002
Cr-Commit-Position: refs/heads/master@{#13230}
Reason for revert:
Breaks chromium.webrtc.fyi
https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120
Original issue's description:
> Reland of IncomingVideoStream refactoring.
> This reland does not contain the non-smoothing part of the original implementation. Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before. This code path is used in Chrome. As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks). Other than that, no change for Chrome.
>
> Original issue's description (with non-smoothing references removed):
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
>
> * Removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> R=mflodman@webrtc.org, nisse@webrtc.org
>
> Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> Cr-Commit-Position: refs/heads/master@{#13219}
TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2084873002
Cr-Commit-Position: refs/heads/master@{#13229}
This reland does not contain the non-smoothing part of the original implementation. Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before. This code path is used in Chrome. As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks). Other than that, no change for Chrome.
Original issue's description (with non-smoothing references removed):
Split IncomingVideoStream into two implementations, with smoothing and without.
* Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
* Removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=chromium:620232
R=mflodman@webrtc.org, nisse@webrtc.org
Review URL: https://codereview.webrtc.org/2078873002 .
Cr-Commit-Position: refs/heads/master@{#13219}
Don't use VideoFrameBuffer::MutableDataY and friends, instead, use
I420Buffer::SetToBlack.
Also introduce static method I420Buffer::Create, to create an object and
return a scoped_refptr.
TBR=marpan@webrtc.org # Trivial change to video_denoiser.cc
BUG=webrtc:5921
Review-Url: https://codereview.webrtc.org/2078943002
Cr-Commit-Position: refs/heads/master@{#13212}
Reason for revert:
Reverting again. The perf regression does not seem to be related to dropping frames.
Original issue's description:
> Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
>
> Original issue's description:
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> TBR=mflodman
>
> Committed: https://crrev.com/e03f8787377bbc03a4e00184bb14b7561b108cbb
> Cr-Commit-Position: refs/heads/master@{#13175}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2071093002
Cr-Commit-Position: refs/heads/master@{#13176}
Original issue's description:
Split IncomingVideoStream into two implementations, with smoothing and without.
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
Further work done:
* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
* I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=chromium:620232
TBR=mflodman
Review-Url: https://codereview.webrtc.org/2071473002
Cr-Commit-Position: refs/heads/master@{#13175}
Reason for revert:
Reverting while we track down the issue on the Win10 bot.
Original issue's description:
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=
>
> Committed: https://crrev.com/1c7eef652b0aa22d8ebb0bfe2b547094a794be22
> Cr-Commit-Position: refs/heads/master@{#13129}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=
Review-Url: https://codereview.webrtc.org/2061363002
Cr-Commit-Position: refs/heads/master@{#13146}
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
Further work done:
* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
* I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=
Review-Url: https://codereview.webrtc.org/2035173002
Cr-Commit-Position: refs/heads/master@{#13129}
Introduce a new method I420Buffer::CropAndScale, and a static
convenience helper I420Buffer::CenterCropAndScale. Use them for almost
all scaling needs.
Delete the Scaler class and the cricket::VideoFrame::Stretch* methods.
BUG=webrtc:5682
R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2020593002 .
Cr-Commit-Position: refs/heads/master@{#13110}
To avoid the case where a single data point or too short window is used,
causing bad behavior due to bad stats, update RateStatistics to return
an Optional rather than a plain rate.
There was also a strange off by one bug where the rate was slightly
overestimated (N + 1 buckets, N ms time window).
These changes requires updates to a number of places, and may very well
cause seeming perf regressions (but the stats were probablty more wrong
previously).
BUG=
R=mflodman@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2029593002 .
Cr-Commit-Position: refs/heads/master@{#13103}
Reason for revert:
Plan to reland with InitToBlack kept, to be able to update Chrome to use the new I420Buffer::SetToBlack method.
Original issue's description:
> Revert of New static method I420Buffer::SetToBlack. (patchset #4 id:60001 of https://codereview.webrtc.org/2029273004/ )
>
> Reason for revert:
> Breaks chrome, in particular, the tests in
>
> media_stream_remote_video_source_unittest.cc
>
> use the InitToBlack method which is being deleted.
>
> Original issue's description:
> > New static method I420Buffer::SetToBlack.
> >
> > Replaces cricket::VideoFrame::SetToBlack and
> > cricket::WebRtcVideoFrame::InitToBlack, which are deleted.
> >
> > Refactors the black frame logic in VideoBroadcaster, and a few of the
> > tests.
> >
> > BUG=webrtc:5682
> >
> > Committed: https://crrev.com/663f9e2ddc86e813f6db04ba2cf5ac1ed9e7ef67
> > Cr-Commit-Position: refs/heads/master@{#13063}
>
> TBR=perkj@webrtc.org,pbos@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5682
>
> Committed: https://crrev.com/271d74078894bb24f454eb31b77e4ce38097a2fa
> Cr-Commit-Position: refs/heads/master@{#13065}
TBR=perkj@webrtc.org,pbos@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/2049513005
Cr-Commit-Position: refs/heads/master@{#13083}
