This change reduces the number of times the Android hardware video
encoder is reconfigured when making an outgoing call. With this change,
the encoder should only be initialized once as opposed to the ~3 times
it happens currently.
Before the fix, the following sequence of events caused the extra
reconfigurations:
1. After the SetLocalDescription call, the WebRtcVideoSendStream is created.
All frames from the camera are dropped until the corresponding
VideoSendStream is created.
2. SetRemoteDescription() triggers the VideoSendStream creation. At
this point, the encoder is configured for the first time, with the
frame dimensions set to a low resolution default (176x144).
3. When the first video frame is received from the camera after the
VideoSendStreamIsCreated, the encoder is reconfigured to the correct
dimensions. If we are using the Android hardware encoder, the default
configuration is set to encode from a memory buffer (use_surface=false).
4. When the frame is passed down to the encoder in
androidmediaencoder_jni.cc EncodeOnCodecThread(), it may be stored in
a texture instead of a memory buffer. In this case, yet another
reconfiguration takes place to enable encoding from a texture.
5. Even if the resolution and texture flag were known at the start of
the call, there would be a reconfiguration involved if the camera is
rotated (such as when making a call from a phone in portrait orientation).
The reason for that is that at construction time, WebRtcVideoEngine2
sets the VideoSinkWants structure parameter to request frames rotated
by the source; the early frames will then arrive in portrait resolution.
When the remote description is finally set, if the rotation RTP extension
is supported by the remote receiver, the source is asked to provide
non-rotated frames. The very next frame will then arrive in landscape
resolution with a non-zero rotation value to be applied by the receiver.
Since the encoder was configured with the last (portrait) frame size,
it's going to need to be reconfigured again.
The fix makes the following changes:
1. WebRtcVideoSendStream::OnFrame() now caches the last seen frame
dimensions, and whether the frame was stored in a texture.
2. When the encoder is configured the first time
(WebRtcVideoSendStream::SetCodec()) - the last seen frame dimensions
are used instead of the default dimensions.
3. A flag that indicates if encoding is to be done from a texture has
been added to the webrtc::VideoStream and webrtc::VideoCodec structs,
and it's been wired up to be passed down all the way to the JNI code in
androidmediaencoder_jni.cc.
4. MediaCodecVideoEncoder::InitEncode is now reading the is_surface
flag from the VideoCodec structure instead of guessing the default as
false. This way we end up with the correct encoder configuration the
first time around.
5. WebRtcVideoSendStream now takes an optimistic guess and requests non-
rotated frames when the supported RtpExtensions list is not available.
This makes the "early" frames arrive non-rotated, and the cached dimensions
will be correct for the common case when the rotation extension is supported.
If the other side is an older endpoint which does not support rotation,
the encoder will have to be reconfigured - but it's better to penalize the
uncommon case rather than the common one.
Review-Url: https://codereview.webrtc.org/2067103002
Cr-Commit-Position: refs/heads/master@{#13173}
Earlier, no statistics were reported if no frames were being delivered
for encoding. This makes statics always be reported regardless of if
there are frames being delivered to the encoder.
Review-Url: https://codereview.webrtc.org/2051403002
Cr-Commit-Position: refs/heads/master@{#13122}
Introduce a new method I420Buffer::CropAndScale, and a static
convenience helper I420Buffer::CenterCropAndScale. Use them for almost
all scaling needs.
Delete the Scaler class and the cricket::VideoFrame::Stretch* methods.
BUG=webrtc:5682
R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2020593002 .
Cr-Commit-Position: refs/heads/master@{#13110}
Every message will now be traced with the location from which it was
posted, including function name, file and line number.
This CL also writes a normal LOG message when the dispatch took more
than a certain amount of time (currently 50ms).
This logging should help us identify messages that are taking
longer than expected to be dispatched.
R=pthatcher@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/2019423006 .
Cr-Commit-Position: refs/heads/master@{#13104}
Splits VideoCapturer::OnFrameCaptured into helper methods,
which enables use of the VideoAdaptation logic without
using a frame factory.
Refactors AndroidVideoCapturer to make adaptation decision
earlier, so we can crop and rotate using
NV12ToI420Rotate.
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/1973873003
Cr-Commit-Position: refs/heads/master@{#12895}
This CL makes the loop stop when all frames have been delivered and
start again when a new frame is inserted.
BUG=webrtc:5680
Review-Url: https://codereview.webrtc.org/2000103002
Cr-Commit-Position: refs/heads/master@{#12860}
Reason for revert:
Should work after cl https://codereview.webrtc.org/1985693002/ is landed, which initializes the frames used by FakeWebRtcVideoCaptureModule. So intend to reland after that, with no changes.
Original issue's description:
> Revert of Delete webrtc::VideoFrame methods buffer and stride. (patchset #2 id:290001 of https://codereview.webrtc.org/1963413004/ )
>
> Reason for revert:
> Speculative revert to see if failures on the DrMemory bot are related to this cl. See e.g. here:
> https://build.chromium.org/p/client.webrtc/builders/Win%20DrMemory%20Full/builds/4243
>
> UNINITIALIZED READ: reading 0x04980040-0x04980060 32 byte(s) within 0x04980040-0x04980060
> # 0 CopyRow_AVX
> # 1 CopyPlane
> # 2 I420Copy
> # 3 webrtc::ExtractBuffer
> # 4 cricket::WebRtcVideoCapturer::SignalFrameCapturedOnStartThread
> # 5 cricket::WebRtcVideoCapturer::OnIncomingCapturedFrame
> # 6 FakeWebRtcVideoCaptureModule::SendFrame
> # 7 WebRtcVideoCapturerTest_TestCaptureVcm_Test::TestBody
> # 8 testing::internal::HandleSehExceptionsInMethodIfSupported<>
>
> Original issue's description:
> > Reland of Delete webrtc::VideoFrame methods buffer and stride. (patchset #1 id:1 of https://codereview.webrtc.org/1935443002/ )
> >
> > Reason for revert:
> > I plan to reland this change in a week or two, after downstream users are updated.
> >
> > Original issue's description:
> > > Revert of Delete webrtc::VideoFrame methods buffer and stride. (patchset #14 id:250001 of https://codereview.webrtc.org/1900673002/ )
> > >
> > > Reason for revert:
> > > Breaks chrome FYI bots.
> > >
> > > Original issue's description:
> > > > Delete webrtc::VideoFrame methods buffer and stride.
> > > >
> > > > To make the HasOneRef/IsMutable hack work, also had to change the
> > > > video_frame_buffer method to return a const ref to a scoped_ref_ptr,
> > > > to not imply an AddRef.
> > > >
> > > > BUG=webrtc:5682
> > >
> > > TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
> > > # Skipping CQ checks because original CL landed less than 1 days ago.
> > > NOPRESUBMIT=true
> > > NOTREECHECKS=true
> > > NOTRY=true
> > > BUG=webrtc:5682
> > >
> > > Committed: https://crrev.com/5b3c443d301f2c2f18dac5b02652c08b91ea3828
> > > Cr-Commit-Position: refs/heads/master@{#12558}
> >
> > TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
> > # Not skipping CQ checks because original CL landed more than 1 days ago.
> > BUG=webrtc:5682
> >
> > Committed: https://crrev.com/d0dc66e0ea30c8614001e425a4ae0aa7dd56c2a7
> > Cr-Commit-Position: refs/heads/master@{#12721}
>
> TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org,nisse@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5682
>
> Committed: https://crrev.com/d49c30cd2fe442f2b5b4ecec8d5cbaa430464725
> Cr-Commit-Position: refs/heads/master@{#12745}
TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org,tommi@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/1979193003
Cr-Commit-Position: refs/heads/master@{#12773}
Reason for revert:
Speculative revert to see if failures on the DrMemory bot are related to this cl. See e.g. here:
https://build.chromium.org/p/client.webrtc/builders/Win%20DrMemory%20Full/builds/4243
UNINITIALIZED READ: reading 0x04980040-0x04980060 32 byte(s) within 0x04980040-0x04980060
# 0 CopyRow_AVX
# 1 CopyPlane
# 2 I420Copy
# 3 webrtc::ExtractBuffer
# 4 cricket::WebRtcVideoCapturer::SignalFrameCapturedOnStartThread
# 5 cricket::WebRtcVideoCapturer::OnIncomingCapturedFrame
# 6 FakeWebRtcVideoCaptureModule::SendFrame
# 7 WebRtcVideoCapturerTest_TestCaptureVcm_Test::TestBody
# 8 testing::internal::HandleSehExceptionsInMethodIfSupported<>
Original issue's description:
> Reland of Delete webrtc::VideoFrame methods buffer and stride. (patchset #1 id:1 of https://codereview.webrtc.org/1935443002/ )
>
> Reason for revert:
> I plan to reland this change in a week or two, after downstream users are updated.
>
> Original issue's description:
> > Revert of Delete webrtc::VideoFrame methods buffer and stride. (patchset #14 id:250001 of https://codereview.webrtc.org/1900673002/ )
> >
> > Reason for revert:
> > Breaks chrome FYI bots.
> >
> > Original issue's description:
> > > Delete webrtc::VideoFrame methods buffer and stride.
> > >
> > > To make the HasOneRef/IsMutable hack work, also had to change the
> > > video_frame_buffer method to return a const ref to a scoped_ref_ptr,
> > > to not imply an AddRef.
> > >
> > > BUG=webrtc:5682
> >
> > TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
> > # Skipping CQ checks because original CL landed less than 1 days ago.
> > NOPRESUBMIT=true
> > NOTREECHECKS=true
> > NOTRY=true
> > BUG=webrtc:5682
> >
> > Committed: https://crrev.com/5b3c443d301f2c2f18dac5b02652c08b91ea3828
> > Cr-Commit-Position: refs/heads/master@{#12558}
>
> TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
> # Not skipping CQ checks because original CL landed more than 1 days ago.
> BUG=webrtc:5682
>
> Committed: https://crrev.com/d0dc66e0ea30c8614001e425a4ae0aa7dd56c2a7
> Cr-Commit-Position: refs/heads/master@{#12721}
TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org,nisse@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/1983583002
Cr-Commit-Position: refs/heads/master@{#12745}
Reason for revert:
I plan to reland this change in a week or two, after downstream users are updated.
Original issue's description:
> Revert of Delete webrtc::VideoFrame methods buffer and stride. (patchset #14 id:250001 of https://codereview.webrtc.org/1900673002/ )
>
> Reason for revert:
> Breaks chrome FYI bots.
>
> Original issue's description:
> > Delete webrtc::VideoFrame methods buffer and stride.
> >
> > To make the HasOneRef/IsMutable hack work, also had to change the
> > video_frame_buffer method to return a const ref to a scoped_ref_ptr,
> > to not imply an AddRef.
> >
> > BUG=webrtc:5682
>
> TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5682
>
> Committed: https://crrev.com/5b3c443d301f2c2f18dac5b02652c08b91ea3828
> Cr-Commit-Position: refs/heads/master@{#12558}
TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/1963413004
Cr-Commit-Position: refs/heads/master@{#12721}
Reason for revert:
Breaks chrome FYI bots.
Original issue's description:
> Delete webrtc::VideoFrame methods buffer and stride.
>
> To make the HasOneRef/IsMutable hack work, also had to change the
> video_frame_buffer method to return a const ref to a scoped_ref_ptr,
> to not imply an AddRef.
>
> BUG=webrtc:5682
TBR=perkj@webrtc.org,magjed@webrtc.org,pbos@webrtc.org,pthatcher@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/1935443002
Cr-Commit-Position: refs/heads/master@{#12558}
To make the HasOneRef/IsMutable hack work, also had to change the
video_frame_buffer method to return a const ref to a scoped_ref_ptr,
to not imply an AddRef.
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/1900673002
Cr-Commit-Position: refs/heads/master@{#12557}
But keep #including scoped_ptr.h in .h files, so as not to break
WebRTC users who expect those .h files to give them rtc::scoped_ptr.
BUG=webrtc:5520
Review URL: https://codereview.webrtc.org/1930463002
Cr-Commit-Position: refs/heads/master@{#12530}
Removes data race inside QualityScaler which were called both from the
codec thread and the invoking thread at the same time.
BUG=webrtc:5678
R=glaznev@webrtc.org
Review URL: https://codereview.webrtc.org/1919883002 .
Cr-Commit-Position: refs/heads/master@{#12517}
Framerate-reduction code is disabled on all platforms, and this code
adds complexity. It's necessary to react fast, especially on mobile
platforms or other bad network conditions and framerate reduction adds
another step between HD and QVGA.
BUG=webrtc:5678, webrtc:5830
R=jackychen@webrtc.org, mflodman@webrtc.org
Review URL: https://codereview.webrtc.org/1885893002 .
Cr-Commit-Position: refs/heads/master@{#12503}
This fix a potential race where the rotation information of a sent frame does not match the encoded frame.
BUG=webrtc:5783
TEST= Run ApprtcDemo on IOs and Android with and without capture to texture and both VP8 and H264.
R=magjed@webrtc.org, pbos@webrtc.org, tkchin@webrtc.org
TBR=tkchin_webrtc // For IOS changes.
Review URL: https://codereview.webrtc.org/1886113003 .
Cr-Commit-Position: refs/heads/master@{#12426}
Increases measure time for downscale back to 5 seconds, this is required
to not over-react on hand-waving or quick device rotations.
Also increase max thresholds for QP a bit to not overreact when quality
still looks somewhat OK. Min thresholds for H264 seemed very low and are
increased to be sure that we can go back up again. The window is still
quite big with the increased max QP.
Also changes libvpx thresholds to use the same thresholds as the
encoder, they were excessively low before and wouldn't adapt on bad QPs
at all before (but rely on >60% framedropping based on bitrates to go
down).
BUG=webrtc:5678
R=stefan@webrtc.orgTBR=glaznev@webrtc.org
Review URL: https://codereview.webrtc.org/1894083002 .
Cr-Commit-Position: refs/heads/master@{#12403}
Instead, use the corresponding method on VideoFrameBuffer. In the process,
reduce code duplication in frame comparison functions used in
the test code.
Make FramesEqual use FrameBufsEqual. Make the latter support texture frames.
The cl also refactors VideoFrame::CopyFrame to use I420Buffer::Copy. This
has possibly undesired side effects of never reusing the frame buffer of
the destination frame, and producing a frame buffer which may use different
stride than the source frame.
BUG=webrtc:5682
Review URL: https://codereview.webrtc.org/1881953002
Cr-Commit-Position: refs/heads/master@{#12373}
Reason for revert:
Regressed behavior is actually desirable (go down to 360p instead of producing super-bad 720p).
Original issue's description:
> Revert of Make QualityScaler more responsive to downgrades. (patchset #3 id:40001 of https://codereview.webrtc.org/1830593003/ )
>
> Reason for revert:
> Speculative revert: want to see if this causes the regression in https://crbug.com/602621
>
> Original issue's description:
> > Make QualityScaler more responsive to downgrades.
> >
> > Permits going from HD to QVGA in 6 seconds instead of 10. Also adds
> > windows for going up quickly in the beginning of a call (before any
> > downscaling happens due to bad quality).
> >
> > BUG=webrtc:5678
> > R=glaznev@webrtc.org, stefan@webrtc.org
> >
> > Committed: https://crrev.com/85829fd90cc4e7a91c9857921b19e8fc126aeb60
> > Cr-Commit-Position: refs/heads/master@{#12219}
>
> TBR=glaznev@webrtc.org,stefan@webrtc.org,pbos@webrtc.org
> # Not skipping CQ checks because original CL landed more than 1 days ago.
> BUG=webrtc:5678
> NOTRY=true
>
> Committed: https://crrev.com/19b4fecf08e3fe215e431a260fb673553c15e569
> Cr-Commit-Position: refs/heads/master@{#12331}
TBR=glaznev@webrtc.org,stefan@webrtc.org,phoglund@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:602621, webrtc:5678
Review URL: https://codereview.webrtc.org/1887493003
Cr-Commit-Position: refs/heads/master@{#12341}
Reason for revert:
Speculative revert: want to see if this causes the regression in https://crbug.com/602621
Original issue's description:
> Make QualityScaler more responsive to downgrades.
>
> Permits going from HD to QVGA in 6 seconds instead of 10. Also adds
> windows for going up quickly in the beginning of a call (before any
> downscaling happens due to bad quality).
>
> BUG=webrtc:5678
> R=glaznev@webrtc.org, stefan@webrtc.org
>
> Committed: https://crrev.com/85829fd90cc4e7a91c9857921b19e8fc126aeb60
> Cr-Commit-Position: refs/heads/master@{#12219}
TBR=glaznev@webrtc.org,stefan@webrtc.org,pbos@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5678
NOTRY=true
Review URL: https://codereview.webrtc.org/1880103002
Cr-Commit-Position: refs/heads/master@{#12331}
This logic currently prevents loopback calls on Nexus 5X when it's
slightly overloaded to maintain input framerate since encoding at ~25fps
with one framedrop results in >70ms between frames naturally.
With this change applied Nexus 5X can maintain ~25fps both in and out
without building excessive latency (>2 frames) (this is now covered by
CPU adaptation outside the codec wrapper).
BUG=
R=glaznev@webrtc.org
Review URL: https://codereview.webrtc.org/1854413004 .
Cr-Commit-Position: refs/heads/master@{#12317}
Permits going from HD to QVGA in 6 seconds instead of 10. Also adds
windows for going up quickly in the beginning of a call (before any
downscaling happens due to bad quality).
BUG=webrtc:5678
R=glaznev@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/1830593003 .
Cr-Commit-Position: refs/heads/master@{#12219}
Boost low QP threashold to 21, otherwise VGA encoding never
scales up even at 2.5 Mbps.
Also reduce high QP threshold to scale down faster.
BUG=b/26504665
R=jackychen@google.com
Review URL: https://codereview.webrtc.org/1717763003 .
Cr-Commit-Position: refs/heads/master@{#11712}
In addition to the code moved from talk/app/webrtc
there were some files in webrtc/api/objctests that still
had the libjingle license header.
BUG=webrtc:5418
TBR=tkchin@webrtc.org
NOTRY=True
Review URL: https://codereview.webrtc.org/1680293005
Cr-Commit-Position: refs/heads/master@{#11552}
The previously disabled warnings that were inherited from
talk/build/common.gypi are now replaced by target-specific disabling
of only the failing warnings. Additional disabling was needed since the stricter
compilation warnings that applies to code in webrtc/.
License headers will be updated in a follow-up CL.
Other modifications:
* Updated the header guards.
* Sorted the includes using chromium/src/tools/sort-headers.py
except for these files:
talk/app/webrtc/peerconnectionendtoend_unittest.cc
talk/app/webrtc/java/jni/androidmediadecoder_jni.cc
talk/app/webrtc/java/jni/androidmediaencoder_jni.cc
webrtc/media/devices/win32devicemanager.cc
The HAVE_SCTP define was added for the peerconnection_unittests target
in api_tests.gyp.
I also checked that none of
SRTP_RELATIVE_PATH
HAVE_SRTP
HAVE_WEBRTC_VIDEO
HAVE_WEBRTC_VOICE
were used by the talk/app/webrtc code.
For Chromium, the following changes will need to be applied to the roll CL that updates the
DEPS for WebRTC and libjingle:
https://codereview.chromium.org/1615433002
BUG=webrtc:5418
NOPRESUBMIT=True
R=deadbeef@webrtc.org, pthatcher@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1610243002 .
Cr-Commit-Position: refs/heads/master@{#11545}
