The original purpose with keeping one pending frame in SurfaceViewRenderer was to reduce latency for the first rendered frame when we are waiting for the Surface to be created. However, it is very dangerous to hold a pending frame indefinitely when used with a SurfaceTexture, because the SurfaceTexture only has one frame and thus holding a frame in the renderer will freeze everything and typically cause timeout crashes.
Review URL: https://codereview.webrtc.org/1435413006
Cr-Commit-Position: refs/heads/master@{#10638}
Reason for revert:
Causes fallback to SW decoder if a renderer is put in the background.
Original issue's description:
> Patchset 1 is a pure
> revert of "Revert of "Android MediaCodecVideoDecoder: Manage lifetime of texture frames" https://codereview.webrtc.org/1378033003/
>
> Following patchsets move the responsibility of calculating the decode time to Java.
>
> TESTED= Apprtc loopback using H264 and VP8 on N5, N6, N7, S5
>
> Committed: https://crrev.com/9cb8982e64f08d3d630bf7c3d2bcc78c10db88e2
> Cr-Commit-Position: refs/heads/master@{#10597}
TBR=magjed@webrtc.org,glaznev@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
Review URL: https://codereview.webrtc.org/1441363002 .
Cr-Commit-Position: refs/heads/master@{#10637}
This CL attempts to annotate accesses on >16 API levels using as
small scopes as possible. The TargetApi notations mean "yes, I know
I'm accessing a higher API and I take responsibility for gating the
call on Android API level". The Encoder/Decoder classes are annotated
on the whole class, but they're only accessed through JNI; we should
annotate on method level otherwise and preferably on private methods.
This patch also fixes some compiler-level deprecation warnings (i.e.
-Xlint:deprecation), but probably not all of them.
BUG=webrtc:5063
R=henrika@webrtc.org, kjellander@webrtc.org, magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1412673008 .
Cr-Commit-Position: refs/heads/master@{#10624}
This make sure that the texture copy is syncronized.
To reproduce the problem I:
Reverted "Revert of "Android MediaCodecVideoDecoder: Manage lifetime of texture frames" https://codereview.webrtc.org/1378033003/"
commit 543b6ca30a43eeb069c699291460ce6bacc7959d.
Reverted "Enable SurfaceViewRenderer for AppRTCDemo"
commit 7076729c57c27aa813760d2038be02c36f4d7649.
and ran ApprtDemo in loopback and changed the orientation a couple of times.
TBR=glaznev@webrtc.org
Review URL: https://codereview.webrtc.org/1437823002
Cr-Commit-Position: refs/heads/master@{#10598}
revert of "Revert of "Android MediaCodecVideoDecoder: Manage lifetime of texture frames" https://codereview.webrtc.org/1378033003/
Following patchsets move the responsibility of calculating the decode time to Java.
TESTED= Apprtc loopback using H264 and VP8 on N5, N6, N7, S5
Review URL: https://codereview.webrtc.org/1422963003
Cr-Commit-Position: refs/heads/master@{#10597}
Add resource name to log outputs to distinguish local renderer from remote renderer.
This Cl also adds some thread checks and factors out a small helper function makeBlack().
Review URL: https://codereview.webrtc.org/1420203003
Cr-Commit-Position: refs/heads/master@{#10596}
This change VideoCapturerAndroid to attempt 3 times with a period of 300ms to open the camera if it fails.
This is so that if another application have it already opened, it would have more time to release it.
BUG=b/25190234
Review URL: https://codereview.webrtc.org/1422023007
Cr-Commit-Position: refs/heads/master@{#10559}
If SurfaceViewRenderer can't keep up with the stream of incoming frames it has to drop frames. Currently, new frames are dropped until the old pending frame is rendered. This CL drops the old pending frame instead.
Review URL: https://codereview.webrtc.org/1417063005
Cr-Commit-Position: refs/heads/master@{#10558}
Also distinguish between camera failures and failures due to that buffers has not been returned.
Adds unit tests for making sure CameraEventHandler.onError is triggered if frames are not returned.
BUG=b/25514149
Review URL: https://codereview.webrtc.org/1415013006
Cr-Commit-Position: refs/heads/master@{#10555}
MediaCodec.stop() call may hang in some rear cases. To avoid
application hang this call need to be done on separate thread and
possible error reported back to application.
Application may elect to continue executing and use another codec
instance for encoding/decoding or stop the call and exit.
BUG=b/24339249
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1425143005 .
Cr-Commit-Position: refs/heads/master@{#10467}
I replaced quitSafely() with a CountDownLatch. The reason for not using ThreadUtils.invokeUninterruptibly() is that I want to stop accepting frames asap, and invokeUninterruptibly() would still accept frames during the waiting time.
BUG=webrtc:4742
Review URL: https://codereview.webrtc.org/1418223002
Cr-Commit-Position: refs/heads/master@{#10393}
The purpose with this change is to support older API levels by replacing EGL14 (API lvl 17) with EGL10 (API lvl 1). The main purpose is to lower API lvl requirement for SurfaceViewRenderer from API lvl 17 to API lvl 15. Also, camera texture capture will work on API lvl < 17 (and texture encode/decode in MediaCodec, but we don't use MediaCodec below API lvl 18?).
GLSurfaceView/VideoRendererGui is already using EGL10.
EGL 1.1 - 1.4 added new functionality, but won't affect performance. We don't need the functionality, so there should be no reason to not use EGL 1.0.
I have profiled AppRTCDemo with Qualcomm Trepn Profiler on a Nexus 5 and Nexus 6 and couldn't see any difference.
Specifically, this CL:
* Update EglBase to use EGL10 instead of EGL14.
* Update imports from EGL14 to EGL10 in a lot of files (plus changing import order in some cases).
* Update VideoCapturerAndroid to always support texture capture.
Review URL: https://codereview.webrtc.org/1396013004
Cr-Commit-Position: refs/heads/master@{#10378}
Add events to track when camera is requested to open,
when first camera frame is available and when camera is
closed.
BUG=b/24271359
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1398793005 .
Cr-Commit-Position: refs/heads/master@{#10306}
The code that depends on the reverted CL is disabled but not removed. NativeHandleImpl is reverted to the previous implementation, and the new implementation is renamed to NativeTextureHandleImpl. Texture capture can not be used anymore, because it will crash in peerconnection_jni.cc.
Reason for revert:
Increased HW decoder latency and crashes related to that. Also suspected cause of video tearing.
Original issue's description:
> This CL should be the last one in a series to finally
> unblock camera texture capture.
>
> The SurfaceTexture.updateTexImage() calls are moved from
> the video renderers into MediaCodecVideoDecoder, and the
> destructor of the texture frames will signal
> MediaCodecVideoDecoder that the frame has returned. This
> CL also removes the SurfaceTexture from the native handle
> and only exposes the texture matrix instead, because only
> the video source should access the SurfaceTexture.
>
> BUG=webrtc:4993
> R=glaznev@webrtc.org, perkj@webrtc.org
>
> Committed: https://crrev.com/91b348c7029d843e06868ed12b728a809c53176c
> Cr-Commit-Position: refs/heads/master@{#10203}
TBR=glaznev
BUG=webrtc:4993
Review URL: https://codereview.webrtc.org/1394103005
Cr-Commit-Position: refs/heads/master@{#10288}
SurfaceViewHelper requires EGL14 that was added in API level 17. Since the SurfaceViewHelper is only neeed when we capture to textures, this cl change back to not use it when we are capturing to byte buffers.
Also, thread.quitsafely was added in level 18. Instead a new ThreadUtil method has been added for this.
BUG=b/24782220
TEST = run
ninja -C out/Debug libjingle_peerconnection_android_unittest && CHECKOUT_SOURCE_ROOT=`pwd` build/android/adb_install_apk.py --debug out/Debug/apks/libjingle_peerconnection_android_unittest.apk && ./third_party/android_tools/sdk/platform-tools/adb shell am instrument -w -e class org.webrtc.VideoCapturerAndroidTest org.webrtc.test/android.test.InstrumentationTestRunner on a device running Android 4.1 (I tried Nexus 7, the first version)
Review URL: https://codereview.webrtc.org/1401023003
Cr-Commit-Position: refs/heads/master@{#10265}
This CL is a small bug fix for "Android SurfaceViewRenderer: Allow to re-init after release() has been called" https://codereview.webrtc.org/1389203003/. It is only possible to clear the last image in release() if init() has been called beforehand.
TBR=hbos
BUG=webrtc:4742
Review URL: https://codereview.webrtc.org/1396573003 .
Cr-Commit-Position: refs/heads/master@{#10223}
This CL makes a thorough reset of all variables in release() and clears the last rendered image so that the SurfaceViewRenderer object can be reinitialized with init() and work properly. This CL also removes an implicit assumption that init() is called before surfaceCreated() - now they can be called in any order.
BUG=webrtc:4742
R=hbos@webrtc.org
Review URL: https://codereview.webrtc.org/1389203003 .
Cr-Commit-Position: refs/heads/master@{#10217}
This adds support for capturing to a texture in the Java part of VideoCapturerAndroid.
After this cl, the C++ also needs modification.
https://codereview.webrtc.org/1375953002/ contains the idea and have a working version where textures can be rendered in local preview.
BUG=webrtc:4993
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1383413002 .
Cr-Commit-Position: refs/heads/master@{#10213}
This CL should be the last one in a series to finally unblock camera texture capture.
The SurfaceTexture.updateTexImage() calls are moved from the video renderers into MediaCodecVideoDecoder, and the destructor of the texture frames will signal MediaCodecVideoDecoder that the frame has returned. This CL also removes the SurfaceTexture from the native handle and only exposes the texture matrix instead, because only the video source should access the SurfaceTexture.
BUG=webrtc:4993
R=glaznev@webrtc.org, perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1378033003 .
Cr-Commit-Position: refs/heads/master@{#10203}
This allows to correctly report first frame event in applications which
use same remote video renderer for multiple calls.
R=wzh@webrtc.org
Review URL: https://codereview.webrtc.org/1384353002 .
Cr-Commit-Position: refs/heads/master@{#10176}
Fixed a problem where eglBase.makecurrent() could be called after the context had been released if SurfaceTextureHelper was first created and immedately disconnected.
Add the possibility to inject a thread to use instead of creating a new.
BUG= webrtc:4993
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1384923002 .
Cr-Commit-Position: refs/heads/master@{#10174}
This CL shouldn't make any functional changes. It adds a new VideoCapturerAndroid.nativeCreateVideoCapturer() instead of always using VideoCapturer.nativeCreateVideoCapturer(). The purpose is to simplify androidvideocapturer_jni and VideoCapturerAndroid.create(). This way, it is possible to use the ctor instead of VideoCapturerAndroid.init() to initialize variables, and they can be made final etc.
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1360173002 .
Cr-Commit-Position: refs/heads/master@{#10171}
VideoRendererGui may need to render incoming frames multiple times. We currently call VideoRenderer.renderFrameDone() while we still hold references to the OES texture. This CL makes a deep copy of the OES texture before calling renderFrameDone(). This will truly release the dependency to the incoming frame, so that video textures sources can rely on the renderFrameDone() callback.
This CL is a part of the plan in https://codereview.webrtc.org/1357923002/.
The texture copy doesn't cause any measurable performance difference on a Nexus 5 using VideoRendererGui in a AppRTCDemo loopback call.
BUG=webrtc:4993
TEST=Revert "Enable SurfaceViewRenderer for AppRTCDemo" https://codereview.webrtc.org/1356603004/ and try AppRTCDemo.
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1370113005 .
Cr-Commit-Position: refs/heads/master@{#10157}
On some devices (confirmed Samsung) the focus mode is not configured correctly by default.
The fix explicitly set the focus mode to FOCUS_MODE_CONTINUOUS_VIDEO if this mode is supported.
BUG=webrtc:4991
Review URL: https://codereview.webrtc.org/1338773002
Cr-Commit-Position: refs/heads/master@{#10128}
This CL refactors RendererCommon.getSamplingMatrix() so it does not have any dependecy to SurfaceTeture. The purpose is to prepare for a change in how texture frames are represented - only the texture matrix will be exposed, not the SurfaceTexture itself. This CL also adds an extra test for RendererCommon.rotateTextureMatrix().
R=hbos@webrtc.org
Review URL: https://codereview.webrtc.org/1375593002 .
Cr-Commit-Position: refs/heads/master@{#10118}
Reason for revert:
The top row in the video stream from the camera is messed up. The byte[] pointer is not the same as GetDirectBufferAddress() apparently.
Original issue's description:
> Android VideoCapturer: Send ByteBuffer instead of byte[]
>
> The purpose with this CL is to replace GetByteArrayElements() and ReleaseByteArrayElements() with GetDirectBufferAddress().
>
> R=hbos@webrtc.org
>
> Committed: https://crrev.com/cb3649b40b3fd6d5bbb0a92003b717e46ce90924
> Cr-Commit-Position: refs/heads/master@{#10091}
TBR=hbos@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.webrtc.org/1377783002
Cr-Commit-Position: refs/heads/master@{#10103}
The purpose with this CL is to replace GetByteArrayElements() and ReleaseByteArrayElements() with GetDirectBufferAddress().
R=hbos@webrtc.org
Review URL: https://codereview.webrtc.org/1372813002 .
Cr-Commit-Position: refs/heads/master@{#10091}
This CL also makes some small non-functional changes in ThreadUtils and EglBase to support SurfaceTextures and SurfaceTextureHelper.
BUG=webrtc:4993
R=hbos@webrtc.org
Review URL: https://codereview.webrtc.org/1368093003 .
Cr-Commit-Position: refs/heads/master@{#10085}
This cl adds a C++ counterpart of the Java SurfaceTextureHandler. It can be used for creating a webrtc::VideoFrames from a native handle and also guarantee that the Java SurfaceTexture is notified when the VideoFrame is no longer in use.
BUG=webrtc:4993
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1366413003 .
Cr-Commit-Position: refs/heads/master@{#10084}
This CL makes the following changes:
* Instead of creating a new thread per startCapture()/stopCapture() cycle, VideoCapturerAndroid has a single thread that is initialized in the constructor and kept during the lifetime of the instance. This is more convenient because then it is always possible to post runnables without if-checks. This way, a lot of synchronize statements can be removed. Also, when the camera thread is preserved after stopCapture() it is possible to post late returnBuffer() calls to the correct thread.
* FramePool now enforces single thread use and returnBuffer() calls are posted to the camera thread. This is important because the camera should only be used from one thread, and we call camera.addCallbackBuffer() in returnBuffer().
* switchCamera() no longer returns false on failure, but instead signals the result via the callback.
* Update the test testCaptureAndAsyncRender() - it's not a valid use case to have outstanding frames when calling PeerConnectionFactory.dispose(). Instead, the renderer implementations should have release() functions that block until all frames are returned. The release() functions need to be called in the correct order with PeerConnectionFactory.dispose() last.
BUG=webrtc:4909
R=hbos@webrtc.org, perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1350863002 .
Cr-Commit-Position: refs/heads/master@{#10025}
