25 Commits

Author SHA1 Message Date
magjed
5a8724564c iOS: Optimize video scaling and cropping
This CL makes scaling and cropping lazy in AVFoundationVideoCapturer and
provides optimized paths for SW and HW encoding. For SW encoding, an
efficient NV12 -> I420 cropping and scaling is implemented in
CoreVideoFrameBuffer::NativeToI420. For HW encoding, an efficient NV12 ->
NV12 cropping and scaling is implemented in
CoreVideoFrameBuffer::CropAndScaleTo. The performance improvement over
the existing cropping and scaling is that it is now done in one step
instead of making an intermediary copy of the Y plane.

There might still be room for improvement in the HW path using some HW
support. That will be explored in a future CL.

BUG=b/30939444

Review-Url: https://codereview.webrtc.org/2394483005
Cr-Commit-Position: refs/heads/master@{#14701}
2016-10-20 10:34:32 +00:00
nisse
09347858f7 Reland of Make cricket::VideoFrame inherit webrtc::VideoFrame. (patchset #1 id:1 of https://codereview.webrtc.org/2402853002/ )
This cl now makes cricket::VideoFrame and cricket::WebRtcVideoFrame aliases for webrtc::VideoFrame.

Reason for revert:
Fixing backwards compatibility issues.

Original issue's description:
> Revert of Make cricket::VideoFrame inherit webrtc::VideoFrame. (patchset #9 id:160001 of https://codereview.webrtc.org/2315663002/ )
>
> Reason for revert:
> Breaks compile for Chromium builds:
> https://build.chromium.org/p/chromium.webrtc.fyi/builders/Linux%20Builder/builds/10761
> https://build.chromium.org/p/chromium.webrtc.fyi/builders/Mac%20Builder/builds/18142
>
> FAILED: obj/remoting/protocol/protocol/webrtc_video_renderer_adapter.o
> ../../remoting/protocol/webrtc_video_renderer_adapter.cc:110:52: error: no member named 'transport_frame_id' in 'cricket::VideoFrame'
>                  weak_factory_.GetWeakPtr(), frame.transport_frame_id(),
>                                              ~~~~~ ^
> 1 error generated.
>
> Please run chromium trybots as described at https://webrtc.org/contributing/#tryjobs-on-chromium-trybots before relanding.
>
> Original issue's description:
> > Make cricket::VideoFrame inherit webrtc::VideoFrame. Delete
> > all methods but a few constructors. And similarly for the
> > subclass cricket::WebRtcVideoFrame.
> >
> > TBR=tkchin@webrtc.org  # Added an include line
> > BUG=webrtc:5682
> >
> > Committed: https://crrev.com/dda6ec008a0fc8d52e118814fb779032e8931968
> > Cr-Commit-Position: refs/heads/master@{#14576}
>
> TBR=perkj@webrtc.org,pthatcher@webrtc.org,pthatcher@chromium.org,tkchin@webrtc.org,nisse@webrtc.org
> NOTRY=True
> NOPRESUBMIT=True
> BUG=webrtc:5682
>
> Committed: https://crrev.com/d36dd499c8f253cbcf37364c2a070c2e8c7100e9
> Cr-Commit-Position: refs/heads/master@{#14583}

TBR=perkj@webrtc.org,pthatcher@webrtc.org,pthatcher@chromium.org,tkchin@webrtc.org,kjellander@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2411953002
Cr-Commit-Position: refs/heads/master@{#14678}
2016-10-19 07:30:35 +00:00
kjellander
d36dd499c8 Revert of Make cricket::VideoFrame inherit webrtc::VideoFrame. (patchset #9 id:160001 of https://codereview.webrtc.org/2315663002/ )
Reason for revert:
Breaks compile for Chromium builds:
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Linux%20Builder/builds/10761
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Mac%20Builder/builds/18142

FAILED: obj/remoting/protocol/protocol/webrtc_video_renderer_adapter.o
../../remoting/protocol/webrtc_video_renderer_adapter.cc:110:52: error: no member named 'transport_frame_id' in 'cricket::VideoFrame'
                 weak_factory_.GetWeakPtr(), frame.transport_frame_id(),
                                             ~~~~~ ^
1 error generated.

Please run chromium trybots as described at https://webrtc.org/contributing/#tryjobs-on-chromium-trybots before relanding.

Original issue's description:
> Make cricket::VideoFrame inherit webrtc::VideoFrame. Delete
> all methods but a few constructors. And similarly for the
> subclass cricket::WebRtcVideoFrame.
>
> TBR=tkchin@webrtc.org  # Added an include line
> BUG=webrtc:5682
>
> Committed: https://crrev.com/dda6ec008a0fc8d52e118814fb779032e8931968
> Cr-Commit-Position: refs/heads/master@{#14576}

TBR=perkj@webrtc.org,pthatcher@webrtc.org,pthatcher@chromium.org,tkchin@webrtc.org,nisse@webrtc.org
NOTRY=True
NOPRESUBMIT=True
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2402853002
Cr-Commit-Position: refs/heads/master@{#14583}
2016-10-09 05:21:40 +00:00
magjed
9e31cca8f4 iOS: Fix rotation bug for back camera
The rotation is currently incorrect for the back camera in landscape
mode. The reason is that device rotation needs to be reversed for the
back camera compared to the front camera. The camera sensor can also be
mounted with a specific orientation. So when front camera rotation goes
through: 0->90->180->270, back camera rotation goes in reverse:
180->90->0->270.

BUG=b/31984246,b/30651939

Review-Url: https://codereview.webrtc.org/2401033002
Cr-Commit-Position: refs/heads/master@{#14580}
2016-10-08 09:57:55 +00:00
nisse
dda6ec008a Make cricket::VideoFrame inherit webrtc::VideoFrame. Delete
all methods but a few constructors. And similarly for the
subclass cricket::WebRtcVideoFrame.

TBR=tkchin@webrtc.org  # Added an include line
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2315663002
Cr-Commit-Position: refs/heads/master@{#14576}
2016-10-07 15:16:59 +00:00
denicija
4f15ca5d74 Replaces the SessionPresets with AVCaptureDeviceFormats.
It eliminates the need for hardcoded enumerations of the supported presets.
Hopefully it enables us to support slightly bigger set of resolutions and make the code slightly cleaner.

BUG=webrtc:6355

Review-Url: https://codereview.webrtc.org/2349223002
Cr-Commit-Position: refs/heads/master@{#14547}
2016-10-06 09:32:10 +00:00
denicija
edbae5e0ac Remove Crit::Scope lock by using atomic bool property.
The clang static analyzer seems unable to resolve cpp locks in ObjC code.
As of current time, the clang analyzer has known limitations documented
http://clang.llvm.org/docs/ThreadSafetyAnalysis.html#known-limitations.
From the documentation: "The analysis currently does not do any checking
inside constructors or destructors.
In other words, every constructor and destructor is treated
as if it was annotated with NO_THREAD_SAFETY_ANALYSIS."
This is 'probably' why the analyzer is unable to resolve the lock when
used in ObjC land (the cpp works fine).
The lock can be removed by using atomic property instead.
It's not on performance critical path and we expect updates on just one queue and reads from others. That's why the thread assurance atomic properties bring is enough.
The CL removes rtc_sdk_peerconnection_objc_warnings_config as well as it's no longer needed.

BUG=webrtc:6308

Review-Url: https://codereview.webrtc.org/2372513004
Cr-Commit-Position: refs/heads/master@{#14450}
2016-09-30 07:21:28 +00:00
Kári Tristan Helgason
15e4ec334c Remove compat for iOS 7/8
BUG=None
R=magjed@webrtc.org, tkchin@webrtc.org

Review URL: https://codereview.webrtc.org/2382713002 .

Cr-Commit-Position: refs/heads/master@{#14448}
2016-09-30 06:56:44 +00:00
kwiberg
77eab70470 Enable the -Wundef warning for clang
NOPRESUBMIT=true
BUG=webrtc:6398

Review-Url: https://codereview.webrtc.org/2358993004
Cr-Commit-Position: refs/heads/master@{#14425}
2016-09-29 00:42:08 +00:00
nisse
edebf45712 Use I420Buffer rather than VideoFrameBuffer when writing pixels.
Prepares for deleting VideoFrameBuffer::MutableDataY{,U,V}. objc
changes extracted from cl https://codereview.webrtc.org/2278883002/.

BUG=webrtc:5921

Review-Url: https://codereview.webrtc.org/2346453002
Cr-Commit-Position: refs/heads/master@{#14236}
2016-09-15 14:20:48 +00:00
Magnus Jedvert
0bade0df3b AVFoundation Video Capturer: Remove thread jump when delivering frames
WebRTC no longer has any restriction on what thread frames should be
delivered on. One possible problem with this CL is that NV21->I420
conversion and scaling is done on the thread that delivers frames, which
might cause fps regressions.

R=nisse@webrtc.org, perkj@webrtc.org, tkchin@webrtc.org

Review URL: https://codereview.webrtc.org/2137503003 .

Cr-Commit-Position: refs/heads/master@{#14021}
2016-09-01 13:15:12 +00:00
magjed
2ab012c41e Implement CVO for iOS capturer
The rotation is currently always applied by AVFoundation by
reconfiguring the capture connection video orientation. This CL sets the
rotation field in the frame instead. This avoids the current flash in
the video when the device is rotated, and also avoids reconfiguring the
local encoder and remote decoder when the device is rotated.

BUG=b/30651939

Review-Url: https://codereview.webrtc.org/2271583003
Cr-Commit-Position: refs/heads/master@{#13916}
2016-08-25 10:25:13 +00:00
tkchin
118402520f Restart capture session if needed on active.
We've seen some cases of nonrecoverable runtime error when entering the foreground. This is a theoretical fix to see if we can restart after willEnterForeground in didBecomeActive instead.

NOTRY=True
BUG=

Review-Url: https://codereview.webrtc.org/2258583004
Cr-Commit-Position: refs/heads/master@{#13903}
2016-08-24 19:06:01 +00:00
kthelgason
4a85abb80e Add support for more resolutions on iOS/macOS
BUG=

Review-Url: https://codereview.webrtc.org/2231033002
Cr-Commit-Position: refs/heads/master@{#13828}
2016-08-19 08:24:49 +00:00
kthelgason
3e3ebe6937 remove unnecessary double allocation
BUG=

Review-Url: https://codereview.webrtc.org/2226933005
Cr-Commit-Position: refs/heads/master@{#13744}
2016-08-15 10:42:08 +00:00
tkchin
6ce738da31 Disable encoder scaling on iPhone4S.
Scaling causes us to work the CPU too much, which very quickly degrades quality. This causes us to at least behave better on good networks.

NOTRY=True
BUG=

Review-Url: https://codereview.webrtc.org/2205763002
Cr-Commit-Position: refs/heads/master@{#13630}
2016-08-03 19:57:18 +00:00
tkchin
9dfa249796 Use 352x288 for iPhone 4S.
NOTRY=True
BUG=

Review-Url: https://codereview.webrtc.org/2206773002
Cr-Commit-Position: refs/heads/master@{#13629}
2016-08-03 18:39:09 +00:00
Sergey Ulanov
19ee1e6eb1 Add cricket::VideoFrame::transport_frame_id() and set it to RTP timestamp.
Passing transport_frame_id() to VideoSink will allow to identify incoming video
frames, which will make it possible to correlate video frames on the
sender and on the receiver.

BUG=chromium:621691
R=mflodman@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2088953002 .

Cr-Commit-Position: refs/heads/master@{#13596}
2016-08-01 20:36:04 +00:00
haysc
7a11ae3c7c Print entire error object for capture session error handler
NOTRY=True
BUG=

Review-Url: https://codereview.webrtc.org/2189133002
Cr-Commit-Position: refs/heads/master@{#13580}
2016-07-29 19:03:58 +00:00
tkchin
d762910592 Use 15fps for iPhone 4S
NOTRY=True
BUG=

Review-Url: https://codereview.webrtc.org/2188883002
Cr-Commit-Position: refs/heads/master@{#13564}
2016-07-28 21:53:01 +00:00
magjed
39607c9e34 AVFoundationVideoCapturer: Output native frames instead of I420 frames
BUG=webrtc:4081

Review-Url: https://codereview.webrtc.org/2135953002
Cr-Commit-Position: refs/heads/master@{#13478}
2016-07-14 15:12:23 +00:00
Taylor Brandstetter
5d97a9a05b Adding more detail to MessageQueue::Dispatch logging.
Every message will now be traced with the location from which it was
posted, including function name, file and line number.

This CL also writes a normal LOG message when the dispatch took more
than a certain amount of time (currently 50ms).

This logging should help us identify messages that are taking
longer than expected to be dispatched.

R=pthatcher@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/2019423006 .

Cr-Commit-Position: refs/heads/master@{#13104}
2016-06-10 21:17:33 +00:00
adam.fedor
fc22e03eb8 Add AVFoundation video capture support to Mac objc SDK (based on iOS)
The AppRTCDemo app on Mac OSX does not show or send local video streams,
as ACFoundation capture session is not compiled in or implemented in
the appropriate places.  This is the first part of a two-part patch
that implements local capture on the Mac for AppRTCDemo

P.S. This is my first patch to WebRTC.   I didn't see any relevant tests, but I could write some if you can point me at a location. Also, I don't have access to the automated tests (I don't think)

BUG=webrtc:3417

Review-Url: https://codereview.webrtc.org/2046863004
Cr-Commit-Position: refs/heads/master@{#13080}
2016-06-09 00:24:44 +00:00
Zeke Chin
5251680ec3 Restart avfoundationvideocapturer on errors.
NOTRY=True

BUG=
R=haysc@webrtc.org

Review URL: https://codereview.webrtc.org/2028403003 .

Cr-Commit-Position: refs/heads/master@{#13041}
2016-06-03 18:57:22 +00:00
tkchin
9eeb6240c9 Build dynamic iOS SDK.
- Places most ObjC code into webrtc/sdk/objc instead.
- New gyp targets to build, strip and export symbols for dylib.
- Removes old script used to generate dylib.

BUG=

Review URL: https://codereview.webrtc.org/1903663002

Cr-Commit-Position: refs/heads/master@{#12524}
2016-04-27 08:54:27 +00:00