87 Commits

Author SHA1 Message Date
sprang
16daaa5a40 Fixed incorrect handling of timestamps in video quality test
* Both timestamps must be unwrapped before comparing
* rtp timestamp delta must be subtracted before unwrapping

BUG=webrtc:5637, webrtc:5537

Review URL: https://codereview.webrtc.org/1774123003

Cr-Commit-Position: refs/heads/master@{#11926}
2016-03-09 09:40:53 +00:00
kwiberg
27f982bbcb Replace scoped_ptr with unique_ptr in webrtc/video/
BUG=webrtc:5520

Review URL: https://codereview.webrtc.org/1751903002

Cr-Commit-Position: refs/heads/master@{#11833}
2016-03-01 19:52:39 +00:00
Peter Boström
b1eaa8df0d Only average positive quality stats.
Removes addition of at least one zero sample in webrtc_perf_tests that
can skew stats differently depending on how often these stats are
updated. Unclear if this skewing is different between now and before.

BUG=chromium:585071, chromium:586216
R=sprang@google.com, sprang@webrtc.org

Review URL: https://codereview.webrtc.org/1727583003 .

Cr-Commit-Position: refs/heads/master@{#11720}
2016-02-23 16:30:55 +00:00
tnakamura
3123cbc489 Associate "score" attribute with SSIM so that future perf graphs know that an increase in SSIM should be considered an improvement.
BUG=chromium:582009

Review URL: https://codereview.webrtc.org/1688503003

Cr-Commit-Position: refs/heads/master@{#11558}
2016-02-10 19:21:58 +00:00
Peter Boström
e449915455 Measure encoding time on encode callbacks.
Permits measuring encoding time even when performed on another thread,
typically for hardware encoding, instead of assuming that encoding is
blocking the calling thread.

Permitted encoding time is increased for hardware encoders since they
can be timed to keep 30fps, for instance, without indicating overload.

Merges EncodingTimeObserver into EncodedFrameObserver to have one post-encode
callback.

BUG=webrtc:5042, webrtc:5132
R=asapersson@webrtc.org, mflodman@webrtc.org

Review URL: https://codereview.webrtc.org/1569853002 .

Cr-Commit-Position: refs/heads/master@{#11499}
2016-02-05 10:13:41 +00:00
Stefan Holmer
10880011d9 Support multiple rtx codecs.
Adds negotiation of rtx codecs for red and vp9. To keep backwards
compatibility with older Chrome versions, this change includes two
hacks:
1. Red packets will be retransmitted over the rtx codec associated with
   vp8 if no rtx codec is associated with red. This is how Chrome does
   it today and ensures that we still can send red over rtx to older
   versions.

2. If rtx packets associated with the media codec (vp8/vp9 etc) are
   received and red has been negotiated, we will assume that the sender
   incorrectly has packetized red inside the rtx header associated with
   media. We will therefore restore it with the red payload type
   instead, which ensures that we can still receive rtx associated with
   red from old versions.

Offering multiple rtx codecs to older versions should not be a problem
since old versions themselves only try to negotiate rtx for vp8.

R=pbos@webrtc.org
TBR=mflodman@webrtc.org
BUG=webrtc:4024
TEST=Verified by running apprtc and emulating packet loss between Chrome with and without the patch.

Review URL: https://codereview.webrtc.org/1649493004 .

Cr-Commit-Position: refs/heads/master@{#11472}
2016-02-03 12:30:10 +00:00
sprang
e1f2f1fbb8 Unwrap timestamps in VideoAnalyzer
We have seen an instance of flakiness of the perf tests where it looked
like timestamp wraparound could be an issue. Better safe...

BUG=

Review URL: https://codereview.webrtc.org/1645463002

Cr-Commit-Position: refs/heads/master@{#11440}
2016-02-01 10:05:00 +00:00
hbos
bab934bffe H.264 video codec support using OpenH264 (http://www.openh264.org/) for encoding and FFmpeg (https://www.ffmpeg.org/) for decoding.
It works on all platforms except Android and iOS (FFmpeg limitation).

Implemented behind compile time flags, off by default.
The plan is to have it enabled in Chrome (see bug), but not in Chromium/webrtc by default.

Flags to turn it on:
- rtc_use_h264 = true
- ffmpeg_branding = "Chrome" (or other brand that includes H.264 decoder)

Tests using H264:
- video_loopback --codec=H264
- screenshare_loopback --codec=H264
- video_engine_tests (EndToEndTest.SendsAndReceivesH264)

NOTRY=True
BUG=500605, 468365
BUG=https://bugs.chromium.org/p/webrtc/issues/detail?id=5424

Review URL: https://codereview.webrtc.org/1306813009

Cr-Commit-Position: refs/heads/master@{#11390}
2016-01-27 09:36:07 +00:00
Peter Boström
dd45eb6801 Remove use-after-free when quality tests stall.
Reduces TSan warnings when running screenshare FullStack tests.

BUG=
R=sprang@webrtc.org

Review URL: https://codereview.webrtc.org/1601033004 .

Cr-Commit-Position: refs/heads/master@{#11305}
2016-01-19 14:22:45 +00:00
Stefan Holmer
9fea80f50d Add audio streams to CallTest and a first A/V call test.
Add audio send and receive streams to CallTest and call the necessary voice engine APIs for the streams to be usable. Verifies the implementation by adding a simple test which monitors outgoing packets and checks that both audio and video is being sent with transport sequence numbers.

Audio streams are using a fake audio device with file input.

The CallTest implementation is to a big degree based on call_perf_tests.cc and should in the future replace a lot of that code.

R=pbos@webrtc.org
TBR=kjellander@webrtc.org

BUG=webrtc:5263

Review URL: https://codereview.webrtc.org/1542653002 .

Cr-Commit-Position: refs/heads/master@{#11171}
2016-01-07 16:43:31 +00:00
danilchap
f6975f4613 [rtp_rtcp] Lint errors cleaned from rtp_utility
R=åsapersson
BUG=webrtc:5277

Review URL: https://codereview.webrtc.org/1539423003

Cr-Commit-Position: refs/heads/master@{#11131}
2015-12-28 18:18:52 +00:00
stefan
ff483617a4 Step 1 to prepare call_test.* for combined audio/video tests.
Also move (and clean up includes) rampup_tests.* to webrtc/call in preparation for combined audio/video ramp-up tests.

No functional changes.

BUG=webrtc:5263

Review URL: https://codereview.webrtc.org/1537273003

Cr-Commit-Position: refs/heads/master@{#11101}
2015-12-21 11:14:05 +00:00
Peter Boström
5811a39f14 Replace EventWrapper in video/, test/ and call/.
Makes use of rtc::Event which is simpler and can be used without
allocating additional objects on the heap.

Does not modify test/channel_transport/.

BUG=
R=mflodman@webrtc.org

Review URL: https://codereview.webrtc.org/1487893004 .

Cr-Commit-Position: refs/heads/master@{#10968}
2015-12-10 12:03:00 +00:00
mflodman
d1590b2571 Lint clean video/ and add lint presubmit check.
BUG=webrtc:5316

Review URL: https://codereview.webrtc.org/1507643004

Cr-Commit-Position: refs/heads/master@{#10953}
2015-12-09 15:08:05 +00:00
Peter Boström
8c38e8b9b9 Clean up PlatformThread.
* Move PlatformThread to rtc::.
* Remove ::CreateThread factory method.
* Make non-scoped_ptr from a lot of invocations.
* Make Start/Stop void.
* Remove rtc::Thread priorities, which were unused and would collide.
* Add ::IsRunning() to PlatformThread.

BUG=
R=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1476453002 .

Cr-Commit-Position: refs/heads/master@{#10812}
2015-11-26 16:45:57 +00:00
pbos
12411ef40e Move ThreadWrapper to ProcessThread in base.
Also removes all virtual methods. Permits using a thread from
rtc_base_approved (namely event tracing).

BUG=webrtc:5158
R=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1469013002

Cr-Commit-Position: refs/heads/master@{#10760}
2015-11-23 22:48:01 +00:00
sprang
ee37de3c13 Add screenshare perf tests with lossy links
This is a re-land of https://codereview.webrtc.org/1409513005/
Fingers crossed, the problems previously seen have been resolved by
https://codereview.webrtc.org/1412233003/

BUG=

Review URL: https://codereview.webrtc.org/1409993011

Cr-Commit-Position: refs/heads/master@{#10751}
2015-11-23 14:10:28 +00:00
stefan
43edf0ffb9 Require negotiation to send transport cc feedback over RTCP.
BUG=4312

Review URL: https://codereview.webrtc.org/1452883002

Cr-Commit-Position: refs/heads/master@{#10735}
2015-11-21 02:05:53 +00:00
Henrik Kjellander
ff761fba82 modules: more interface -> include renames
This changes the following module directories:
* webrtc/modules/audio_conference_mixer/interface
* webrtc/modules/interface
* webrtc/modules/media_file/interface
* webrtc/modules/rtp_rtcp/interface
* webrtc/modules/utility/interface

To avoid breaking downstream, I followed this recipe:
1. Copy the interface dir to a new sibling directory: include
2. Update the header guards in the include directory to match the style guide.
3. Update the header guards in the interface directory to match the ones in include. This is required to avoid getting redefinitions in the not-yet-updated downstream code.
4. Add a pragma warning in the header files in the interface dir. Example:
#pragma message("WARNING: webrtc/modules/interface is DEPRECATED; "
                "use webrtc/modules/include")
5. Search for all source references to webrtc/modules/interface and update them to webrtc/modules/include (*.c*,*.h,*.mm,*.S)
6. Update all GYP+GN files. This required manual inspection since many subdirectories of webrtc/modules referenced the interface dir using ../interface etc(*.gyp*,*.gn*)

BUG=5095
TESTED=Passing compile-trybots with --clobber flag:
git cl try --clobber --bot=win_compile_rel --bot=linux_compile_rel --bot=android_compile_rel --bot=mac_compile_rel --bot=ios_rel -m tryserver.webrtc

R=stefan@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1417683006 .

Cr-Commit-Position: refs/heads/master@{#10500}
2015-11-04 07:32:04 +00:00
sprang
ce4aef16ee Adding support for simulcast and spatial layers into VideoQualityTest
This is a re-land of https://codereview.webrtc.org/1353263005/
which was reverted because of perf-regressions. Changes since that CL:

* Change LayerFilteringTransport to send a padding packet instead of
  dropping it for data that should be filtered out. This prevents
  confusion due to changed sequence numbers.

* Changed timing of stats poller thread in VideoAnalyzer. Startup was
  racy wrt initializion of send_stream_.

* Minor formatting issues.

PERF NOTE: This change will affect some performance numbers slightly.
In particular, {encode_frame_rate, encode_time_ms,
encode_usage_percent, media_bitrate_bps} will change due to timing
of the measurements.

BUG=
R=pbos@webrtc.org
TBR=mflodman@webrtc.org

Review URL: https://codereview.webrtc.org/1412233003

Cr-Commit-Position: refs/heads/master@{#10483}
2015-11-02 15:23:24 +00:00
Stefan Holmer
1295297153 Register header extensions in RtpRtcpObserver to avoid log spam.
BUG=webrtc:5118
R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/1416783006 .

Cr-Commit-Position: refs/heads/master@{#10450}
2015-10-29 14:13:35 +00:00
Henrik Kjellander
98f53510b2 system_wrappers: rename interface -> include
BUG=webrtc:5095
R=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1413333002 .

Cr-Commit-Position: refs/heads/master@{#10438}
2015-10-28 17:17:50 +00:00
stefan
f116bd0d7a Call OnSentPacket for all packets sent in the test framework.
Required a bit of refactoring to make it possible to pass a Call to DirectTransport on construction. This also lead to me having to remove the shared lock between PacketTransport and RtpRtcpObserver. Now RtpRtcpObserver has a SetTransports method instead of a SetReceivers method.

BUG=webrtc:4173

Review URL: https://codereview.webrtc.org/1419193002

Cr-Commit-Position: refs/heads/master@{#10430}
2015-10-27 15:29:47 +00:00
sprang
7a975f75e7 Revert of Adding support for simulcast and spatial layers into VideoQualityTest (patchset #10 id:180001 of https://codereview.webrtc.org/1353263005/ )
Reason for revert:
Temporarily reverting as this causes some issues with perf tests. Especially tests with packet loss no longer works.

Original issue's description:
> Adding support for simulcast and spatial layers into VideoQualityTest
>
> The CL includes several changes:
> - Adding flags describing the streams and spatial layers.
> - Reorganizing the order of the flags, to make them easier to maintain.
> - Adding a member .params_ to VideoQualityAnalyzer.
>     (instead of passing it to every member function manually)
> - Updating VideoAnalyzer to support simulcast.
>     (select appropriate ssrc and fix timestamps which are sometimes increased by 1)
> - VP9EncoderImpl already had code for automatic calculation of bitrate for each layer.
>     Changing to first read bitrates and resolution ratios from the flags, if specified.
>     If not specified, reverting to the old code are setting the values automatically.
> - Changing the parameters in LayerFilteringTransport, replacing
>     xx_discard_thresholds with selected_xx, to make it easier to use for the end user.
>
> Committed: https://crrev.com/87f83a9a27d657731ccb54025bc04ccad0da136e
> Cr-Commit-Position: refs/heads/master@{#10215}

TBR=pbos@webrtc.org,mflodman@webrtc.org,ivica@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.webrtc.org/1397363002

Cr-Commit-Position: refs/heads/master@{#10252}
2015-10-12 13:33:24 +00:00
ivica
87f83a9a27 Adding support for simulcast and spatial layers into VideoQualityTest
The CL includes several changes:
- Adding flags describing the streams and spatial layers.
- Reorganizing the order of the flags, to make them easier to maintain.
- Adding a member .params_ to VideoQualityAnalyzer.
    (instead of passing it to every member function manually)
- Updating VideoAnalyzer to support simulcast.
    (select appropriate ssrc and fix timestamps which are sometimes increased by 1)
- VP9EncoderImpl already had code for automatic calculation of bitrate for each layer.
    Changing to first read bitrates and resolution ratios from the flags, if specified.
    If not specified, reverting to the old code are setting the values automatically.
- Changing the parameters in LayerFilteringTransport, replacing
    xx_discard_thresholds with selected_xx, to make it easier to use for the end user.

Review URL: https://codereview.webrtc.org/1353263005

Cr-Commit-Position: refs/heads/master@{#10215}
2015-10-08 12:13:37 +00:00
ivica
c1cc854d54 Fixing perf regression caused by refactoring full stack tests
Calling CreateCapturer after CreateStreams. The wrong order of calling those methods seems to have caused perf regressions.

Testing has been done here: https://codereview.webrtc.org/1371113004/

BUG=chromium:534220

Review URL: https://codereview.webrtc.org/1394463002

Cr-Commit-Position: refs/heads/master@{#10212}
2015-10-08 10:44:11 +00:00
ivica
8d15bd6dab Reland of Collecting encode_time_ms for each frame (patchset #1 id:1 of https://codereview.webrtc.org/1383283005/ )
Reason for revert:
The reverted commit didn't affect the tests, but the one before: https://codereview.webrtc.org/1385563005/

I've run the test that was failing (EndToEndTest.AssignsTransportSequenceNumbers) locally multiple times, and it works fine (finishes successfully in 150-170ms).

Original issue's description:
> Revert of Collecting encode_time_ms for each frame (patchset #13 id:220001 of https://codereview.webrtc.org/1374233002/ )
>
> Reason for revert:
> Breaks EndToEndTest.AssignsTransportSequenceNumbers in video_engine_tests
> on several bots:
> http://build.chromium.org/p/client.webrtc/builders/Linux64%20Debug/builds/5507
> http://build.chromium.org/p/client.webrtc/builders/Mac64%20Debug/builds/4815
> http://build.chromium.org/p/client.webrtc/builders/Win%20SyzyASan/builds/3272
> http://build.chromium.org/p/client.webrtc/builders/Linux%20Memcheck/builds/4414
>
> It seems very unfortunate that it breaks on _exactly_ the bot configs that aren't covered by the CQ trybots.
>
> Original issue's description:
> > Collecting encode_time_ms for each frame.
> >
> > Also, in Sample struct, replacing double with the original type.
> > It makes more sense to save the original data as truthful as possible, and then
> > convert it to double later if necessary (in the plot script).
> >
> > Committed: https://crrev.com/092b13384e57b33e2003d9736dfa1f491e76f938
> > Cr-Commit-Position: refs/heads/master@{#10184}
>
> TBR=sprang@webrtc.org,pbos@webrtc.org,mflodman@webrtc.org,asapersson@webrtc.org,ivica@webrtc.org
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
>
> Committed: https://crrev.com/810447972425e890bc7911af27f894b86e9b7e6f
> Cr-Commit-Position: refs/heads/master@{#10185}

TBR=sprang@webrtc.org,pbos@webrtc.org,mflodman@webrtc.org,asapersson@webrtc.org,kjellander@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.webrtc.org/1390163002

Cr-Commit-Position: refs/heads/master@{#10195}
2015-10-07 09:43:25 +00:00
kjellander
8104479724 Revert of Collecting encode_time_ms for each frame (patchset #13 id:220001 of https://codereview.webrtc.org/1374233002/ )
Reason for revert:
Breaks EndToEndTest.AssignsTransportSequenceNumbers in video_engine_tests
on several bots:
http://build.chromium.org/p/client.webrtc/builders/Linux64%20Debug/builds/5507
http://build.chromium.org/p/client.webrtc/builders/Mac64%20Debug/builds/4815
http://build.chromium.org/p/client.webrtc/builders/Win%20SyzyASan/builds/3272
http://build.chromium.org/p/client.webrtc/builders/Linux%20Memcheck/builds/4414

It seems very unfortunate that it breaks on _exactly_ the bot configs that aren't covered by the CQ trybots.

Original issue's description:
> Collecting encode_time_ms for each frame.
>
> Also, in Sample struct, replacing double with the original type.
> It makes more sense to save the original data as truthful as possible, and then
> convert it to double later if necessary (in the plot script).
>
> Committed: https://crrev.com/092b13384e57b33e2003d9736dfa1f491e76f938
> Cr-Commit-Position: refs/heads/master@{#10184}

TBR=sprang@webrtc.org,pbos@webrtc.org,mflodman@webrtc.org,asapersson@webrtc.org,ivica@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.webrtc.org/1383283005

Cr-Commit-Position: refs/heads/master@{#10185}
2015-10-06 18:34:14 +00:00
ivica
092b13384e Collecting encode_time_ms for each frame.
Also, in Sample struct, replacing double with the original type.
It makes more sense to save the original data as truthful as possible, and then
convert it to double later if necessary (in the plot script).

Review URL: https://codereview.webrtc.org/1374233002

Cr-Commit-Position: refs/heads/master@{#10184}
2015-10-06 14:13:50 +00:00
stefan
1d8a506405 Add a PacketOptions struct to webrtc::Transport.
This allows us to pass packet meta data, such as transport sequence
number, to libjingle and further down to the socket implementation. A
similar struct already exist in libjingle, see rtc::PacketOptions in asyncpacketsocket.h.

BUG=4173

Review URL: https://codereview.webrtc.org/1376673004

Cr-Commit-Position: refs/heads/master@{#10144}
2015-10-02 10:39:40 +00:00
pbos
2d566686a2 Unify Transport and newapi::Transport interfaces.
BUG=webrtc:1695
R=stefan@webrtc.org
TBR=mflodman@webrtc.org

Review URL: https://codereview.webrtc.org/1369263002

Cr-Commit-Position: refs/heads/master@{#10096}
2015-09-28 16:59:36 +00:00
Erik Språng
6b8d355168 Reland "Wire up send-side bandwidth estimation."
Revert was patchset #8 id:140001 of https://codereview.webrtc.org/1338203003/

The culprit was RTC_DCHECK(poller_thread_->Start()); in rampup_test.cc

BUG=webrtc:4173
R=stefan@webrtc.org

Review URL: https://codereview.webrtc.org/1362303002 .

Cr-Commit-Position: refs/heads/master@{#10052}
2015-09-24 13:07:17 +00:00
Erik Språng
c9bbeb0354 Revert of Wire up send-side bandwidth estimation. (patchset #8 id:140001 of https://codereview.webrtc.org/1338203003/ )
Reason for revert:
Breaking some Android bots.
https://chromegw.corp.google.com/i/client.webrtc/builders/Android32%20Tests%20%28L%20Nexus5%29

Original issue's description:
> Wire up send-side bandwidth estimation.
>
> BUG=webrtc:4173
>
> Committed: https://crrev.com/ef165eefc79cf28bb67779afe303cc2365885547
> Cr-Commit-Position: refs/heads/master@{#10012}

TBR=stefan@webrtc.org, kjellander@webrtc.org
NOPRESUBMIT=false
NOTREECHECKS=false
NOTRY=false
BUG=webrtc:4173

Review URL: https://codereview.webrtc.org/1362923002 .

Cr-Commit-Position: refs/heads/master@{#10029}
2015-09-23 11:52:01 +00:00
ivica
2d4e6c5d9d Fixing camera capture for video_loopback
In the middle of refactoring, I replaced the VideoCapturer with
FrameGeneratorCapturer, to reuse the code, and with that disabled the camera.
Now adding capturer_ element to VideoQualityTest and ignoring
frame_generator_capturer_ from the parent class test::CallTest.

Review URL: https://codereview.webrtc.org/1356933005

Cr-Commit-Position: refs/heads/master@{#10023}
2015-09-23 08:57:13 +00:00
ivica
d4818e7304 Using static frame generator when no scrolling
In screensharing full stack tests, instead of using YuvFileGenerator by default
when no scrolling is used, I always used ScrollingImageFileGenerator.
That possibly slowed down the test a little bit, at least for the slowed
devices, as it unnecessarily copied few MBs per frame.

BUG=chromium:534220

Review URL: https://codereview.webrtc.org/1359783002

Cr-Commit-Position: refs/heads/master@{#10014}
2015-09-22 12:47:34 +00:00
sprang
ef165eefc7 Wire up send-side bandwidth estimation.
BUG=webrtc:4173

Review URL: https://codereview.webrtc.org/1338203003

Cr-Commit-Position: refs/heads/master@{#10012}
2015-09-22 12:10:58 +00:00
ivica
5d6a06c1d2 Refactoring full stack and loopback tests
Refactoring full stack, video and screenshare tests to use the same code basis
for parametrization and initialization. This patch is done on top of recently
commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but
virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer
in full stack, except moving it to video_quality_test.cc.
Also, full_stack_samples.cc (build target) was removed and replaced with
-output_filename and -duration cmdline arguments in video_loopback and
screenshare_loopback.

The important things to review:
- video_quality_test.h
    Is the structure of Params good? (examples of usage can be found in
    full_stack.cc, video_loopback.cc and screenshare_loopback.cc)
- video_quality_test.cc
    Is the initialization correct? The case for using Analyzer and using local
    renderer are different, can they be further merged?
- webrtc_tests.gypi

Reproducing the different bitrate settings the full stack and loopback tests had
was a little bit tricky. To support both simultaneously, I added BitrateConfig
to the Params struct, as well as separate start_bitrate and target_bitrate flags
for loopback tests.

Note: Side-by-side diff for video_quality_test.cc compares that file directly
with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible.

Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold
args to loopback tests. This was removed here. Support for streams and SVC
will be added in a CL following this one.

Review URL: https://codereview.webrtc.org/1308403003

Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 12:30:30 +00:00