From 70cd086644f883d28eb1f01d859c2950abe08e47 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Henrik=20Bostr=C3=B6m?= Date: Fri, 21 May 2021 11:58:17 +0200 Subject: [PATCH] SEA: Only spawn multi-layered encoders if active layers > 1. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit With this CL, SimulcastEncoderAdapter no longer configures its encoder as multi-layered if we only have a single active layer. Instead we create a single single-layered encoder for that one and only active layer. When using VP8 SW encoder this means that LibvpxVp8Encoder is configured to only prepare a single video frame which avoids the cost of scaling down to layers that we do not send. (A multi-layered LibvpxVp8Encoder is required to scale even layers we don't encode.) When profiling this CL I found very small but measurable gains for representative downscale factors of 20.1 ms of 60 s profile. This is just 0.0335% CPU so it's not much, but skipping a downscale might be worth a lot more if we have to map/unmap buffers or do GPU round-trips in the future (which I have not measured). When downscaling to factors 4 and 2 due to libyuv having a "fast-path" for these (i.e. no adaptation active), zero difference was found for NV12. For I420 there was small regression of 16.1 ms (0.026% CPU) for this one edge-case. It's possible to work around this, but considering the tiny changes we're talking about, I really don't think it's worth the additional complexity. I'll file a bug on libyuv about scaling factors 2+2 vs 4 and leave it at that. Bug: webrtc:12603 Change-Id: Id462140c6a829cf6b460baae868e94243f477db3 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/219683 Commit-Queue: Henrik Boström Reviewed-by: Ilya Nikolaevskiy Cr-Commit-Position: refs/heads/master@{#34092} --- media/engine/simulcast_encoder_adapter.cc | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/media/engine/simulcast_encoder_adapter.cc b/media/engine/simulcast_encoder_adapter.cc index bee7b23c4e..3af022ab17 100644 --- a/media/engine/simulcast_encoder_adapter.cc +++ b/media/engine/simulcast_encoder_adapter.cc @@ -344,20 +344,24 @@ int SimulcastEncoderAdapter::InitEncode( // Two distinct scenarios: // * Singlecast (total_streams_count == 1) or simulcast with simulcast-capable - // underlaying encoder implementation. SEA operates in bypass mode: original - // settings are passed to the underlaying encoder, frame encode complete - // callback is not intercepted. + // underlaying encoder implementation if active_streams_count > 1. SEA + // operates in bypass mode: original settings are passed to the underlaying + // encoder, frame encode complete callback is not intercepted. // * Multi-encoder simulcast or singlecast if layers are deactivated - // (total_streams_count > 1 and active_streams_count >= 1). SEA creates - // N=active_streams_count encoders and configures each to produce a single - // stream. + // (active_streams_count >= 1). SEA creates N=active_streams_count encoders + // and configures each to produce a single stream. + int active_streams_count = CountActiveStreams(*inst); + // If we only have a single active layer it is better to create an encoder + // with only one configured layer than creating it with all-but-one disabled + // layers because that way we control scaling. + bool separate_encoders_needed = + !encoder_context->encoder().GetEncoderInfo().supports_simulcast || + active_streams_count == 1; // Singlecast or simulcast with simulcast-capable underlaying encoder. - if (total_streams_count_ == 1 || - encoder_context->encoder().GetEncoderInfo().supports_simulcast) { + if (total_streams_count_ == 1 || !separate_encoders_needed) { int ret = encoder_context->encoder().InitEncode(&codec_, settings); if (ret >= 0) { - int active_streams_count = CountActiveStreams(*inst); stream_contexts_.emplace_back( /*parent=*/nullptr, std::move(encoder_context), /*framerate_controller=*/nullptr, /*stream_idx=*/0, codec_.width,