Skip to content

Commit

Permalink
[Impeller] Implement framebuffer-fetch via subpasses in Vulkan withou…
Browse files Browse the repository at this point in the history
…t extensions. (flutter#50154)

* Subpasses are not exposed in the HAL and the need for subpasses in Vulkan can
  be determined based on the presence and use of input-attachments in the
  shaders. This information is already reflected by the compiler. Because of
  this, all references to subpasses have been removed from APIs above the HAL.
* `RenderPassBuilderVK` is a lightweight object used to generate render passes
  to use either with the pipelines (compat, base, or per-subpass) or during
  rendering along with the framebuffer. Using the builder also sets up the
  right subpass dependencies. As long as the builder contains compatible
  attachments and subpass counts, different subpasses stamped by the builder
  (via the `Build` method) are guaranteed to be compatible per the rules in the
  spec.
* Pass attachments are now in the `eGeneral` layout. There was no observable
  difference in performance when manually inserting the right transitions.
  Except, a lot of transitions needed to be inserted. If we need it, we can add
  it back in short order. I wouldn't be averse to adding it if reviewers
  insist.
* Additional pipeline state objects need not be created as the sub-pass
  self-dependencies are sufficient to setup the render-pass.
* Speaking of the `rasterization_order_attachment_access` extension, its use has
  been removed in this patch. I am prototyping adding it back to measure the
  overhead introduced by manual subpass management. If the overhead is
  measurable, we can use the extension on devices that have it as an added
  optimization.
* The complexity of command encoding remains linear (to the number of commands)
  per pass.
* This patch only works on a single color attachment being used as an input
  attachment. While this is sufficient for current use cases, the Metal
  implementation is significantly more capable since the multiple attachments
  and attachment types (depth) are already supported. Rounding out support for
  this is in progress.
  • Loading branch information
chinmaygarde authored Jan 31, 2024
1 parent c4247c5 commit f4fbabf
Show file tree
Hide file tree
Showing 36 changed files with 1,195 additions and 914 deletions.
4 changes: 4 additions & 0 deletions ci/licenses_golden/licenses_flutter
Original file line number Diff line number Diff line change
Expand Up @@ -5478,6 +5478,8 @@ ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/pipeline_vk.cc + ../..
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/pipeline_vk.h + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/queue_vk.cc + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/queue_vk.h + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_builder_vk.cc + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_builder_vk.h + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_vk.cc + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_vk.h + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/impeller/renderer/backend/vulkan/resource_manager_vk.cc + ../../../flutter/LICENSE
Expand Down Expand Up @@ -8335,6 +8337,8 @@ FILE: ../../../flutter/impeller/renderer/backend/vulkan/pipeline_vk.cc
FILE: ../../../flutter/impeller/renderer/backend/vulkan/pipeline_vk.h
FILE: ../../../flutter/impeller/renderer/backend/vulkan/queue_vk.cc
FILE: ../../../flutter/impeller/renderer/backend/vulkan/queue_vk.h
FILE: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_builder_vk.cc
FILE: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_builder_vk.h
FILE: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_vk.cc
FILE: ../../../flutter/impeller/renderer/backend/vulkan/render_pass_vk.h
FILE: ../../../flutter/impeller/renderer/backend/vulkan/resource_manager_vk.cc
Expand Down
45 changes: 15 additions & 30 deletions impeller/entity/contents/content_context.cc
Original file line number Diff line number Diff line change
Expand Up @@ -240,64 +240,49 @@ ContentContext::ContentContext(
if (context_->GetCapabilities()->SupportsFramebufferFetch()) {
framebuffer_blend_color_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kColor), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kColor), supports_decal});
framebuffer_blend_colorburn_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kColorBurn), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kColorBurn), supports_decal});
framebuffer_blend_colordodge_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kColorDodge), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kColorDodge), supports_decal});
framebuffer_blend_darken_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kDarken), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kDarken), supports_decal});
framebuffer_blend_difference_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kDifference), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kDifference), supports_decal});
framebuffer_blend_exclusion_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kExclusion), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kExclusion), supports_decal});
framebuffer_blend_hardlight_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kHardLight), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kHardLight), supports_decal});
framebuffer_blend_hue_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kHue), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kHue), supports_decal});
framebuffer_blend_lighten_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kLighten), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kLighten), supports_decal});
framebuffer_blend_luminosity_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kLuminosity), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kLuminosity), supports_decal});
framebuffer_blend_multiply_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kMultiply), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kMultiply), supports_decal});
framebuffer_blend_overlay_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kOverlay), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kOverlay), supports_decal});
framebuffer_blend_saturation_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kSaturation), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kSaturation), supports_decal});
framebuffer_blend_screen_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kScreen), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kScreen), supports_decal});
framebuffer_blend_softlight_pipelines_.CreateDefault(
*context_, options_trianglestrip,
{static_cast<Scalar>(BlendSelectValues::kSoftLight), supports_decal},
UseSubpassInput::kYes);
{static_cast<Scalar>(BlendSelectValues::kSoftLight), supports_decal});
}

blend_color_pipelines_.CreateDefault(
Expand Down
4 changes: 1 addition & 3 deletions impeller/entity/contents/content_context.h
Original file line number Diff line number Diff line change
Expand Up @@ -826,15 +826,13 @@ class ContentContext {

void CreateDefault(const Context& context,
const ContentContextOptions& options,
const std::initializer_list<Scalar>& constants = {},
UseSubpassInput subpass_input = UseSubpassInput::kNo) {
const std::initializer_list<Scalar>& constants = {}) {
auto desc =
PipelineT::Builder::MakeDefaultPipelineDescriptor(context, constants);
if (!desc.has_value()) {
VALIDATION_LOG << "Failed to create default pipeline.";
return;
}
desc->SetUseSubpassInput(subpass_input);
options.ApplyToPipelineDescriptor(*desc);
SetDefault(options, std::make_unique<PipelineT>(context, desc));
}
Expand Down
33 changes: 0 additions & 33 deletions impeller/entity/entity_unittests.cc
Original file line number Diff line number Diff line change
Expand Up @@ -2689,39 +2689,6 @@ TEST_P(EntityTest, DecalSpecializationAppliedToMorphologyFilter) {
expected_constants);
}

TEST_P(EntityTest, FramebufferFetchPipelinesDeclareUsage) {
auto content_context =
ContentContext(GetContext(), TypographerContextSkia::Make());
if (!content_context.GetDeviceCapabilities().SupportsFramebufferFetch()) {
GTEST_SKIP() << "Framebuffer fetch not supported.";
}

ContentContextOptions options;
options.color_attachment_pixel_format = PixelFormat::kR8G8B8A8UNormInt;
auto color_burn =
content_context.GetFramebufferBlendColorBurnPipeline(options);

EXPECT_TRUE(color_burn->GetDescriptor().UsesSubpassInput());
}

TEST_P(EntityTest, PipelineDescriptorEqAndHash) {
auto desc_1 = std::make_shared<PipelineDescriptor>();
auto desc_2 = std::make_shared<PipelineDescriptor>();

EXPECT_TRUE(desc_1->IsEqual(*desc_2));
EXPECT_EQ(desc_1->GetHash(), desc_2->GetHash());

desc_1->SetUseSubpassInput(UseSubpassInput::kYes);

EXPECT_FALSE(desc_1->IsEqual(*desc_2));
EXPECT_NE(desc_1->GetHash(), desc_2->GetHash());

desc_2->SetUseSubpassInput(UseSubpassInput::kYes);

EXPECT_TRUE(desc_1->IsEqual(*desc_2));
EXPECT_EQ(desc_1->GetHash(), desc_2->GetHash());
}

// This doesn't really tell you if the hashes will have frequent
// collisions, but since this type is only used to hash a bounded
// set of options, we can just compare benchmarks.
Expand Down
14 changes: 11 additions & 3 deletions impeller/fixtures/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ import("//flutter/testing/testing.gni")
impeller_shaders("shader_fixtures") {
name = "fixtures"

# 2.3 adds support for framebuffer fetch in Metal.
metal_version = "2.3"

# Not analyzing because they are not performance critical, and mipmap uses
# textureLod, which uses an extension that malioc does not support.
analyze = false
Expand All @@ -16,8 +19,9 @@ impeller_shaders("shader_fixtures") {
"array.vert",
"box_fade.frag",
"box_fade.vert",
"colors.vert",
"colors.frag",
"colors.vert",
"half.frag",
"impeller.frag",
"impeller.vert",
"inactive_uniforms.frag",
Expand All @@ -27,12 +31,16 @@ impeller_shaders("shader_fixtures") {
"mipmaps.frag",
"mipmaps.vert",
"sample.comp",
"sepia.frag",
"sepia.vert",
"simple.vert",
"stage1.comp",
"stage2.comp",
"simple.vert",
"swizzle.frag",
"test_texture.frag",
"test_texture.vert",
"half.frag",
"texture.frag",
"texture.vert",
]

if (impeller_enable_opengles) {
Expand Down
17 changes: 17 additions & 0 deletions impeller/fixtures/sepia.frag
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

out vec4 frag_color;

layout(input_attachment_index = 0) uniform subpassInputMS subpass_input;

void main() {
// https://github.com/chinmaygarde/merle/blob/3eecb311ac8862c41f0c53a5d9b360be923142bb/src/texture.cc#L195
const mat4 sepia_matrix = mat4(0.3588, 0.2990, 0.2392, 0.0000, //
0.7044, 0.5870, 0.4696, 0.0000, //
0.1368, 0.1140, 0.0912, 0.0000, //
0.0000, 0.0000, 0.0000, 1.0000 //
);
frag_color = sepia_matrix * subpassLoad(subpass_input, 0);
}
14 changes: 14 additions & 0 deletions impeller/fixtures/sepia.vert
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

uniform UniformBuffer {
mat4 mvp;
}
uniform_buffer;

in vec3 vertex_position;

void main() {
gl_Position = uniform_buffer.mvp * vec4(vertex_position, 1.0);
}
11 changes: 11 additions & 0 deletions impeller/fixtures/swizzle.frag
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

out vec4 frag_color;

layout(input_attachment_index = 0) uniform subpassInputMS subpass_input;

void main() {
frag_color = subpassLoad(subpass_input, 0).gbra;
}
13 changes: 13 additions & 0 deletions impeller/fixtures/texture.frag
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

in vec2 interporlated_texture_coordinates;

out vec4 frag_color;

uniform sampler2D texture_contents;

void main() {
frag_color = texture(texture_contents, interporlated_texture_coordinates);
}
18 changes: 18 additions & 0 deletions impeller/fixtures/texture.vert
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

uniform UniformBuffer {
mat4 mvp;
}
uniform_buffer;

in vec3 vertex_position;
in vec2 texture_coordinates;

out vec2 interpolated_texture_coordinates;

void main() {
gl_Position = uniform_buffer.mvp * vec4(vertex_position, 1.0);
interpolated_texture_coordinates = texture_coordinates;
}
2 changes: 2 additions & 0 deletions impeller/renderer/backend/vulkan/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ impeller_component("vulkan") {
"pipeline_vk.h",
"queue_vk.cc",
"queue_vk.h",
"render_pass_builder_vk.cc",
"render_pass_builder_vk.h",
"render_pass_vk.cc",
"render_pass_vk.h",
"resource_manager_vk.cc",
Expand Down
20 changes: 7 additions & 13 deletions impeller/renderer/backend/vulkan/allocator_vk.cc
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,6 @@ AllocatorVK::AllocatorVK(std::weak_ptr<Context> context,
allocator_.reset(allocator);
supports_memoryless_textures_ =
capabilities.SupportsDeviceTransientTextures();
supports_framebuffer_fetch_ = capabilities.SupportsFramebufferFetch();
is_valid_ = true;
}

Expand All @@ -167,8 +166,7 @@ static constexpr vk::ImageUsageFlags ToVKImageUsageFlags(
PixelFormat format,
TextureUsageMask usage,
StorageMode mode,
bool supports_memoryless_textures,
bool supports_framebuffer_fetch) {
bool supports_memoryless_textures) {
vk::ImageUsageFlags vk_usage;

switch (mode) {
Expand All @@ -188,9 +186,7 @@ static constexpr vk::ImageUsageFlags ToVKImageUsageFlags(
} else {
vk_usage |= vk::ImageUsageFlagBits::eColorAttachment;
}
if (supports_framebuffer_fetch) {
vk_usage |= vk::ImageUsageFlagBits::eInputAttachment;
}
vk_usage |= vk::ImageUsageFlagBits::eInputAttachment;
}

if (usage & static_cast<TextureUsageMask>(TextureUsage::kShaderRead)) {
Expand Down Expand Up @@ -267,8 +263,7 @@ class AllocatedTextureSourceVK final : public TextureSourceVK {
const TextureDescriptor& desc,
VmaAllocator allocator,
vk::Device device,
bool supports_memoryless_textures,
bool supports_framebuffer_fetch)
bool supports_memoryless_textures)
: TextureSourceVK(desc), resource_(std::move(resource_manager)) {
FML_DCHECK(desc.format != PixelFormat::kUnknown);
vk::ImageCreateInfo image_info;
Expand All @@ -285,9 +280,9 @@ class AllocatedTextureSourceVK final : public TextureSourceVK {
image_info.arrayLayers = ToArrayLayerCount(desc.type);
image_info.tiling = vk::ImageTiling::eOptimal;
image_info.initialLayout = vk::ImageLayout::eUndefined;
image_info.usage = ToVKImageUsageFlags(
desc.format, desc.usage, desc.storage_mode,
supports_memoryless_textures, supports_framebuffer_fetch);
image_info.usage =
ToVKImageUsageFlags(desc.format, desc.usage, desc.storage_mode,
supports_memoryless_textures);
image_info.sharingMode = vk::SharingMode::eExclusive;

VmaAllocationCreateInfo alloc_nfo = {};
Expand Down Expand Up @@ -433,8 +428,7 @@ std::shared_ptr<Texture> AllocatorVK::OnCreateTexture(
desc, //
allocator_.get(), //
device_holder->GetDevice(), //
supports_memoryless_textures_, //
supports_framebuffer_fetch_ //
supports_memoryless_textures_ //
);
if (!source->IsValid()) {
return nullptr;
Expand Down
1 change: 0 additions & 1 deletion impeller/renderer/backend/vulkan/allocator_vk.h
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ class AllocatorVK final : public Allocator {
ISize max_texture_size_;
bool is_valid_ = false;
bool supports_memoryless_textures_ = false;
bool supports_framebuffer_fetch_ = false;
// TODO(jonahwilliams): figure out why CI can't create these buffer pools.
bool created_buffer_pool_ = true;

Expand Down
7 changes: 7 additions & 0 deletions impeller/renderer/backend/vulkan/barrier_vk.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ namespace impeller {
/// and the Vulkan spec. The docs for the various member of this
/// class are based on verbiage in the spec.
///
/// A useful mnemonic for building a mental model of how to add
/// these barriers is to build a sentence like so; "All commands
/// before this barrier may continue till they encounter a <src
/// access> in the <src pipeline stage>. And, all commands after
/// this barrier may proceed till <dst access> in the <dst pipeline
/// stage>."
///
struct BarrierVK {
vk::CommandBuffer cmd_buffer = {};
vk::ImageLayout new_layout = vk::ImageLayout::eUndefined;
Expand Down
Loading

0 comments on commit f4fbabf

Please sign in to comment.