SwiftUI How to Apply Shader "whatever" is behind the view

Question

I am writing a program that has some images where you can scroll through and I also have a rectangle-like view that is called PixeleteView where I want to pixelate the content behind, it is easy to apply my shader to the view and work however I want to pixelate the images below without applying my shader to the images.

What I mean is that I want to apply an effect just like blur or some sort however on the background content of the current view meaning taking the background content redrawing it in the current background and applying the effect to it or this is what a I initially thought however I couldn't do that.

I am new to Swift and tried several things and coudn't find a working solution not even partially.

// ContentView

struct ContentView: View {
    var body: some View {
        ZStack() {
            Content()
            PixeleteView()
                
        }
    }
}

Content is just a scrollable images and texts, however it can be anything.

Inside my PixeleteView I want to apply my shader

// PixeleteView

struct PixeleteView: View {
    var body: some View {
       .
       .
       .
       .background(
        
        //
        // This is where I want my shader to be
        // However instead of this it must change the background of 
        // this view.
        //
        // You can think of this like
        // Rectangle()
        //    .fill(.ultraThinMaterial)
        //
       )
       .clipShape(RoundedRectangle(cornerRadius: 50))
    }

And my shader is jsut a simple metal file to pixelete the content With .layerEffect applied to

// MySahder.metal

#include <metal_stdlib>
#include <SwiftUI/SwiftUI_Metal.h>
using namespace metal;


[[ stitchable ]] half4 pixellate(float2 position, SwiftUI::Layer layer, float strength) {
    float min_strength = max(strength, 0.0001);
    float coord_x = min_strength * round(position.x / min_strength);
    float coord_y = min_strength * round(position.y / min_strength);
    return layer.sample(float2(coord_x, coord_y));
}

So I believe I could bring the idea of what I am trying to do instead of using Rectangle().fill(.ultraThinMaterial) I want to implement my own effect

Here is an example when I run this

// Example

Image(systemName: "figure.run.circle.fill")
    .font(.system(size: 300))
    .layerEffect(ShaderLibrary.pixellate(.float(10)), maxSampleOffset: .zero)

This code produces the example on the static image however what I want is to apply this effect to whatever is behind.

Example usage:

And in order to describe what I really want here is the Figma representation of what I want to implement In order to understand I recomend taking a look at the image

Desired Result:

Again I want to reuse this effect on many places so I do not really want to apply this to individual view inside content I believe there has to be less-complex way to achieve this.

To be clear again, my problem is:

I want to apply "my shader" to the View; however, I want it to manipulate/distort the content behind it. That part is important

I tried to apply the effect directly to .ultraThinMaterial however it did not worked

Answer 1

A .layerEffect works a bit like .blur, which means it modifies the complete view it is applied to. This is different to a ShapeStyle, which is used to fill a Shape. So using a .layerEffect is different to filling a shape with a Material.

If you only want a portion of the base view to be pixelated, you need to apply the layer effect to a copy of the base view and show this as a separate layer. This top layer can then be clipped or masked to a smaller area.

One way to do this is as follows:

• show the content twice, as two layers of a ZStack
• apply the shader to the top layer
• apply a mask to the top layer, to confine the shader to a specific area
• use .matchedGeometryEffect to position the mask
• the source for the .matchedGeometryEffect can be shown in the background of the ScrollView
• a shadow effect can also be applied to the top layer, after applying the mask.

The example below shows it working. I found that it was necessary to use a VStack in mainContent, it didn't work reliably when a LazyVStack is used.

struct ContentView: View {
    @Namespace private var ns

    var body: some View {
        ScrollView {
            ZStack {
                mainContent
                mainContent
                    .background(.background)
                    .layerEffect(
                        ShaderLibrary.pixellate(.float(10)),
                        maxSampleOffset: .zero
                    )
                    .mask(
                        RoundedRectangle(cornerRadius: 20)
                            .matchedGeometryEffect(id: 0, in: ns, isSource: false)
                    )
                    .shadow(radius: 20)
            }
        }
        .background {
            Color.clear
                .frame(width: 350, height: 250)
                .matchedGeometryEffect(id: 0, in: ns)
        }
    }

    private var mainContent: some View {
        VStack {
            Image(.image1)
                .resizable()
                .scaledToFit()
            Image(.image2)
                .resizable()
                .scaledToFit()
            Image(systemName: "figure.run.circle.fill")
                .font(.system(size: 300))
            Image(.image3)
                .resizable()
                .scaledToFit()
            Image(.image4)
                .resizable()
                .scaledToFit()
        }
    }
}

---

EDIT You were asking in a comment, if it is possible to get it working without replicating the underlying view. I wasn't able to find a way when using SwiftUI. However, you could wrap the replication in a ViewModifier, to make it easier to use.

• The mask can be passed in as a parameter.
• An overlay can be used for the replicated layer, instead of wrapping the content in a ZStack. This way it is strictly a view modifier, as opposed to a view wrapper.

struct Pixelated<M: View>: ViewModifier {
    let mask: M

    func body(content: Content) -> some View {
        content
            .overlay {
                content
                    .background(.background)
                    .layerEffect(
                        ShaderLibrary.pixellate(.float(10)),
                        maxSampleOffset: .zero
                    )
                    .mask(mask)
                    .shadow(radius: 20)
            }
    }
}

For convenience, you might like to define an accompanying View extension:

extension View {
    func pixelated<M: View>(mask: () -> M) -> some View {
        modifier(Pixelated(mask: mask()))
    }
}

Here is how it can be used for the same content as before:

struct ContentView: View {
    @Namespace private var ns

    var body: some View {
        ScrollView {
            VStack {
                // Images, as before
            }
            .pixelated {
                RoundedRectangle(cornerRadius: 20)
                    .matchedGeometryEffect(id: 0, in: ns, isSource: false)
            }
        }
        .background {
            Color.clear
                .frame(width: 350, height: 250)
                .matchedGeometryEffect(id: 0, in: ns)
        }
    }
}