Jacob’s Tech Tavern

Jacob’s Tech Tavern

Advanced Core Image

Use Metal to create custom Core Image kernels

Jacob Bartlett's avatar
Jacob Bartlett
Jun 17, 2024

Last time, we walked through the basics of Core Image. But even very clever mixing-and-matching of filters isn’t quite sufficient to differentiate your camera app from the competition.

To elevate your app’s snaps above the fray, we need to get down with the pixels.

Song by Disturbed Created with memegenerator

Since its inception in 2005, Core Image was designed to be pluggable, meaning you could subclass CIFilter to create your own filters. These are first-class citizens that mix-and-match with built-in system filters in your customised rendering recipe.

These subclasses might themselves be combinations of built-in CIFilters, or customised pixel shaders written in OpenGL Shader Language (GLSL). These custom shaders, which operate on every pixel of an image, are referred to as Core Image kernels.

These days, Core Image kernels can be written in Metal. This brings ergonomic benefits such as improved compile-time errors and debugging. Performance is also boosted thanks to Metal’s direct GPU access; unbottlenecked by driver software.

Today, we’re going to build on my previous article, Core Image: The Basics, and write our own custom Core Image kernels for CoreImageToy (download it on the App Store!). We’re going to cover: 

  • Configuring Xcode for Core Image kernels

  • Setting up a CIFilter subclasses to use custom kernels

  • Color filter kernels

  • Warp filter kernels

  • Texture filter kernels 

With these advanced technique in your tool-belt, your only bottleneck becomes your skill and imagination.

If you’re not familiar with Metal, I recommend looking at my tutorial Metal in SwiftUI: How to Write Shaders. It assumes zero existing Metal knowledge, explaining fundamental theory from scratch. Through three code-along mini-projects, I guide you through the basics of writing colour, distortion and layer effects you can use in your SwiftUI views!

The project here is entirely open-source, so check out CoreImageToy on GitHub if you get stuck following along.

Configuring Xcode

Following along the WWDC videos in chronological order proved to be confusing, because it took Apple a few tries to work out the ergonomics. Specifically, how to tell the Linker how to compile our Metal files for use with Core Image.

In 2020, the state-of-the-art was an arcane multi-step pipeline of Build Rules, using xcodebuild commands to tell Xcode how to convert .ci.metal files into linkable metal libraries.

These days, we can simply add a linker flag in Xcode Build Settings, under Metal Linker — Build Options:

-framework CoreImage

Update: 12 November 2025

Looks like now you have to add this flag on Xcode 26:

-fcikernel

Why do they keep changing the ergonomics!?


Linking is the final stage of the compiler, where object files (containing assembly code) are combined together into an executable app. This flag ensures that code in the Core Image dynamic library is linked with — and made accessible to — your Metal library.

Adding the ‘-framework CoreImage’ Metal linker flag

This allows us to import CoreImage into our .metal files.

#include <CoreImage/CoreImage.h>

By default, Xcode compiles all your .metal source files into default.metallib resource. This will be important soon.

As we work, I’ll apply our filters to this beautiful photo of Rosie as a kitten.

Original photo of Rosie, our resident alpha cat

Colour Filter Kernels

Colour filters are similar to the .colorEffect() modifier for Metal in SwiftUI.

Making the CIFilter

Our custom filter subclasses the CIFilter base class. We’re going to add a film-grain effect to our photos, creating a nostalgic early-2000s disposable camera effect to our images.

The subclassing boilerplate is similar for all our custom filters, so it’s worth taking the time to get your head around it. 

final class GrainyFilter: CIFilter {
    
    // 1
    @objc dynamic public var inputImage: CIImage?

    // 2    
    override public var outputImage: CIImage? {
        guard let input = inputImage else {
            return nil
        }
        return Self.kernel.apply(extent: input.extent, 
                                 arguments: [input])
    }
    
    // 3
    static private var kernel: CIColorKernel = { () -> CIColorKernel in
        getKernel(function: "grainy")
    }()
}

1 — Input Image

The inputImage needs to be marked @objc dynamic in order to expose it to the key-value coding used by Core Image — when using the filter, we set the input image like so:

filter.setValue(originalImage, forKey: kCIInputImageKey)

2 — Output Image

The outputImage is a computed property, applying the Core Image kernel to the input image. 

The kernel is applied to the ‘extent’ of the input image — that is, the rectangular region in which the image’s pixels are rendered. 

Because the kernel operates on each pixel individually, it’s important to specify exactly which pixels we want to included so we don’t waste processing. Technically, a CIImage has infinite extent, meaning there is no limit on their dimensions unless we actually specify.

The arguments are what we pass into the shader function itself. Here, we pass in the image. We’ll see this in action momentarily.

3 — Color Kernel

We finally need to include a reference to our kernel function — here, we have a CIColorKernel, which is specialised for processing pixel colour information in images.

Our getKernel function is a little (global) helper method:

func getKernel(function: String) -> CIColorKernel {
    let url = Bundle.main.url(forResource: "default", 
                              withExtension: "metallib")!
    let data = try! Data(contentsOf: url)
    return try! CIColorKernel(functionName: function, 
                              fromMetalLibraryData: data)
}

This fetches the default.metallib bundle which is created at compilation. It parses the Metal library as raw data and produces a CIColorKernel object — the Core Image kernel itself — from a function name defined in the library. Here, we’re requesting the grainy function.

Metal Color Kernels

Due to the Metal linker flag set up in Xcode, we can import Core Image into .metal files, which are compiled together into the above metallib file by Xcode.

If you haven’t started coding along, begin now — you don’t learn shaders without getting your hands dirty!

Film grain is an emergent effect of the chemical process used to record images. Film is covered with light-sensitive silver halide particles, which vary in size. Larger particles are more light sensitive, brightening part of the film, while smaller particles show up darker. The random distribution of particles and sizes leads to the grainy effect we know and love.

To create this film-grain effect, I adapted the Random Noise shader from my Metal in SwiftUI tutorial. Mathematically, I want to partially overlay this noise over my images, making each pixel randomly a little brighter or darker.

Random noise effect, using sample code adapted from The Book of Shaders.

The original function signature of our SwiftUI noise shader looked like this:

#include <metal_stdlib>
using namespace metal;

[[ stitchable ]] 
half4 randomNoise(float2 position, half4 color) {
    float value = fract(sin(dot(dest.coord() / 1000, 
                            float2(12.9898, 78.233))) * 43758.5453);
    return half4(value, value, value, color.a);
}

We can adapt this to use Core Image with a few changes:

#include <metal_stdlib>
#include <CoreImage/CoreImage.h>
using namespace metal;

[[ stitchable ]]
float4 grainy(coreimage::sample_t s, coreimage::destination dest) {
    float value = fract(sin(dot(dest.coord() / 1000, 
                            float2(12.9898, 78.233))) * 43758.5453);
    float3 noise = float3(value, value, value);
    return float4(s.rgb + noise, s.a);
}

Albeit with some more coreimage:: namespacing, everything should be conceptually quite familiar:

  • coreimage::sample_t s is the input colour of a pixel, with a vector type float4 (for r, g, b, and a components).

  • The coreimage::destination dest is the destination pixel to be rendered on-screen. 

Applying this filter in our code, we can simply create an instance of GrainyFilter() and use it the same as any other CIFilter.

apply(filter: GrainyFilter(), to: originalImage)

If you want a refresher of how to apply Core Image filters, refer to my piece on Core Image Basics.

Applying the filter, we get instant results.

Over-bright GrainyFilter applied to Rosie

This is working, however we are making it way too bright. Since each pixel is adding between 0 and 1 brightness to its colour, many pixels simply go past an RGB of 1, 1, 1, and clip into full white.

We can easily damp this out by attenuating the magnitude of the noise component, capping the maximum value at 0.1.

[[ stitchable ]]
float4 grainy(coreimage::sample_t s, coreimage::destination dest) {
    float value = fract(sin(dot(dest.coord() / 1000, 
                            float2(12.9898, 78.233))) * 43758.5453);
    float3 noise = float3(value, value, value) * 0.1;
    return float4(s.rgb + noise, s.a);
}
Attenuated GrainyFilter applied to Rosie

This looks much better, but I’m a perfectionist and this is still making the image 10% brighter than the original. To keep average brightness the same, I’m adding a damping factor of -0.1 to each colour component in my kernel function.

[[ stitchable ]]
float4 grainy(coreimage::sample_t s, coreimage::destination dest) {
    float value = fract(sin(dot(dest.coord() / 1000, 
                            float2(12.9898, 78.233))) * 43758.5453);
    float3 noise = float3(value, value, value) * 0.1;
    float3 dampingFactor = float3(0, 0, 0) * -0.1;
    return float4(s.rgb + noise + dampingFactor, s.a);
}
Dampened GrainyFilter applied to Rosie

Retro perfection. Who needs silver halide?

More Colour Filters

Colour filters are conceptually the simplest Core Image kernels, so if you’re coding along, take some time to play around! For inspiration, here’s a few examples I created on my own:

An alien filter which swaps about the RGB components to turn cats into little green men:

[[ stitchable ]]
float4 alien(coreimage::sample_t s) {
    return float4(s.b, s.r, s.g, s.a);
}
Alien filter applied to Rosie (it looks even better on humans)

A spectral filter which constructs a greyscale version of the image, inverts the colours, and darkens each pixel on a cubic scale for a ghostly effect:

[[ stitchable ]]
float4 spectral(coreimage::sample_t s) {
    float3 grayscaleWeights = float3(0.2125, 0.7154, 0.0721);
    float avgLuminescence = dot(s.rgb, grayscaleWeights);
    float invertedLuminescence = 1 - avgLuminescence;
    float scaledLumin = pow(invertedLuminescence, 3);
    return float4(scaledLumin, scaledLumin, scaledLumin, s.a);
}
Spectral filter applied to Rosie

A normalise filter which squares each RGB value — this very smoothly darkens the image to give an OLED-style impression of higher dynamic range in the image. 

[[ stitchable ]]
float4 normalize(coreimage::sample_t s) {
    return float4(pow(s.r, 2), pow(s.g, 2), pow(s.b, 2), 1.0);
}    
Normalize filter applied to Rosie

You’ll notice none of these require the optional coreimage::destination argument, since the same processing is applied to each pixel regardless of position.

Warp Filter Kernels

Warp filters produce a distortion on the source image, shuffling the pixels around at your command. They’re analogous to the .distortionEffect() modifier you can use in SwiftUI.

Subscribe for free to continue reading

Enter your email and get every post from Jacob’s Tech Tavern.
Already have an account? Sign in
© 2026 Jacob Bartlett · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture