Apple Animation Through the Ages: 2014

Try out real code from NeXTSTEP to Mac OS X to SwiftUI

Timeline

1989 — NeXTSTEP Generation: Display PostScript

2001 — Apple’s Renaissance: Quartz & OpenGL

2007 — The Modern Era: Core Animation

2014 — The Performance Age: Metal

2019 — The Declarative Revolution: SwiftUI

---

2014 — The Performance Age: Metal

Fast-forward another seven years.

Steve Jobs is dead. iPhone has eaten the world.

Core Animation — and the UI framework built on top of it, UIKit — has spawned app giants like WhatsApp, Uber, and Candy Crush. For the engineers crafting cutting-edge mobile games, however, there was a creeping problem.

What’s wrong with Core Animation?

The high-level abstractions provided by Core Animation made it trivial to get started and build something great, but abstraction has a cost. Inherently, when you aren’t micromanaging every clock cycle, you’re trading efficiency for ergonomics.

Core Animation is phenomenal for cookie-cutter tasks like animating the opacity of UIButtons or dragging UITableView cells about. But what if you’re rendering a space shooter, a breakneck racing game, or a fully immersive farming simulator?

OpenGL ES (Open Graphics Library for Embedded Systems) was the standard for high-performance graphics APIs on iOS. It was a low-level 3D graphics API that gave fine-grained control over exactly what you were rendering to the screen.

OpenGL ES sounds great. What’s the problem?

OpenGL ES is a third-party library. This meant the software interfaces with the graphics hardware — the GPU — through a driver. Drivers are software that creates an interface to hardware via low-level instructions. These graphics library drivers themselves had to run on the CPU.

In the early 2010s, graphics chips were improving at a breathtaking clip. Their performance was finally starting to overtake that of CPUs. This led to a situation where the OpenGL ES driver, running on sluggish CPU hardware, bottlenecks the performance of graphics code running on the GPU.

Graphics engineers — the boffins building graphics engines like Unreal Engine or Unity — needed to get closer to the hardware. Closer to the Metal.

Key concepts of Metal

Before we start coding, let’s ensure we’re all on the same page with a brief conceptual overview of how the Metal framework actually works — it’ll be a little overwhelming if you’ve never worked with graphics pipelines before, and that’s okay.

• Shaders are small functions that run on the CPU to handle rendering. The name is historical — back in prehistoric OpenGL days, they just controlled the shading of vector shapes.

• Vertex shaders transform 3D coordinates in vector space to 2D coordinates, which can map nicely to a plane — such as your screen.

• Fragment shaders, a.k.a. pixel shaders, color the individual pixels on a screen when a shape is rasterised — that is, transformed from a 2D vector image to dots on a screen.

• Metal Shading Language is the language we use to write these shaders. It’s based on C++ 2011 — with several language features trimmed down and some graphics-focused features mixed in.

• MTLDevice is an abstraction in MetalKit representing the hallowed hardware of the GPU itself.

• MTLCommandBuffer is the place where drawing instructions are stored before they’re piped over to the hardware for massively parallelised execution.

• MTLCommandQueue manages the above command buffers while maintaining order of execution.

Does your brain hurt?

Great, that means you’re learning!

Let’s write some code

Since we’re finally in ✨ the future ✨, we can create an iOS app that runs on Swift. Code along with me, or check out the sample project yourself.

While I really wanted to write this section in Swift 1.0, which was the state-of-the-art in 2014, it isn’t possible to get a build of Xcode that both (1) contains the Swift 1.0 runtime and (2) runs on MacOS Ventura.

I could argue that I want to let you run the projects for yourself, but frankly, I’m just a bit lazy.

As you’ll be familiar with by now, let’s start by adding the Metal and MetalKit frameworks to the Xcode project:

Now we come to the main event — adding a Shaders.metal file to your project. This is the C++-based Metal Shading Language I mentioned before. It’s pretty terse, packing a lot of semantics into a few lines.

#include <metal_stdlib>
using namespace metal;

vertex float4 vertex_main(constant float4* vertices [[buffer(0)]], uint vid [[vertex_id]]) {
    return vertices[vid];
}

fragment float4 fragment_main() {
    return float4(1, 0, 0, 1);
}

• metal_stdlib contains the helper functions and types that turn out useful for writing shaders.

• using namespace metal allows us to namespace and prevents us from needing to prefix all our methods with the Metal framework name.

• The vertex_main function is prefixed vertex float4 , denoting it as the vertex shader, which maps 3D coordinates to a screen and returns a 4D vector — that is four numbers. This tells the GPU where to look for the vector data it wants to draw.

• The fragment_main function is the fragment shader that colors the pixels on-screen— here, it simply returns a red color in RGB format.

The Swift APIs for Metal

You can relax a little. We’re coming back to more familiar-looking Swift APIs for now. First, we set up MetalView.swift:

final class MetalView: MTKView {

    var commandQueue: MTLCommandQueue!
    var pipelineState: MTLRenderPipelineState!
    
    override init(frame frameRect: CGRect, device: MTLDevice?) {
        super.init(frame: frameRect, device: device)
        commandQueue = device?.makeCommandQueue()!
        createRenderPipelineState()
        clearColor = MTLClearColor(red: 0.8, green: 0.8, blue: 0.8, alpha: 1.0)
    }

    required init(coder: NSCoder) {
        fatalError()
    }

// ... 

An MTKView is initialised with a simple rectangular frame and a MTLDevice. As I mentioned earlier, this is an object that represents the GPU.

Instantiating this view is dead easy now that we’re working with UIKit — you can add this view into the project template’s ViewController.swift:

import UIKit
import MetalKit

final class ViewController: UIViewController {
    
    var metalView: MetalView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        guard let metalDevice = MTLCreateSystemDefaultDevice() else {
            fatalError("Metal is not supported on this device")
        }
        
        metalView = MetalView(frame: view.bounds, device: metalDevice)
        view.addSubview(metalView)
    }
}

The MTLCreateSystemDefaultDevice method in MetalKit allows the framework to locate the GPU hardware and return a metalDevice object representing it. With it, we initialise our MetalView and set up our rendering pipeline.

To get our Metal rendering kicked off, we need to fill out two more non-trivial components in our MetalView. createRenderPipelineState() and draw(_ rect: CGRect).

First, we’ll set up the render pipeline:

func createRenderPipelineState() {
    let library = device?.makeDefaultLibrary()
    let vertexFunction = library?.makeFunction(name: "vertex_main")
    let fragmentFunction = library?.makeFunction(name: "fragment_main")
    
    let pipelineDescriptor = MTLRenderPipelineDescriptor()
    pipelineDescriptor.vertexFunction = vertexFunction
    pipelineDescriptor.fragmentFunction = fragmentFunction
    pipelineDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
    
    pipelineState = try? device?.makeRenderPipelineState(descriptor: pipelineDescriptor)
}

Here, we tell the GPU which functions from Shaders.metal to use for the vertex and fragment shaders. We then set up a rendering pipeline that takes vertices and colors and processes them into on-screen pixels.