Copy-on-write teaches you EVERYTHING about Swift Internals 🐮

isKnownUniquelyReferenced through the standard library, compiler, and runtime

There is a global function in Swift that, when you fully understand it, will teach you everything you need to know about Swift internals.

You’ll never guess what it is, because I guarantee you’ve used it less than 3 times in the real world, if you’ve even heard of it.

isKnownUniquelyReferenced().

If you know, you know: this function powers the copy-on-write optimisation (a.k.a. CoW, a.k.a.k.a. 🐮).

It pretty much only ever comes up if a) you’re a library author, or b) you’re interviewing someone and want to rate their power level when it comes to Swift internals*

*I never let interviews get to this point, because I burn 45 minutes responding to their first question about weak references.

I wanted to understand how this function worked, and little did I know that it would take me on an odyssey through every single layer and sublayer of the Swift source code. Seriously. Every layer.

As it happens, it’s pretty much the perfect candidate for doing this. type(of:) comes close, but doesn’t quite stick the landing in as satisfying a way.

Will learning all this make your hairline recede another inch? Yes.

Are you going to come on a fun journey to learn Swift Internals™? Also yes.

Let’s work through the various sublayers of the p̶l̶a̶n̶e̶t̶ Swift Source Code and discover once and for all how isKnownUniquelyReferenced works.

I kind of wanted to do a Donkey Kong Bananza bit with each section representing a different sublayer but I truly did not think enough people would get the reference.

Journey to the centre of the 🐮

(yes we’re really going through all these)

• What is 🐮?

  • Value and reference semantics

  • 🐮 In the Swift Standard Library

• Implementing 🐮

• The Swift Standard Library

• Builtins

• (Interlude) The Swift Compiler

  • Our isKnownUniquelyReferenced Compass

• The Abstract Syntax Tree

  • Builtins on the AST

  • Synthesising a Builtin Function

  • The Builtin Module

  • AST Recap

• Swift Intermediate Language

• LLVM Intermediate Representation

• The Swift Runtime

  • Checking the Docs

  • SwiftObject

  • HeapObject

• SwiftShims

• The isKnownUniquelyReferenced Meme

• Last Orders

---

What is 🐮?

For anyone who hasn’t brushed up on their interview prep recently:

🐮 optimises the performance of a Swift struct to get the best of both worlds:

• easy-to-reason-about value semantics.

• low memory overhead from reference semantics.

“Uhh, I just said I wasn’t doing interviews, can you not drop terms like “reference semantics” as if I remember what they are?” I hear you say from the future.

Right. Sorry.

---

Value and reference semantics

Value semantics = copies are independent entities

Reference semantics = copies point to the same underlying memory on the heap

structs that utilise 🐮 store their data in a memory buffer on the heap.

When the struct is copied, all the properties are copied, including the 64-bit pointer to the backing data, referencing the memory address 0x00000000d34db33f.

This is a shallow copy. Each copy of the reference still points to the same underling memory. This underlying memory is shared. This is reference semantics.

When the value of the data on a copy of a 🐮 struct changes, value semantics kick in.

The struct allocates a new buffer of memory on the heap, copies the updated data there, then points at the new buffer. A deep copy. It leaves the original memory block, and other instances of the struct pointing to it, unchanged.

---

🐮 In the Swift Standard Library

Many fundamental types in the Swift Standard Library utilise the 🐮 (copy-on-write) optimisation:

• Array

• Set

• Dictionary

• String

• Data (actually an impostor from Foundation, but it’s one of the gang).

When using these data structures in your code, or even types that contain them, you reap the benefits of the underlying 🐮 optimisation for free.

But it’s possible to implement 🐮 in your own types, too!

---

Implementing 🐮

Very senior iOS engineers will tell you how you can implement your own types that utilise 🐮. Check out this sample robbed straight from Apple’s Swift Optimisation Tips:

In short, you place your struct, T, in Box.

Box wraps T in a reference, Ref, placing the structure on the heap.

You can read the value inside Box trivially.

When you write the value, we check if the memory inside Ref is uniquely referenced, that is, if there is only one pointer to this instance. If it is unique, the value mutates in-place. Otherwise, the heap memory buffer under Ref is copied into a new address before mutation. The Ref pointer inside Box changes to point to this new address.

This illustrative example shows how isKnownUniquelyReferenced can be used to implement 🐮, but it’s kind of sh*tty for real life code: you’re polluting your type with slower reference access semantics to make copying a bit more efficient.

It’s only a good idea when copying is expected and there is enough memory that it makes sense to place it on the heap.

A mysterious function, isKnownUniquelyReferenced, is the secret sauce that makes this optimisation possible.

As it turns out, it’s fucking mysterious, because I originally spent about 3 weeks traversing megabytes of C++ (in 2023, before AI tooling could even touch this kind of task) to understand what the hell it was doing.

---

The Swift Standard Library

First things first.

Let’s download the Swift source code, search for isKnownUniquelyReferenced, and take a gander at the implementation.

Or, you can just CMD+Click that bad boy in Xcode.

It’s holed up in ManagedBuffer.swift, underneath the definition of ManagedBufferPointer. I don’t love putting global functions in the same file as semi-related data structures, but who am I to question the wisdom of our Cupertinoverlords?

Here’s the implementation of isKnownUniquelyReferenced I’ve been searching for in all its glory:

All in a day’s work.

I’ll head to the pub now for a well-deserved beverage.

---

Builtins

I’ve got my beer!

Uh, I suppose I don’t have much else on.

Perhaps we can dive a little bit deeper and find out what _isUnique is doing.

Much deeper. Including Builtins, The Swift Compiler, The Abstract Syntax Tree, Swift Intermediate Language, LLVM Intermediate Representation, The Swift Runtime, SwiftShims, and The isKnownUniquelyReferenced Meme.

---

We find _isUnique defined pretty close by in Builtin.swift:

Loving the doc comments from Apple; let’s as an industry strive for more quantum physics references in our source code!

We need to find out what this Builtin.isUnique(&object) guy is trying to do.

This is where things get interesting.

We find this Builtin function defined somewhere totally different: Builtins.def.

BUILTIN_SIL_OPERATION(IsUnique, "isUnique", Special)

Why is this located in include/AST? What is a SIL operation? Huh???

Before we can progress any further, we need to understand a bit of theory about the Swift Compiler and its dad, LLVM.

---

(Interlude) The Swift Compiler

The Swift Compiler processes Swift source code files into efficient machine code. It’s a pipeline with several Swift-specific optimisation stages.

Swift compilation process: Image from Jesse Squires on Realm Academy

Let’s get our heads around each step in some detail:

1. .swift files are parsed and turned into a data structure known as an Abstract Syntax Tree (AST), making it easy for algorithms to traverse.

2. Semantic analysis is performed on the AST, performing tasks such as type-checking, evaluating protocol conformance, and checking variable scopes.

3. Swift Intermediate Language (SIL) is generated from the AST. This is a halfway point between raw Swift code and the low-level LLVM machine code.

4. This SIL is optimised through various passes. ARC is optimised, generics are specialised, code is inlined, and devirtualization replaces dynamic dispatch.

5. Optimised SIL is transformed into LLVM IR (Intermediate Representation). This is a high-level, language-independent kind of assembly language.

6. LLVM is a compiler toolchain: a frontend that processes any language into IR, and a backend that translates IR into machine instructions for any CPU. LLVM runs multiple optimisation passes on IR, enhancing performance for the target CPU.

7. The LLVM backend transforms the LLVM IR into actual machine code and produces object .o files. These contain these assembly instructions, metadata, strings, and debug information.

8. In the final compilation step, the Linker combines various object files with libraries into a single executable, which the OS can load to run a Swift application.

I go into way more detail here:

---

Our isKnownUniquelyReferenced Compass

With that substantial segue completed, let’s return to our isKnownUniquelyReferenced deep-dive.

Now that we understand how the Swift Compiler works, we have a compass with which to orient ourselves while we dredge the depths of the Swift source code.

Instead of inspecting the 859 individual instances of isUnique we find when we CMF+F the Swift codebase, we can:

1. Find out how the isUnique Builtin function is applied to the Abstract Syntax Tree.

2. Work out how isUnique looks when it is transformed into SIL (Swift Intermediate Language).

3. Determine how isUnique behaves when converted to LLVM Intermediate Representation.

4. Understand how these low-level instructions check whether an object is uniquely referenced.

(I didn’t know this at the time, but this only took me halfway).

---

The Abstract Syntax Tree

Last time we left off, we found the Builtin declaration for the isUnique method in the AST/ folder, Builtins.def:

/// isUnique only returns true for non-null, native swift object
/// references with a strong reference count of one.
BUILTIN_SIL_OPERATION(IsUnique, "isUnique", Special)

.def files contain exported C++ macro definitions. Think of them like header files.

The include/ folder defines a public interface to the Swift Compiler, which is made available to stdlib/, the Swift Standard Library.

---

Builtins on the AST

First, we need to work out how the Builtin isUnique function gets applied to the Abstract Syntax Tree.

We find the Builtin being synthesised here in Builtins.cpp:

Several methods show up here: IsUnique, IsUnique_native (with extra safety checks), and BeginCOWMutation, which the Swift Standard Library uses to internally implement 🐮 for Array, ArraySlice and ContiguousArray.

These variants all produce the same getIsUniqueOperation, hung like so many baubles onto the AST (abstract syntax tree). This itself calls getBuiltinFunction to return a pointer to a ValueDecl, which represents a function signature.

You’ll notice the function signature returns Int1, a single-bit Integer. This is actually the underlying “Builtin” backing store used to implement the Swift Bool!

---

Synthesising a Builtin Function

getBuiltinFunction is actually implemented in this same file:

With most code samples in this article, I’ve stripped out most of the code to make the key moving parts clearer, but feel free to look at the full source yourself. There’s still a whole bunch of code, so I apologise for my crappy redaction.

There are 3 critical steps to follow here:

1. A reference to the Builtin module is retrieved from the ctx, the AST context, a repository of shared information the compiler uses to generate functions.

2. The generic inout parameter for isUnique and the result type, Int1, are defined here, to synthesise the signature of the FuncDecl.

3. id, the Builtin function identifier, is used to synthesise the function declaration we want from the Builtin module, i.e. IsUnique.

In summary, a FuncDecl, a function declaration, is synthesised. This implements the isUnique method on the Builtin module, and gets hung on the Abstract Syntax Tree.

All this processing allows Builtins to behave just like regular Swift functions.

---

The Builtin Module

Swift’s Builtin module itself contains a set of low-level functions and operations that map to LLVM IR instructions, bypassing Swift’s ordinary type safety mechanisms.

As we have seen, Builtins are used extensively in the Standard Library for performance, but Apple doesn’t trust us mere mortals to utilise them ourselves.

To be honest, I’m 2 Hepcats deep at the pub, and I wouldn’t trust me either.

Gipsy Hill brewery, pls sponsor my Substack, I’ll send you my media kit, you can pay me in beer.