Advanced async testing: Unstructured concurrency
Write robust and maintainable software using modern language features
In the first two chapters, we built our testing foundation: We developed techniques to separate concerns through dependency injection and then created mock versions of our injected services.
Subsequently, in Part III, we wrote our first asynchronous tests that checked whether our services were called, and handled cases for successful and failed calls to these services.
We're now equipped to handle testing all sorts of tests that involve using the vanilla async/await
keywords and, as a bonus, the same principles apply to writing unit tests for async let
, taskGroup
, and throwingTaskGroup
.
In the realm of asynchronous code, however, there are cases where a straightforward linear test won't work. Welcome to the world of unstructured concurrency.
Part II Mocking like a Pro
Part III Unit Testing with async/await
Part V (interlude) - Implement your Data Access layer with Combine
Part VI Combine, async/await, and Unit Testing
Part IV: Unstructured concurrency
Let's first define, briefly, what we mean by unstructured concurrency.
The asynchronous programming language features introduced to Swift in 2021 include their crown jewel, the async/await keywords, but if you know Apple - or if you're a regular punter at my tech tavern - you'll know these keywords barely scratch the surface of asynchronous programming. Advanced features are revealed through progressive disclosure as you learn more about asynchronous programming and start to require more advanced use cases.
Structured concurrency allows us to treat async code as if it's good old-fashioned linear synchronous code. Execution of your code is suspended at the await
keyword (yielding the thread to other code), and resumes at a later time.
Structured concurrency covers:
The bread-and-butter
async/await
keywordsasync let
to run methods and fetch properties in paralleltaskGroup
(and its variants) to run multiple jobs simultaneously
There's another set of concurrency features known as unstructured concurrency which behave differently. I refer, of course, to the ever-mysterious Task
. This is called unstructured because the Task
executes code outside of the context in which it is created; and does not suspend execution at its call site. In short, it distinguishes itself from the nice, linear, ordered, structured code to which we’ve become accustomed.
What is a Task?
A
Task
is a 'unit of work' which can run concurrently with other work.Task
s are used to encapsulate asynchronous computation and can be cancelled, paused, and resumed.
Why would we need unstructured concurrency?
Doesn't having these unstructured
Task
s defeat the whole purpose of this new, cleaner paradigm for concurrency?
In some ways, yes, but think about it in this way:
In Swift, the main()
function called at the start of your app is synchronous*. Everything it subsequently adds to the call stack is synchronous.
Asynchronous code can only be called from an asynchronous context; so how can we start using unstructured concurrency in the first place?
Enter Task
. It allows you to create a brand-new asynchronous execution context.
That's right - structured concurrency could not exist without unstructured concurrency somewhere up the chain.
*yes, I know that you can make the
main()
function async in command-line apps. You're very clever for also knowing that.
Problems with Task and Testing
Let's return to Bev, my trusty boozy side-project.
With Task
, you create a new asynchronous context from a synchronous context using a closure, like this method in BeerViewModel:
1 func refreshBeers() {
2 Task {
3 await repository.loadBeers()
4 }
5 }
Why might this be tough to test? It looks simple enough.
Let's try and write a simple linear test like we did in Part III, in BeerViewModelTests.swift
. Since there are no async functions called, we can't just mark the test async
and call it a day.
1 func test_refreshBeers_tellsRepositoryToLoad() {
2 mockBeerRepository.stubLoadBeersResponse = .success([])
3 sut.refreshBeers()
4 XCTAssertEqual(mockBeerRepository.loadBeersCallCount, 1)
5 }
Let's step through to make sure we're all on the same page.
Our test is trying to check whether the
refreshBeers()
method is talking to our repository as we expect.We’ve set our mocks up so that our tests fail if we don't set a stub, so we set one here.
We call the
refreshBeers()
method on our view model.Finally, we're checking whether
loadBeersCallCount
on our mock repository was incremented - i.e., that the function was called.
Let's run our test in Xcode.
Oh no!
Seems like our test failed. Let's try and step through the execution of test_refreshBeers_tellsRepositoryToLoad()
using breakpoints, to see if we can spot the issue:
Breakpoint #1 - Calling refreshBeers()
Breakpoint #2 - Our test assertion
Breakpoint #3 - Our loadBeers()
method called inside the Task
It looks like our assumptions about order of execution is wrong.
Which, naturally, is pretty important to get right in the hermetically-sealed environment of unit tests.
The Task
creates an asynchronous execution context within the closure. Despite our burning desire to see a sea of green ticks, all things are equal under the watchful eye of the Swift runtime. Tim Apple isn't in the business of nepotism, so your unit test code doesn't get special treatment. The code we carelessly flung into an asynchronous context has to wait for the runtime to offer a thread on which to execute.
Meanwhile, the refreshBeers()
method finishes - it's a linear function on a synchronous execution context, after all. After the Task
is created, the function's job is done. Now, it returns to test_refreshBeers_tellsRepositoryToLoad()
and continues to the next line of the test code - our assertion.
Our assertion fails because loadBeersCallCount
is still 0. The unstructured Task
produced by refreshBeers()
doesn't get to jump the queue.
We've identified the problem, which is 80% of the job. After calling sut.refreshBeers()
in our test, we need to suspend execution of our test until we know that the code inside our MockRepository
has been called.
In short, we have to wait.
If, following our breakpoints, you were to inspect our mock after the await repository.loadBeers()
method is called, you would find that loadBeersCallCount
equals 1
as expected. So we need to find a way to delay the execution of the rest of our test and wait for this to be set.
Fortunately, the XCTest framework has always had an approach for waiting, which predates async/await by 7 years - expectations!
XCTestExpectation
Expectations allow us to test asynchronous operations. It behaves like a 'promise' that an operation will complete in the future.
There are 3 components to expectations:
Setting up the expectation as a local property
Waiting for the expectation, with a timeout
Fulfilling the expectation
If the expectation is not fulfilled in time, your test fails.
In the world of unstructured concurrency, we are using closures to move execution into a context separate from the function in which your Task
is created. Expectations are the natural fit for closure-based asynchronous operations.
Let's update our test to handle an expectation:
1 func test_refreshBeers_tellsRepositoryToLoad() {
2 mockBeerRepository.stubLoadBeersResponse = .success([])
3 let exp = expectation(description: #function)
4 // ???
6 sut.refreshBeers()
7 waitForExpectations(timeout: 1)
8 XCTAssertEqual(mockBeerRepository.loadBeersCallCount, 1)
9 }
The start, middle, and end of this test are the same as before, except we've created an expectation and waited for it before our assertion. We're still calling sut.refreshBeers()
and asserting that the loadBeersCallCount
on our repository has been incremented. We pick 1 second as a timeout, because we don't expect the runtime to take long finding a thread on which to execute.
But how do we fulfill our expectation?
This is where we need to think a little.
We need to fulfill the expectation - that is, call exp.fulfill()
- once we know that loadBeersCallCount
has already been incremented. Since our refreshBeers()
function simply sets the Task
up and nothing else, that won't be any help to us. But maybe there's something that changes as a result of await repository.loadBeers()
which we can use.
Leveraging our mocks
Fortunately, we are all mocking masters because we have all read Part II of this series, Mocking like a Pro. Stepping through the code, we see that we, naturally, increment loadBeersCallCount
in the body of the loadBeers()
function. So we need to trigger the expectation once this incrementing is complete.
Fortunately, we have set up our mocks like professionals and have infrastructure in place to do just that:
public final class MockBeerRepository: BeerRepository {
public var beersPublisher = PassthroughSubject<LoadingState<[Beer]>, Never>()
public init() { }
public var stubLoadBeersResponse: Result<[Beer], Error>?
public var didLoadBeers: (() -> Void)?
public var loadBeersCallCount = 0
public func loadBeers() async {
defer { didLoadBeers?() }
loadBeersCallCount += 1
beersPublisher.send(stubLoadBeersResponse!)
}
}
This defer { }
statement delays the execution of the wrapped closure until the end of the function execution - ensuring it's the last thing that happens. This makes it the perfect place to fulfill our expectation. That's exactly what the didLoadBeers
property is here for - we can pass in closures to fulfill expectations that run after the mock method is called.
The
defer
keyword even lets us to execute code after a functionreturn
s a value. This makes it an extremely powerful tool when writing tests that depend on mocks.
If you want to understand more about how this beersPublisher
works in our repository, sit tight for Part V - (interlude) - Implement your Data Access layer with Combine.
Now, we can implement our full test:
1 func test_refreshBeers_tellsRepositoryToLoad() {
2 mockBeerRepository.stubLoadBeersResponse = .success([])
3 let exp = expectation(description: #function)
4 mockBeerRepository.didLoadBeers = { exp.fulfill() }
6 sut.refreshBeers()
7 waitForExpectations(timeout: 1)
8 XCTAssertEqual(mockBeerRepository.loadBeersCallCount, 1)
9 }
And it passes!
To summarise how far we've come; we utilised the power of our mocks - and our understanding of the Swift concurrency model - to create green unit tests for code that leverages unstructured concurrency.
Expectations in async tests
Expectations come from a world long before async/await, and so originally had no concept of interoperation - essentially they didn't really work in a test marked async
, so until very recently we needed to awkwardly wrap our async methods in a Task
.
But since Xcode 14.3, this changed. Nowadays, you can make expectations in async tests and wait for them using the await
keyword:
await fulfillment(of: [exp], timeout: 1)
I find myself reaching for this tool in 3 scenarios:
You have an unstructured
Task
being triggered as part of aprivate
function. This is called by an asyncinternal
orpublic
method which you can call from your test.Your async method might set a synchronous closure, which includes a
Task
.Instead of using a mock's closure to fulfill our expectation, we use Combine to listen to changes to a
@Published
property in a view model, which itself changes in response to anasync
method.
There are a number of tricks to get Combine playing well with async/await and I'm going to explain them all in Part VI - Combine, async/await, and Unit Testing.
Here's a quick example of how Scenario #1 looks:
func test_asyncMethod_callsSyncMethod_createsUnstructuredTask() async {
let exp = expectation(description: #function)
mockAPI.didCallFunctionInsideUnstructuredTask = { exp.fulfill() }
await sut.someAsyncMethod()
await fulfillment(of: [exp], timeout: 1)
XCTAssertEqual(mockAPI.functionInsideUnstructuredTaskCallCount, 1)
}
Advanced use of defer
in our mocks
There are instances we can go further: we can add conditionals into the deferred didLoadBeers
closure to fulfill our expectation only when a specific condition is fulfilled.
This is helpful when you might be calling the mock method multiple times with different answers - expectations may only be fulfilled a single time, and will cause tests to fail if called twice. Plus, you might only want your assertion to be checked after a specific value arrives.
Here's one example of waiting for a value from a repository to exist before we trigger our assertion. The value of userPublisher
might be .idle
, .loading
, .success(user)
, or .failure(error)
. Our test cycles through .idle
and .loading
before we land on .success
, the value from which we want to make our assertion against (I knew English GCSE would come in handy one day).
func test_loadUser_setsName() async {
mockUserAPI.stubGetUserResponse = .success(User(name: "Jacob")
let exp = expectation(description: #function)
sut.userPublisher
.sink(receiveValue: {
guard case .success(let user) = $0 else { return }
exp.fulfill()
})
.store(in: &cancelBag)
await sut.loadUser()
await fulfillment(of: [exp], timeout: 1)
XCTAssertEqual(sut.user.name, "Jacob")
}
Here, we listen to the value from our repo and only fulfill the expectation when the user
arrives wrapped in the .success
response from our userPublisher
.
Combine and async tests
The last example touches on a seldom-discussed issue when it comes to asynchronous unit testing.
In 2019, coinciding with SwiftUI 1.0, Apple released the Combine framework to try their hand at a functional reactive programming paradigm. This was Apple's most significant new concurrency-related release since Grand Central Dispatch in 2009.
Since async/await was released in 2021, Combine has somewhat fallen by the wayside, however it is a fairly complete and robust approach to concurrency that is often the best way to solve a problem. More importantly, since Combine is an important implementation detail of SwiftUI (it underpins @ObservableObject
), it is vital to understand how to work with it in order to test your view models properly.
The inter-operation between Combine and async/await is not always intuitive. My final two chapters dive headfirst into this maelstrom to demonstrate how you how you can get these two concurrency approaches to cooperate - both in your code and in your unit tests.
Conclusion
In this article, we've explored the problems you might encounter when utilising unstructured concurrency. We explored first-hand the execution order of our code during unit tests, and why the straightforward approach we used with basic async
functions didn't work when invoking Task
.
We looked at using the 'legacy' technique of XCTest expectations
to handle this closure-based code. We leveraged the power of our mocks and the under-appreciated defer
keyword to ensure that our test execution happened in the precise order we define. Finally, we looked at more advanced use cases, which we could solve by using the brand-new async
-compatible expectation fulfilment
API.
In the penultimate chapter of this masterclass, I'll give a refresher on Combine and demonstrate how you can use it to implement a reactive Data Access Layer in your app. Stay tuned for Part V - (interlude): Implement your Data Access layer with Combine.
Part II Mocking like a Pro
Part III Unit Testing with async/await
Part V (interlude) - Implement your Data Access layer with Combine
Part VI Combine, async/await, and Unit Testing
Another gem! Thanks for sharing your thoughts and this information :D