Mobile has always been the DevOps problem child.
You can’t just deploy—teams have to run the gauntlet of app review with each release, and you’re in trouble if you didn’t feature-flag the latest regression.
On iOS, you can’t even just build—in his quest to capture every dollar that’s not stapled down, Tim Cook dictates that we must build on Mac hardware.
The result? Infra teams frequently abdicate mobile DevOps, and mobile engineers are left to make it work, often within the same environment as everyone else.
Today, I usually roll Fastlane scripts on a self-hosted Mac Mini using GitHub Actions. We looked at paying for a MacOS cloud runner but, after seeing the number of zeroes in the price, our ops team didn’t speak to me for a week.
It’s been a hot minute since I evaluated the CI/CD space in detail—the last time was on a banking client pre-pandemic. That’s how I found out the hard way that banks really don’t like IP leaving their premises. In the end, we ended up throwing away my research and hosting an on-prem Xcode Server instance, glued together with bash scripts and cron jobs 🤦♂️.
Sponsored content notice
This article is sponsored content, but views expressed are my own—I am providing an honest assessment as I compare mobile DevOps solutions.
The space is more vibrant these days. I’ve been looking into the new kid on the block, Appcircle. Founded in 2019, it claims to be the Enterprise-Grade Fully Automated Mobile DevOps Platform. I want to find out whether it’s worth migrating away from GitHub Actions for my projects.
For my evaluation, I’ll use Bev, my trusty open-sauce project, as the bachelorette looking for her continuously-integrating prince charming (this metaphor needs work). I’ll set up a CI/CD pipeline on both GitHub Actions and Appcircle from scratch and compare how they perform on a few key metrics:
Ease of setup
Infrastructure management
Build speed
Permissions management
Unfair advantages
Ease of setup
First things first.
While much of CI/CD is set-and-forget (at least, until the new version of Android Studio or Xcode breaks everything again), the ease of setup and ease of making changes are vital when considering the toolset you want to be stuck with.
The time to set up the first build is a critical early consideration for any ops tool. Do you need a roomful of staff engineers working round-the-clock to get that first ✅ or can the intern get it done on their lunch break?
Setting up GitHub Actions
In a classic case of Microsoft bundling, Satya very cheekily cross-sells GitHub Actions in the very UI of your repo.
Always happy to take a shortcut, I tap the iOS lane to see where we end up. It conveniently brings us to a… less than friendly starter workflow.
The flow contains raw xcodebuild commands, which aren’t exactly clear.
steps:
- name: Build
env:
scheme: ${{ 'default' }}
platform: ${{ 'iOS Simulator' }}
run: |
# xcrun xctrace returns via stderr, not the expected stdout (see https://developer.apple.com/forums/thread/663959)
device=`xcrun xctrace list devices 2>&1 | grep -oE 'iPhone.*?[^\(]+' | head -1 | awk '{$1=$1;print}' | sed -e "s/ Simulator$//"`
if [ $scheme = default ]; then scheme=$(cat default); fi
if [ "`ls -A | grep -i \\.xcworkspace\$`" ]; then filetype_parameter="workspace" && file_to_build="`ls -A | grep -i \\.xcworkspace\$`"; else filetype_parameter="project" && file_to_build="`ls -A | grep -i \\.xcodeproj\$`"; fi
file_to_build=`echo $file_to_build | awk '{$1=$1;print}'`
xcodebuild build-for-testing -scheme "$scheme" -"$filetype_parameter" "$file_to_build" -destination "platform=$platform,name=$device"
Like all professionals, we like to wrap our xcodebuild commands in Ruby, using a Fastlane script for our deployments:
lane :deploy do
match
gym
api_key = app_store_connect_api_key()
deliver(api_key: api_key)
endThis allows us to make far cuddlier GitHub Action workflows for automatic deployment, whenever we push and merge code into main:
on:
push:
branches: [ main ]
jobs:
deploy:
runs-on: macos-13
steps:
- uses: actions/checkout@v3
- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 15.1
- name: Create gc_keys # set secrets for `fastlane match`
run: echo $GOOGLE_CLOUD_KEYS >> gc_keys.json
- name: App Store Deploy
run: fastlane deployGitHub Actions also has an ecosystem of free and paid community-contributed workflows, which you can place seamlessly alongside your own commands.
The buck stops here with GitHub Actions. Setting up our deployment workflow and Fastlane scripts—including secrets, the Fastlane session, App Store credentials, and certificates—can be a complex task.
Fastlane match is a wonderful tool which takes away some of the pain of managing certificates and profiles on CI; but it’s a tricky beast to get set up which involves storing certificates on third-party cloud services.
While maximising control, the hands-off approach of GitHub Actions leads to 2-3 days of solid work for most experienced seniors just to get up and running. The big benefit, however, is a highly portable workflow which should run on any MacOS machine.
Setting up Appcircle
Appcircle is simultaneously much easier and more overwhelming.
Appcircle lives in a GUI, which contains modules to keep our ops team, mobile devs, and QA testers focused on their own domain.
Instead of spiriting me away from my repo to a hieroglyphical wall of xcodebuild commands, Appcircle begins with a blank canvas, and a helpful note asking me to connect a repo.
You can connect repos from GitHub, Bitbucket, Azure, GitLab, or even self-hosted repos via SSH.
Unfortunately, we can’t simply migrate from GitHub Actions by porting over our workflow files. Appcircle eschews the DevOps addiction to YAML and provides a straightforward GUI buffet of Configurations, Workflows, and Triggers.
Appcircle leans heavily on what-you-see-is-what-you-get: you configure your pipeline using drag-and-drop, assembling build steps directly in the GUI. DevOps purists can rest easy however: every one of these build steps is an open-source repo with a YAML configuration & shell script.
Appcircle also makes it simple to add third-party integrations as build steps such as Sonarqube, Detekt, Maestro Cloud, and Azure DevOps Bot.
The big win, however, comes in the form of managed certificates and provisioning profiles. Instead of setting up automation to retrieve these from third party storage, Appcircle can store, manage, and even create these internally.
My build worked first time without straying very far from the defaults.
We can set this to deploy automatically into App Store Connect or the Google Play Store with some brief setup in the Store Submission module of the dashboard.
Finally, Appcircle allows enterprise users to self-host, setting up a runner as a MacOS VM image to install in your own environment. This VM is fully managed, so infra teams won’t need to take on any of the painful versioning work.
Ease of setup evaluation
While GitHub Actions is substantially easier to find (at least, if you’re hosting on GitHub!), you are left to your own devices when it comes to setting up workflows. Your end result is a very portable set of automation scripts which should run anywhere, but dependencies on third-parties for certificate storage.
Appcircle offers sensible defaults, holds your hand through the pain-points of certificate management, and just works.
If I wanted to refine and estimate JIRA tickets for each approach (using the ubiquitous Fibonacci-numbered story points as a proxy for hours) they might look something like:
Set up CI/CD on GitHub Actions using Fastlane (21 points)
Set up CI/CD on Appcircle (2 points)
Getting Appcircle up and running is 10x simpler than GitHub Actions.
Infrastructure Management
All SAAS CI/CD services offer cloud-hosted build machines to run jobs, saving engineers from either the unpredictable sluggishness caused by local runs, and the janky sysadmin task of keeping your build machine alive.
Of course, some organisations—from highly regulated financial businesses to stealth startups protecting their IP—might want to self-host their CI/CD solution. Every org is different, and so supporting both approaches is table stakes for any enterprise-grade mobile DevOps solution.
GitHub Actions infra management
GitHub Actions offers all the basic options you might expect—Linux, Windows and Mac build machines. These images are also pretty seamless to set up through your workflow file—for example, to roll a custom MacOS machine (necessary for an iOS build), we simply set the runs-on property on our workflow job.
We can also take advantage of open-source workflows such as setup-xcode to select the right IDE version, so our deployment pipeline might look something like this:
jobs:
deploy:
runs-on: macos-13
steps:
- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 15.1Unfortunately, this open-source workflow is still only able to select Xcode versions pre-installed on MacOS cloud runner images.
We’re subject to the whims of GitHub project managers get to decide when the new build machines come out for OS releases: Sonoma machines aren’t even in beta yet. This can make GitHub Actions a bottleneck for your own iOS version target.
The versioning story gets even worse if you’re targeting iOS 17+. I’ve run into unexplained issues with SwiftData, Apple’s new declarative persistence framework, when running GitHub Actions workloads on the cloud.
These forces tend to drive most mobile GitHub Actions users to self-hosting—a process which usually involves petitioning your procurement team for additional Mac hardware.
Self-hosting is phenomenally easy to set up, and you can toggle your machine listeners via a simple shell command—sh actions-runner/run.sh. You can go further and configure your local machines to run the listener automatically in the background to handle your builds and deployments.
These are pretty simple to manage, via a system of tags which tell which workflow on which self-hosted machine it’s allowed to run. When rolling your own devices, you can have confidence they behave most of the time. This works great as long as you’re easily able to tinker with your build machine via screen-share.
If you need to keep your IP away from the cloud, GitHub offers GitHub Enterprise Server which can run on-prem using VMs hosted in your datacenter. This can be configured to support GitHub Actions, however there is no support for MacOS workloads—you will still need to set up a Mac machine as a self-hosted runner.
Appcircle infra management
Migrating to Appcircle is night and day.
In keeping with its do stuff that just works philosophy, Appcircle offers 2 nondescript drop-downs in the default CI configuration:
Machine pool—either Cloud Intel, Cloud ARM, or self-hosted
Xcode version (not required in Android builds)
We don’t need to manage OS versions, we don’t need to keep up with Xcode. There is a simple SLO for enterprise customers: the latest Xcode within 24 hours, including betas and release candidates. Today’s default M1 fleet runs MacOS Ventura 13.5.2, which is non-negotiable. This version is kept up-to-date alongside Xcode versions to ensure all OS versions are supported.
This management is really the killer app for Appcircle. Your infra team no longer has to muddle around with OS and Xcode versioning, because it’s all handled. This is even handled via the on-prem solution—Appcircle installs its own VM image.
Self-hosting has a similarly well-documented process to GitHub Actions—download the app locally, configure the runner, and set it to run your builds in the background.
Self-hosted runners have their own section in the build menu for simple management. Where Appcircle improves upon GitHub Actions, however, is how simple these machines are to use.
Instead of mucking around with sets of string-typed tags in your runner, and matching them up with your YAML workflow file, the Appcircle self-hosted machines in your organisation conveniently appear in the pipeline configuration dropdown.
Infrastructure management evaluation
While GitHub Actions gives you far more configurability, this falls over with the weight of enormous administrative and engineering complexity.
GitHub Actions’ hands-off approach to mobile CI creates a reliance on open-source contribution to fill in the gaps. Even worse, their sluggish approach to supporting new OS versions makes this process particularly painful on MacOS.
Appcircle takes on this OS and IDE versioning overhead itself.
With their plug-and-play build machines, and simple-to-operate self-hosted runners, this comparison isn’t even close.
Build Speed
This section pits GitHub Actions and Appcircle head-to-head on the speed of their flagship cloud-hosted Mac machines.
I support two, pretty standard, workflows in my project:
On Pull Request: I want to run my unit and UI tests before merging code.
On Push: I want to create a signed release build and submit to the App Store.
I’ll run both sets of workflows on each platform to ensure a fair test.
Benchmarking on my M1
To use as a baseline, I’m going to run my tests on my not-particularly-state-of-the-art 2020 M1 MacBook Pro, with 8GB RAM. It runs MacOS Sonoma (14.0) and Xcode 15.1.
This gives us a likely best-case scenario—a dedicated machine performing no other work.
The test suite ran in 2m 22s. That’s the time to beat.
Our deployment step runs lightning-fast, clocking in at just over a minute, 1m 2s.
