MacOS EC2 Instance integrated as Jenkins node for iOS build and test
With the introduction of macOS EC2 instance by AWS in November , 2020 it is now possible to build, run and test the macOS applications in the AWS ecosystem. In this article, I will go through the steps of provisioning a macOS EC2 instance and using it as a Jenkins agent to run the iOS builds for mobile apps in a pre-existing Jenkins setup running on EKS.
But first, let me take you through some of the limitations of the macOS instance :
- They are currently only available in 5 regions — US East (N. Virginia), US East (Ohio), US West (Oregon), EU (Ireland), and Asia-Pacific (Singapore) AWS Regions
- They run on dedicated hosts — this makes the starting and shutting down of the instance take longer and it also requires additional steps to start/terminate
- Billing is per second with a 24-hour minimum allocation period — this means that even if you shut down the instance 5 minutes after starting it, you still pay for it until the 24 hour mark, after the 24 hours are up, billing switches to the regular per second method
Keeping these limitations in mind, we needed a solution that lets us have a macOS build agent attached to our existing Jenkins which runs on an EKS cluster in a region where the macOS instance isn’t available.
For this example, let us consider that our Jenkins Cluster is setup in the Sydney region and I will use the North Virginia region to host the macOS instance.
The below architecture diagram outlines the setup:
Here is an explanation around the architecture :
- Jenkins is running on an EKS Cluster within the Sydney region
NOTE — I have not included the Jenkins EKS Cluster setup for this article, to keep the focus on the macOS instance setup only
- The macOS instance is provisioned in the North Virginia region (macOS instance is available only in certain regions)
- The 2 VPCs ( Jenkins VPC , macOS VPC) are connected through cross -region VPC peering
- The macOS instance is setup to build iOS and macOS projects only
- The instance is added as a build agent to the existing Jenkins running on EKS
If you happen to have your Jenkins setup in the same region as the macOS instance you can ignore some of the steps here.
I have divided the implementation into 4 high level steps:
- Setup the VPC : Setup a VPC in the North Virginia region for this example, with a cross region VPC peering to integrate with the existing Jenkins VPC
- Launch the instance : Create a dedicated host and launch the instance utilising it
- Connect and prepare it to build and test iOS apps: Connect to the instance using ssh and install all the necessary binaries on the instance to build and test the iOS app
- Setup the instance to run as a Jenkins agent
Let’s get into the details now.
1. Setup the VPC
I am keeping the VPC setup simple here.
I will be using a single availability zone only since I need a static private IP associated to the instance, the reason for this is, if there is a situation where I need to terminate the instance, I don’t have to go around changing the IP address in the Jenkins master configurations. Also, since it is going to be used only just for iOS builds which is a very small part of our build and deployment requirements, I do not need it to be highly available.
The instance will be launched in the private subnet because of security reasons, it will have the necessary access to the internet via the NAT gateway present in the public subnet.
The VPC will have a cross region VPC peering connection with the Jenkins VPC which is hosted in the Sydney region.
On the network side of things, we go by the principle of least privilege and keep everything closed out, just ensuring that the instance can make calls to git(port 22) and reach to to the internet to install packages like xcode, yarn etc (port 443, 80).
2. Launch and prepare the instance
Let’s have a look at the CloudFormation template for the instance.
As discussed earlier, the macOS instance are available on dedicated host to run on and I also wanted to add a private static IP address to it, so I have added these 2 components as well in the template.
Key call outs here:
- We need a static IP to ensure we do not have to go back to the Jenkins configuration to change the host reference in case the instance gets terminated
- Keep the disk space 100gb+ as we ran into memory issues while doing the xocde installation on the instance
- Install java on the instance to attach it as a Jenkins agent
NOTE — “It is quintessential to install the same version of java on the agent otherwise it errors out saying “remote connection to the agent failed”
- Stopping the instance takes a minimum of 15 minutes , be mindful of this while setting up any automation on the instance start/stop
NOTE from AWS Documentation
“When you stop or terminate a Mac instance, Amazon EC2 performs a scrubbing workflow on the underlying Dedicated Host to erase the internal SSD, to clear the persistent NVRAM variables, and if needed, to update the bridgeOS software on the underlying Mac mini. This ensures that Mac instances provide the same security and data privacy as other EC2 Nitro instances. It also enables you to run the latest macOS AMIs without manually updating the bridgeOS software. During the scrubbing workflow, the Dedicated Host temporarily enters the pending state. If the bridgeOS software does not need to be updated, the scrubbing workflow takes up to 50 minutes to complete. If the bridgeOS software needs to be updated, the scrubbing workflow can take up to 3 hours to complete”
3. Connect and prepare it to build and test iOS apps
You can connect to the instance by either using ssh or apple remote desktop(VNC)
- In order to setup the vnc connection, refer to the aws documentation
- The macOS AMIs includes the Command Line Tools for Xcode, Homebrew etc, so install whatever else you need for the build, for example xcode, yarn, apple developer certificates etc and then test a local build on the instance
- Once the instance is ready, bake a golden AMI out of it, that you can use to spin up the instance if you need to do so again
4. Setup the instance as a Jenkins agent
Use these settings to connect to the MacOS instance as a permanent node exclusively for running the MacOS builds:
- Install the “SSH build agent” plugin for Jenkins
- Create an SSH key to be used for the connection (Since I have an EKS setup, I created a secret and accessed it through the Kubernates credentials plugin)
- Login to Jenkins as an admin and navigate to “manage nodes and cloud”
- Configure the node using these settings:
Once the configuration is saved, click on run agent and you should see something similar in the agent logs.
That’s it, the agent is attached successfully to Jenkins Master, you are ready to run your macOS builds. Make sure you use the right “label” in the job to ensure the builds run on this instance.
In case you are setting up your jenkins on eks from scratch and/or you want to use an automated way of doing the jenkins side of things, refer to the aws github repo.
Conclusion and Next Steps
That is how seamlessly you can now enhance your Jenkins server to perform macOS builds and tests utilising the newly launched AWS EC2 macOS instances.
As a next step, once the instance is available in my region (Sydney, in this case), I will move the instance in the same VPC I have Jenkins running. We might also look at the recently launched Xcode cloud once it is in a more matured state.