Why
Say you want to try the latest commit of your favorite package or you want to apply a patch to a software (like when there’s no configuration option to get it to do what you want). I like Nix cause it let’s you do this trivially. But it gets annoying really quickly if you have made such changes to a handful of packages and you have to wait a long time for everything to build when you update.
I use the following justfile so I can “update” my computer with just update and it will push the updated lock files with a new branch update.
update:
git branch -D update || true
git switch -c update
nix flake update
git add flake.lock
git commit -s -m "flake update"
git push -f
git switch main
Then, I wait and see if the builds succeed in the CI and run just upgrade to actually “upgrade” all of my computers:
upgrade:
git switch update
sudo nixos-rebuild switch -L --flake . --use-substitutes
nixos-rebuild switch -L --flake .#chunk --target-host root@2a0f:85c1:840:2bfb::1 --use-substitutes
home-manager -L switch --flake .
git switch main
git merge update
git branch -d update
The build won’t take as long cause it will reuse the cache from the CI.
I have been using GitHub Actions and garnix for my needs. I like GitHub Actions because it’s free. It is suitable for most of my needs, but there are some things that garnix does so well. I have found Garnix to be much smarter at reusing caches to make builds faster. And their computers are so much powerful than GitHub’s. On my “tests”, garnix could build chromium in 1h 57m but my github runner would get killed after six hours.
Of course, I have no use building chromium so let’s look at an example that actually matters. I tried to build an android kernel for my phone, but my github runner would consistently get killed after about 45 minutes when lld would cause it to go out-of-memory. Even on my own computer (that has 12GB memory), the OOM killer terminated my build after 3-4 hours which was annoying. But, garnix apparently built it in 14 minutes! This is too good to be true and I don’t know if it’s because their machines are actually that powerful or because it was being smart and reusing caches from someone else’s build (the repo I used the template from also uses garnix, but the kernel is obviously different so idk?). This was actually the longest run I could for my repository.
How
Garnix has a free tier that comes with 1,500 CI minutes per month. This has been enough for my needs and will likely be the same for you. I will also show you my setup on GitHub Actions in case that’s the only option available to you. Garnix is very easy to setup because it’s built with nix in mind and comes with a built-in cache. With GitHub Actions, you have to setup the cache separately with something like cachix or a self-hosted one with attic. Cachix is only free if you don’t store more than 5 GB of stuff in your cache. That’s not a lot, but could be usable if you regularly purge the cache. Attic is not too difficult to setup, but has been somewhat hit-or-miss in my usage, but still a solid option. Specifically, I get either random crashes or the cache throws a 5XX error for some artifacts and blocks the entire build because of how nix handles cache failures. To use attic, you need to provide your own backend which can be either an s3(-compatible) bucket or a local filesystem.
Garnix
Go to https://garnix.io/ and sign up with your GitHub account. You will be asked whether you want to add garnix to some specific repos or add it to your account so that it can see all (including privates!) of your repos (current and future ones). It can actually pickup new repos on it’s own and start building if you have a flake.nix file at the root.
You will need to add a garnix.yaml file at the root of your repository. If you don’t provide one, garnix will assume the following defaults:
builds:
exclude: []
include:
- '*.x86_64-linux.*'
- defaultPackage.x86_64-linux
- devShell.x86_64-linux
- homeConfigurations.*
- darwinConfigurations.*
- nixosConfigurations.*
As you can see this will build packages and devShells for x86_64-linux and any homeConfigurations, darwinConfigurations, or nixosConfigurations you might have. Adjust this as you see fit. You can read more about the config file here.
To make use of the build cache, you will need to add https://cache.garnix.io to substituters and cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g= to trusted-public-keys in your nix.conf. If you are using NixOS, you can add something like the following to your configuration:
nix = {
settings = {
trusted-public-keys = [
"cache.garnix.io:CTFPyKSLcx5RMJKfLo5EEPUObbA78b0YQ2DTCJXqr9g="
];
substituters = [
"https://cache.garnix.io"
];
};
};
Note that garnix uses the same public cache for everyone! (at least on the free tier). I don’t know if this makes you vulnerable to cache poisoning attacks? You may want to keep this mind and if you know more about this please let me know :)
That’s it. Now if garnix can find anything it can build in your flake.nix, it will build and cache it for you.
GitHub Actions
First sign up for a cachix account and create a cache. You will see a public key for your cache on a page like the following:
Add something like this to your nix.conf like we did above:
nix = {
settings = {
trusted-public-keys = [
"cything.cachix.org-1:xqW1W5NNL+wrM9wfSELb0MLj/harD2ZyB4HbdaMyvPI="
];
substituters = [
"https://cything.cachix.org"
];
};
};
Now go to the settings page of your cachix account and create an auth token (keep this secret!):
After you get your auth token, navigate to Settings -> Secrets and variables -> Actions in your GitHub repository’s settings. Add a secret named CACHIX_AUTH_TOKEN with the token you just created as the value. Should look like this (ignore the other tokens):
Make sure that you add it as a secret and not a variable or else it will be printed in your build logs.
Okay, now your cache should be ready to go. We need to now create a workflow that builds stuff and uploads to our cache. This is going to be somewhat simplified version of what I use myself:
jobs:
build-machines:
strategy:
matrix:
machine:
# hostname of your nixosSystem's
- chunk
- ytnix
os:
- ubuntu-latest
# add others as you need
# - macos-latest
# - ubuntu-24.04-arm
runs-on: ${{ matrix.os }}
steps:
# you might need this if your build causes the runner to run out of disk space
- name: Maximize build disk space
uses: easimon/maximize-build-space@v10
with:
overprovision-lvm: true
swap-size-mb: 1024
remove-dotnet: 'true'
remove-android: 'true'
remove-haskell: 'true'
remove-codeql: 'true'
remove-docker-images: 'true'
build-mount-path: /nix
- name: Install Nix
uses: cachix/install-nix-action@v30
- name: Sync repository
uses: actions/checkout@v4
with:
persist-credentials: false
# this is what uploads our builds to the cache (also makes our CI use the cache from previous runs)!
- uses: cachix/cachix-action@v14
with:
name: cything # replace with the name of your cachix cache
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
useDaemon: false
installCommand: nix profile install nixpkgs#cachix
# this is equivalent to `nixos-rebuild build`
- run: nix build -L .#nixosConfigurations.${{ matrix.machine }}.config.system.build.toplevel
For reference, this is what I use for my android kernel build:
jobs:
build-packages:
runs-on: ubuntu-latest
continue-on-error: false
steps:
- name: Maximize build disk space
uses: easimon/maximize-build-space@v10
with:
overprovision-lvm: true
swap-size-mb: 1024
remove-dotnet: 'true'
remove-android: 'true'
remove-haskell: 'true'
remove-codeql: 'true'
remove-docker-images: 'true'
build-mount-path: /nix
- name: Install Nix
uses: cachix/install-nix-action@v30
- name: Sync repository
uses: actions/checkout@v4
with:
persist-credentials: false
- run: nix build -L .
- uses: actions/upload-artifact@v4
with:
name: result
path: result/