Henry Rovnyak

How to make NixOS actually work on a Raspberry Pi 4 Jul 21, 2024 Jul 21, 2024
Jul 21 2024 Jul 21 2024 Jul 14 2024

One day, I was on my Raspberry Pi and decided I should probably update Raspbian. However, I completely botched the job and irreparably destroyed the OS and everything I had so carefully orchestrated on it. I figured I might as well try out NixOS, since it’s abilities to conveniently configure anything and to easily roll back broken configurations should prevent anything like that from happening again.

It turned out to be not so easy. This article contains everything I had to figure out through hours of debugging so you don’t have to do the same. I’ll be talking about the RPI 4 since that’s what I have. Milage may vary on other models. First, here are some invaluable resources you will absolutely need:

Resources#

I owe a debt of gratitude to Carlos Vaz for this wonderful blog post about making a functional base installation of NixOS on a Raspberry Pi. My installation follows the guide, but using btrfs instead of zfs. I won’t cover base installation because that article covers it well.

Here’s a NixOS wiki article about the Raspberry PI 4.

This lets you easily search for Nix packages. In the top bar, you can switch to “NixOS options” which allows you to search for NixOS options. That’s super handy as well.

This is the compendium of options for Home Manager, which you absolutely want to use because it will allow you to configure lots of userspace programs within the NixOS configuration.

I strongly encourage you to use Agenix. It’s a super convenient tool for storing secrets in your Nix config using your SSH public and private keys for encryption. I was intimidated by it at first but it’s really not so bad.

Here is my NixOS config for my Raspberry Pi.

Help! My RPI needs to be exorcised!#

If your Raspberry Pi acts like it’s being posessed by a demon (randomly turning off, not booting for mysterious reasons), it’s most likely a power issue rather than a NixOS issue. It could also be a hardware failure, but let’s pray that it isn’t. A lightning bolt symbol in the corner is a smoking gun, but if it’s not there, that doesn’t necessarily mean it’s okay.

To solve this, first verify that your power supply is actually designed to power a Raspberry Pi. If you just found a plug out of your drawer, it’s probably giving however much voltage and power it wants instead of what the Raspberry Pi needs. Second, verify that whatever you have plugged into your RPI isn’t sucking too much power. If you have that problem, you can solve it by using a USB hub with a separate power supply or by using hardware that consumes less power.

BTRFS Backups#

When you install your system, it’s very important to make sure root is mounted from a subvolume instead of the “root” of your BTRFS partition. When root is on a subvolume, if you accidentally rm -rf something, you can simply delete your root subvolume and mv a snapshot in it’s place. You can’t do that if your root isn’t on a subvolume, and that will turn into a huge headache for you later.

Configuring BTRBK on NixOS#

Here’s where the cool parts of NixOS come in…

To configure BTRBK on NixOS, you first want to insert pkgs.btrbk into your environment.systemPackages.

Then, you need to choose a directory to make snapshots available in (I’ll choose /btrbk-snapshots). You’ll want to install a systemd service that will create the folder if it doesn’t already exist. You could create it by hand, but declaring the service in your config would make reinstallation of your system seamless. The following configuration will do the trick:

systemd.services.btrbk-snapshots = {
  enable = true;
  description = "Make the btrbk-snapshots directory if it doesn't exist";
  wantedBy = [ "multi-user.target" ];
  # Make sure to change the directory to the one you want
  script = ''
    if [ ! -d /btrbk-snapshots ]; then
      mkdir /btrbk-snapshots
    fi
  '';
};

Third, you want a oneshot service that will execute the btrbk command when started:

systemd.services."btrbk-snapshot" = {
  script = ''
    exec /run/current-system/sw/bin/btrbk -q run
  '';
  serviceConfig = {
    Type = "oneshot";
    User = "root";
  };
};

Then, you want a systemd timer that will execute the oneshot service at boot as well as once every hour:

systemd.timers."btrbk-snapshot" = {
  wantedBy = ["timers.target"];
  timerConfig = {
    OnBootSec = "0m";
    OnUnitActiveSec = "1h";
    Unit = "btrbk-snapshot.service";
  };
};

Finally, you can create your BTRBK config:

environment.etc = {
  # Change the configuration as you please
  "btrbk/btrbk.conf".text = ''
    timestamp_format long
    snapshot_preserve_min 16h
    snapshot_preserve 48h 7d 3w 4m 1y

    volume /
    snapshot_dir btrbk-snapshots
    subvolume .
  '';
};

Why doesn’t GPIO work?#

Some GPIO libraries (at least the one I was using) will read /proc/cpuinfo for your Raspberry Pi’s hardware information. The problem is that the Raspberry Pi kernel modifies /proc/cpuinfo to provide extra info, but NixOS by default ships with the mainline linux kernel. To use the Raspberry Pi kernel (and get better hardware support), you can import the RPI hardware config from the NixOS Hardware repo:

imports = [
  # other imports
  "${builtins.fetchGit { url = "https://github.com/NixOS/nixos-hardware.git"; }}/raspberry-pi/4"
]

If your library fails to mmap /dev/mem, then you’ll need to include the kernel parameters iomem=relaxed and strict-devmem=0 because the linux kernel disallows that by default:

boot.kernelParams = [
  "iomem=relaxed"
  "strict-devmem=0"
];

Why does it say I’m running out of storage when I rebuild even though my drive is not nearly full?#

This one’s an oopsie on my part. NixOS builds in the /tmp directory and I configured /tmp to use tmpfs (which “stores” everything in memory). When I rebuild anything large (like the linux kernel), it ran out of space because the RPI doesn’t have much memory. To fix the problem, you can stop using tmpfs on /tmp or you can configure a big enough swap partition.

Why does changing kernel paramaters have no effect?#

When NixOS writes kernel parameters, it writes them into the bootloader config. But for whatever reason, the kernel ignores the parameters written there and instead uses whatever’s in /boot/cmdline.txt. Assuming you’re using systemd-boot (like the blog post I linked earlier), you can use the following config to copy the parameters to the right spot:

boot.loader.systemd-boot.extraFiles."cmdline.txt" = builtins.toFile "cmdline.txt" "${builtins.toString config.boot.kernelParams} init=/nix/var/nix/profiles/system/init";

Why are there no WiFi (or other) drivers?#

Something you can try is removing systemd-boot and enabling the generic bootloader:

boot.loader.generic-extlinux-compatible.enable = true;

I have no idea why this works, but for whatever reason using systemd-boot prevents some drivers from loading. Unfortunately, this does prevent the previous workaround for fixing kernel parameters from working since it relies on systemd-boot options. If you know what’s going on here, please let me know.

That’s all folks!