Skip to main content

Using an airgapped computer

In the Key Management article we discussed the need to keep private signing keys on a dedicated, separate hardware token.

However, even if you store the signing keys in a hardware token and the signing happens inside the token, you still need a computer somewhere! The computer communicates with the token: to instruct it to generate a key, or to send the APK to the token for signing and to receive the signature. The PINs to access the hardware token are also entered by human users on the computer and sent to the token over USB.

So this computer becomes critical for security. If the computer is compromised with malware, it could sniff the PIN and maliciously instruct the token to create a signature. (This risk is mitigated by hardware tokens that require a physical touch for every operation, such as YubiKeys.)

Therefore, using the hardware token should be done from a hardened computer that is dedicated to this single task. The computer should be as locked down as possible.

General ideas​

Bootstrapping a secure, transparent, trustworthy system, from the hardware to the OS, is an entire rabbit hole on its own. Depending on your threat model, and how much time you want to put in, there are many avenues to pursue. On the hardware level, do you prefer open source hardware (transparency), or locked-down hardware like Chromebooks (security)? On the firmware level, you could use Coreboot (if your hardware is supported). On the operating system level, you can choose from Tails, NixOS, or a minimal Debian install. And do you keep it fully offline, or do you occasionally connect it to a network (for example, to install updates)?

Because this is such a huge topic, this guide does not discuss it in depth. Instead, we use a minimal Debian install, as described below. For the current state of FMD as a project, this seemed to be the best balance between security and setup effort.

Setting up a minimal Debian system​

This is a quick overview of how to install a minimal Debian system. For details, see the Debian installation guide.

  1. Find a local mirror.
  2. Download the following files for your architecture: debian-13.1.0-amd64-netinst.iso, SHA512SUMS, SHA512SUMS.sign. For example: https://debian.ethz.ch/debian-cd/13.1.0/amd64/iso-cd/
  3. Verify the ISO image:
    1. Install SequoiaPGP: sudo apt install sq
    2. Look up Debian's official the PGP key fingerprints.
      1. Ideally, do this from multiple vantage points on the internet. For example, via two different ASes, such as your home ISP and your mobile provider (if they differ).
    3. Fetch Debian's public certificate from their keyserver: sq network search --server hkps://keyring.debian.org DF9B9C49EAA9298432589D76DA87E80D6294BE9B
    4. Mark the certificate as trusted (using the values looked up above): sq pki link add --cert="DF9B 9C49 EAA9 2984 3258 9D76 DA87 E80D 6294 BE9B" --userid="Debian CD signing key <debian-cd@lists.debian.org>"
    5. Verify the PGP signature of the checksum file: sq verify --signature-file debian-SHA512SUMS.sign debian-SHA512SUMS
      1. If your ISO is signed by one of Debian's other keys, repeat the above for that certificate.
    6. Verify the integrity of the ISO file using the checksum file sha512sum -c debian-SHA512SUMS. The expected output is: debian-13.1.0-amd64-netinst.iso: OK
  4. Write the ISO image to a USB stick.
  5. Boot your computer from the USB stick and complete the installation.
    1. Choose strong, unique passwords.
    2. In the partitioning step, choose "Encrypted LVM" to enable disk encryption.
    3. In the software installation step, de-select the desktop environments. Don't install any extra software, except the minimal system utilities.
  6. Make sure that your UEFI/BIOS is locked down (supervisor password configured, SecureBoot enabled).

This minimal Debian install is only a baseline for a dedicated signing computer. Note that there is still a bootstrapping problem: how can we verify the integrity of the ISO file on a potentially compromised system?

Installing the dependencies​

No matter which hardware token you are using, install the following packages which will be needed and/or useful. They provide the scm-hsm-tool, pkcs11-tool, and certtool commands.

sudo apt install opensc opensc-pkcs11 gnutls-bin

What to run on the computer​

Creating the signing key and signing should be done on the hardened computer. However, this computer should not be used for building the artifacts (such as APKs).

Building the unsigned APK should be done on a separate computer. This can be your normal development laptop, or a dedicated build server. Ideally, you compile the APK on multiple machines (such as your developer laptop and the CI pipeline) and compare the checksums to make sure that they match using Reproducible Builds.

Then you can copy the unsigned APK to the offline computer (via USB stick, or by temporarily attaching it to a network and using rsync). Before signing, you can again verify the checksum to make sure your that are about to sign the expected file.