Deploy software easily and securely using nix-deploy
nix-deploy
is a small utility for easily and securely deploying software from
one nix store into another nix store. With nix-deploy
you can copy a program
and its run-time dependencies to another computer; you can also copy an entire
operating system configuration and activate it.
This utility will do the following things for you:
- Generate a pair of signing keys on the remote computer if none exist
- Exchange the signing keys with the local computer
- Deploy the closure using
nix-copy-closure
- If the closure is a system configuration, activate it on the remote computer
using
switch-to-configuration
This post assumes familiarity with Nix, the Nix store, and nix-copy-closure
. A
refresher on these topics is in Appendix A.
Motivation
We developed nix-deploy
because:
nix-copy-closure
doesn’t exchange signing keys for the user; signing keys are important because the signature of a deployed closure verifies the closure’s integrity (i.e. that it is tamper or corruption free)- We needed the system configuration deploy functionality of NixOps, without the overhead of NixOps
The solution to key exchange
nix-copy-closure
works perfectly well if:
- A cryptographic signature of the closure is not required
- A cryptographic signature of the closure is required and the signing keys are already exchanged
However, nix-copy-closure
is cumbersome if a cryptographic signature is
required but signing keys do not exist or have not yet been exchanged.
nix-deploy path
solves this problem by assuming that signing is desired,
generating signing keys and exchanging them as needed, before running
nix-copy-closure
on the user’s behalf.
For example we can easily and securely deploy the hello-2.10.drv
build product
to remote-host
:
$ nix-instantiate '<nixpkgs>' --attr hello
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
$ nix-store \
--realise \
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
/nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
$ nix-deploy path \
--to parnell@remote-host \
--path \
/nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
[+] Downloading: /etc/nix/signing-key.sec
[+] Installing: /etc/nix/signing-key.sec
This will prompt you for your `sudo` password
[sudo] password for parnell:
[+] Downloading: /etc/nix/signing-key.pub
[+] Installing: /etc/nix/signing-key.pub
[+] Copying /nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
copying 2 missing paths (20.00 MiB) to ‘parnell@remote-host’...
… we can run the hello
executable on remote-host
:
$ ssh parnell@remote-host \
/nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10/bin/hello
Hello, world!
The solution to NixOps-free deployment
NixOps does three things:
- instantiate and manage resources that will host and run a system configuration defined in the deployment specification for NixOps
- build the system configuration and software defined in the deployment specification for NixOps
- deploy the system configuration to the instantiated resources and activate it
A “system configuration” is a bootable NixOS system configuration; Appendix B provides more detail.
nix-deploy system
subsumes NixOps by extracting the deployment and activation
parts. Since we can securely deploy any nix store path to a remote computer,
nothing prevents us from deploying a build product for a complete system
configuration.
nix-deploy
simply copies the closure to the target computer, and then executes
the switch-to-configuration
script found within the build product of the
system configuration it deployed.
The switch-to-configuration
script switches the target computer’s system to
the system configuration of which the script is a product. This is the exact
same mechanism used by nixos-rebuild switch
.
Finally, we can test the activation of a system configuration via the
--dry-activate
option:
$ nix-deploy system \
--to parnell@remote-host \
--dry-activate \
--path \
/nix/store/vrg7l1zxih48m2k88fdg1byld72lrjcg-nixos-system-unnamed-17.09.2182
these paths will be fetched (0.97 MiB download, 3.19 MiB unpacked):
/nix/store/3canvs63nkqcqiqiv6mzj0j5g4rawb44-libressl-2.5.5-bin
/nix/store/c1xj32krgc7d6sc6pqzzkcgxyb5a16sd-libressl-2.5.5
...
[+] Installing system: /nix/store/vrg7l1zxih48m2k88fdg1byld72lrjcg-nixos-system-unnamed-17.09.2182
copying 412 missing paths (523.49 MiB) to ‘34.201.68.87’
...
Conclusion
A key design goal of Nix is to make building software as pure and reproducible as possible. Purity is useful because we can trust that when we ask for the closure of a build product we get everything required to run that product. Reproducibility is useful because we can trust that we can (a) detect if we are missing some dependency in the graph at evaluation-time instead of run-time and (b) retrieve a cached binary build product satisfying that dependency (or else build it). This means we can reliably build and securely deploy binary artifacts across systems without use of containers or virtual machines, provided that the source and target computers are compatible.
Nix provides all of the fundamentals to accomplish this and nix-deploy
adds
some convenience around them.
You can find nix-deploy
on
Hackage and
Github.
Appendix A
There are a couple of important things to know when deploying software with Nix.
First, Nix comes with a utility named nix-copy-closure
; it is useful for
copying the closure of a path in a nix store to another nix store.
A path in the nix store can be:
- A derivation instantiated with
nix-instantiate
$ nix-instantiate '<nixpkgs>' --attr hello → /nix/store/*-hello-2.10.drv
- A build product from the evaluation of a derivation
$ nix-store --realise /nix/store/*-hello-2.10.drv → /nix/store/*-hello-2.10
Second, a closure of a path in the nix store is the graph of that path’s
dependencies. You obtain the closure for a path in the nix store using the
nix-store
utility.
For example, if we want to obtain the closure for a path in the nix store that is a derivation, we can:
$ nix-instantiate '<nixpkgs>' --attr hello
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
$ nix-store \
--query \
--requisites \
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
/nix/store/0jmw4ra9acm0d8n0vbxkrwryicj47yss-cc-wrapper.sh
/nix/store/3q8hxw1ysf60vr9iivzh865ldf5wnb0c-utils.sh
...
Obtaining the closure for a build product is just as straightforward:
$ nix-instantiate '<nixpkgs>' --attr hello
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
$ nix-store \
--realise /nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
/nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
$ nix-store \
--query \
--requisites /nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
/nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49
/nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
Therefore we have two types of deployment in Nix: source deployment, which
corresponds with copying the closure of a nix store path that is a derivation;
and, binary deployment, which corresponds to copying the closure of a nix
store path that is a build product.
nix-copy-closure
deploys both types; assuming the target computer has Nix
installed and SSH credentials configured. Note that nix-copy-closure
only
copies what is missing of the closure on the target computer.
We can perform a source deployment of the hello
package and its closure to a
computer addressed at remote-host
and a user parnell
configured with working
SSH credentials:
$ nix-instantiate '<nixpkgs>' --attr hello
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
$ nix-copy-closure \
--to parnell@remote-host /nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
… and we can perform a binary deployment of the hello
package and its
closure to the same remote computer:
$ nix-instantiate '<nixpkgs>' --attr hello
/nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
$ nix-store \
--realise /nix/store/yx6vm61402bxfpx7z3yxq7r1zmv7cqmy-hello-2.10.drv
/nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
$ nix-copy-closure \
--to parnell@remote-host /nix/store/1y6ckg6khrdsvll54s5spcmf3w6ka9k4-hello-2.10
Thus, we can deploy anything that is a derivation or a build product of a
derivation in the nix store to another computer without ever thinking about the
dependencies!
This means that we can deploy a build recipe and its closure (all of its build-time dependencies) to another computer, build it on that computer, and deploy the build product and its closure (all of its run-time dependencies) back to the original computer.
This is the idiomatic mechanism for using Nix to deploy anything built with Nix, using Nix.
Appendix B
A system configuration is a bootable NixOS-based computer operating system with a kernel, filesystem specification, init system (both default init scripts and user-specific init scripts), and user-land software.
With NixOS we can easily construct a minimal system configuration for an EC2 machine and deploy that system configuration to an EC2 instance.
$ cat minimal-ec2-nixos.nix
let
nixos-ec2 = import <nixpkgs/nixos> {
system = "x86_64-linux";
configuration = {
imports = [
<nixpkgs/nixos/modules/virtualisation/amazon-image.nix>
];
ec2.hvm = true;
users.users.parnell = {
group = "users";
extraGroups = [
"wheel" "disk" "audio" "video" "vboxusers"
"networkmanager" "systemd-journal"
];
};
};
};
in
nixos-ec2.system
… which we can instantiate:
$ nix-instantiate minimal-ec2-nixos.nix
/nix/store/y3aalvgw4v62f5w0hy2vlaz91ynmp3kf-nixos-system-unnamed-17.09.2182.drv
… and build:
$ nix-store \
--realise \
/nix/store/bfzijb8xsppldrj814hkvj7swij6xjd9-nixos-system-nixos-17.09.2182.drv
these derivations will be built:
/nix/store/10lmxlmrbvr7k26l133jr1mwjxjfv74y-etc-nixos.conf.drv
/nix/store/h7jc20dvz1qk3h3jxx0wwp7p9nlkcp7p-grub-config.xml.drv
/nix/store/1m56czmckmmy4mwpgsisqiff8wc1cijq-users-groups.json.drv
/nix/store/f25wpc46yjlk30h0im8gwmni94ahpmvf-system-path.drv
...
/nix/store/vrg7l1zxih48m2k88fdg1byld72lrjcg-nixos-system-unnamed-17.09.2182
We can see that the build product of
/nix/store/bfzijb8xsppldrj814hkvj7swij6xjd9-nixos-system-nixos-17.09.2182.drv
is a NixOS Linux system configuration with a kernel, init ramdisk image, and a
systemd init system:
$ tree /nix/store/vrg7l1zxih48m2k88fdg1byld72lrjcg-nixos-system-unnamed-17.09.2182
/nix/store/vrg7l1zxih48m2k88fdg1byld72lrjcg-nixos-system-unnamed-17.09.2182
├── activate
├── bin
│ └── switch-to-configuration
├── configuration-name
├── etc -> /nix/store/cagpxljdhmrsrgwjwiq1q5y2jv28pyfv-etc/etc
├── extra-dependencies
├── fine-tune
├── firmware -> /nix/store/337bpg5m7ynry8yc0wmmwwdp8bpdqg7d-firmware/lib/firmware
├── init
├── init-interface-version
├── initrd -> /nix/store/d4j5awvlbzplzq0jl4bhxy1j46ggy7f6-initrd/initrd
├── kernel -> /nix/store/gi2bg1sdibi0d1s692cgf8k5h2p20a95-linux-4.9.65/bzImage
├── kernel-modules -> /nix/store/jiqq02yxvi637s3zrfjzmlyxiddiwk8j-kernel-modules
├── kernel-params
├── nixos-version
├── sw -> /nix/store/03fkz4ck643zx23ag5i45pgnlzqrm9b6-system-path
├── system
└── systemd -> /nix/store/cd2r3b7j655vfdnvfwci71dn4yyaxa0p-systemd-234
7 directories, 12 files
We’ve built an entire Linux system configuration from a declarative
specification. Furthermore the product of that build is a nix store path which
means we can ask for its closure:
$ nix-store \
--query \
--requisites \
/nix/store/vrg7l1zxih48m2k88fdg1byld72lrjcg-nixos-system-unnamed-17.09.2182
/nix/store/h1a1ncbkkhapzm0509plqjlfrgxw22f3-glibc-2.25-49
/nix/store/8lldk3r2hjikfmnrff5sc3alnz8y0can-libmnl-1.0.4
...
… thus, we can deploy nixos-system-unnamed-17.09.2182
and all of its runtime
dependencies!
Notes
Thanks to Joel Stanley (@intractable) and Gabriel Gonzalez (@GabrielG439) for reading drafts and providing feedback.