How to start using Nix

Written on 2023-04-16

This is a opinionated, work-in-progress[1] series about how to get started with Nix. It intends to cover the most common use cases for end-users, but I might also cover the packaging side later. My premise is that Nix is a useful tool for many use cases, some features being less approachable than others. I believe that by incrementally introducing Nix' features to a new user in a certain order, the learning curve can be smoothed out.

Here are some goals for this series:

Be beginner-friendly: I expect no prior knowledge about how to use Nix
Be incremental: we will start small and introduce more features as we go, with each step being uniquely useful (for some use cases anyways)
Be practical: usefulness and brevity are prioritised over correctness and completeness (though I don't intend to be misleading)
Be a starting point for further exploration: I will cover some simple, common use cases, but you are encouraged to find out more about the features that are relevant to you
Be opinionated: there are often multiple ways of doing one thing with Nix, but I will only cover what I consider to be the right approach

as well as some explicit non-goals:

I do not intend to sell you on Nix. Though I might write an article about this in the future, I assume that you are already interested in Nix for at least one use case
This is not documentation or a technical reference. Refer to the official docs for correctness and completeness
I do not aim for deep understanding. Nix is a very open tool that offers fine-grained control over its operation, but it is not required to understand it fully for it to be useful
I will not explain Nix from first principles. The whole Nix ecosystem is based on a few fundamental concepts, but I believe that introducing them first hinders the learning experience, as opposed to starting out with approachable and immediately useful features

Without further ado, let's get started.

Install Nix

Get Nix

Use whatever installation method is recommended on your platform. As of 2023-04-15, the main methods are:

Using Nix' official installer
Using your platform's official package (distributions such as Debian and Arch provide officially supported Nix integrations)
Using Determinate System's installer, which might eventually replace the official installer

For reference, I am currently using version 2.13.3. Any version greater than or equal to 2.4.0 should be adequate, but the more recent, the better.

Configure Nix

Add the following line to $HOME/.config/nix/ or /etc/nix/nix.conf:

experimental-features = nix-command flakes

This will enable the use of the unified CLI and the flakes feature.

Flakes add a standardised entry-point to Nix: all projects that use them use the same conventions, which makes Nix much more approachable. Flakes also let you pin dependencies, which lets you always get the same version of a package. Just use flakes. I won't tell you how to not use them anyways.

The unified CLI is the use of commands that look like nix build as opposed to nix-build. It has much better ergonomics, interacts better with flakes and is meant to replace the legacy nix-something commands.

Do not be scared about the experimental status of those features: most of the Nix ecosystem uses/supports them and they have been stable and reliable in practice for a long time.

Warning

You will sometimes encounter documentation or examples that use legacy (i.e. non-flakes, non-unified CLI) Nix. Try to avoid them and look for more recent information if possible. In practice, avoid anything that:

Mentions "channels"
Has expressions with angle brackets like <nixpkgs>
Uses hyphenated commands like nix-build or nix-shell

`nix run`

You can run programs at arbitrary URIs with nix run. For example:

nix run git+https://github.com/cbarrete/screech

will download the code for my screech program as well as its dependencies (the Rust toolchain), build it locally and run it. It will not be added to your PATH, but it will remain on your disk, so running the command a second time will be instantaneous. This command works because the repository contains a flake.nix file that describes how to build screech.

Some website like Github and Gitlab are popular enough that Nix has a shorthand for them. For example, the previous command is equivalent to:

nix run github:cbarrete/screech

I will use that notation going forward.

You can pass arguments to the program that you are running after --. Any arguments before that will be passed to the nix command itself. For example (requires an already existing test.wav file):

nix run github:cbarrete/screech -- interpolate test.wav

The screech repository actually contains multiple programs: the CLI that we've run so far, but also a Python script named fzf_screech that wraps the screech command with FZF, a Go fuzzy finder. This script can also be run like so:

nix run github:cbarrete/screech#fzf_screech

This will again fetch the repository with all of its dependencies, build everything and run the script. Because Python, Go and FZF are popular packages, they are available in a binary cache that Nix uses instead of building everything locally, but Nix is otherwise able to build everything from scratch.
As you might expect, running

nix run github:cbarrete/screech#screech

runs the CLI program, which is the default one when no #... is specified at the end.
As a mnemonic, I think of # as the "flake" sign because it looks a little like a snowflake and I read the previous command as "run the screech program from the github:cbarrete/screech flake.

As noted earlier, Nix can run flakes at different kinds of URIs, including local paths. In particular, nix run runs the flake in the current directory's repository.

Running individual programs out of their own repositories is nice, but there exists a repository of packages called Nixpkgs, which contains the definitions of many packages. It is where most people get their software from when using Nix:

nix run github:nixos/nixpkgs#htop
nix run github:nixos/nixpkgs#neovim
nix run github:nixos/nixpkgs#chromium

Once again, because some repositories (mostly Nixpkgs) are very popular, there exists even terser shorthands for them:

nix run nixpkgs#git
nix run nixpkgs#lf
nix run nixpkgs#ripgrep

You can list them with nix registry list and add your own to the registry for convenience. I like to add an n entry for Nixpkgs so that I can write e.g. nix run n#fd instead of

nix run
  nixpkgs#fd

nix run is a great command to get familiar with at first as it lets you run programs without polluting your environment. Nixpkgs contains more package definitions than most distributions and they are usually more up to date, and any repository with a flake.nix can easily be tried out and discarded without having to install any build dependencies manually.

`nix shell`

nix run is great, but is not always the most convenient: it is clunky to integrate in shell pipes, cannot provide commands for existing scripts without modifying them, and requires more typing than just running the command itself.
A great command for those use cases is nix shell, which opens a new shell and adds its arguments the PATH:

> fortune
-bash: fortune: command not found
> nix shell nixpkgs#fortune
> fortune
Today is the first day of the rest of your life.
> exit
exit
> fortune
-bash: fortune: command not found

As you can see, the packages are only added to the PATH in the shell created by nix shell, but remain on disk afterwards, so subsequent calls are instantaneous.

Registry pinning

As you use nix run and nix shell with Nixpkgs, you might be frustrated that they sometimes seem to fetch something from the internet and take longer to execute than expected. That's because Nix gets you the most recent packages by default: think of it as performing the equivalent of an apt update every time you run a command.

An easy fix for this is to pin the version of Nixpkgs being used with nix registry pin nixpkgs. From there on, there will be no more sporadic downloads when you issue commands that involve Nixpkgs.

`nix develop`

nix shell is nice, but it can be inconvenient to use when you want to pull in some dependencies to work on a project:

typing out the full list of packages is inconvenient, error-prone and verbose (which can be mitigated by putting the nix shell invocation in a shell script)
nix shell will get the latest version of packages by default when getting them from urls (which is the case when using the default nixpkgs flake), but you probably want to pin them as to always get a known-working environment
you might find that typing nix shell any time you enter a project and exiting that shell when you are done is too cumbersome

Registry pinning, covered earlier, can help with the second issue when using nix shell, but it is a local modification that is not shared across systems or with peers; and it requires you to either have one registry entry per dependency, per project, or be stuck with the same version across multiple projects.

nix develop can solve the first 2 issues. The third one can be solved by using tools like direnv with its Nix integration. I have no experience with direnv, but many people swear by it.

You can think of nix develop as a structured nix shell, which reads from a file which packages to install, which environment variables to setup and which setup hooks to run. This is all configured in a flake.nix file (the same kind of file that we use to describe packages, as previously mentioned).

A flake.nix file mainly contains a collection of inputs and a collection of outputs. Inputs are dependencies and outputs are anything produced (aka derived) from those inputs: packages (like screech, fzf_screech or htop), development environment (as will be shown shortly), but also many other things (like linting or formatting steps, CI job descriptions or even whole system configurations). For now, let's focus on a simple development environment example:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";

  outputs = { nixpkgs, ... }:
  let
    pkgs = nixpkgs.legacyPackages.x86_64-linux;
  in {
    devShells.x86_64-linux.default = pkgs.mkShell {
      packages = [ pkgs.jq pkgs.curl ];
    };
  };
}

I'm not going to explain everything in detail (feel free to learn about the Nix language, derivations and flake inputs and outputs if you want more detail), but here are the main ideas:

we have a single input, nixpkgs, which comes from the Nixpkgs GitHub repository, following the unstable branch
our outputs are defined by a function that takes our nixpkgs input and some more arguments that we don't care about (...) and returns a set of outputs
we have a single output: a development shell defined for x64 Linux systems, which contains jq and curl
in the process we've defined a variable pkgs that lets us access the packages and some helper functions (like mkShell) that Nixpkgs provides for x64 Linux systems
don't mind the legacyPackages bit, it is Nixpkgs-specific, as it existed long before flakes were a thing

You might dislike the verbosity, especially around having to explicitly state your supported systems, but it starts to be useful as your flake grows and there are ways to hide it. I just want to keep things simple and straightforward for now and not introduce unnecessary abstractions.

Now we can run nix develop to enter a shell containing jq and curl (Note that if you're in a git repository, you'll need to git add your flake.nix, because Nix ensures that everything that it uses is tracked). You will see that a flake.lock file was created: it contains the versions of your dependencies (in this case only Nixpkgs) so that any time you run nix develop, including on another machine, you will get exactly the same development shell.

Let's add more things to our development environment. Two useful things include setting up environment variables and running a shell script upon entry. I'll also throw in another dependency for good measure:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  inputs.screech.url = "github:cbarrete/screech";

  outputs = { nixpkgs, screech, ... }:
  let
    pkgs = nixpkgs.legacyPackages.x86_64-linux;
    screech-pkgs = screech.packages.x86_64-linux;
  in {
    devShells.x86_64-linux.default = pkgs.mkShell {
      packages = [ pkgs.jq pkgs.curl screech-pkgs.fzf_screech ];

      shellHook = "echo Hello $MY_ENV_VAR";

      MY_ENV_VAR = "world";
    };
  };
}

Sure enough, upon nix develop you will be greeted with a "Hello world" message, will have a MY_ENV_VAR variable available and access to fzf_screech.

As long as you keep your flake.lock, you will always get the same versions of packages. You can update your lockfile by running nix flake update.

Note that Nix respects its input's lockfiles. This is good, as it means that each input will have exactly the dependencies that it needs (one of the big selling points of Nix), but it also means that you might end up using more disk space and bandwidth to get multiple versions of the same package. If you want to avoid that, you can add screech.inputs.nixpkgs.follows = "nixpkgs"; to your inputs. This will override the version of Nixpkgs that screech uses e.g. to get the Rust toolchain and Python, so that it is the same as "your" Nixpkgs (the one in your lockfile).

This should be enough to get you started with building your own reproducible development environment. You can use them to remove your reliance on global system setup, eliminate the need to maintain lists of packages to install for your projects to build, ensure that you don't pollute your system over time when installing invasive packages like Tex or Tensorflow, and avoid issues with incompatible/unavailable versions. All reliably, on any system.

Garbage collection and store optimisation

After using Nix for a while, you might wonder about how you can reclaim the disk space taken by package versions that you no longer use: it is convenient that Nix caches them on disk, but you likely won't want to keep them around forever.

The answer is simple: just run nix store gc. This will go over your Nix store (the place where all Nix packages are located, at /nix/store) and remove all packages that are no longer referenced. Because we have not covered how to persist package installations yet, this will almost empty your Nix store every time you run the command. If you want a less nuclear approach, you can limit how much disk will be reclaimed like so: nix store gc --max 1G. If you want a more granular approach, you can even use nix store delete /nix/store/some-path, though this requires you to specify which version of which package you want to get rid of specifically.

While on the topic of saving disk space, a command that you can regularly use is nix store optimise. Nix will deduplicate the files in the store by hard-linking identical files. You can also set auto-optimise-store to true in your nix.conf file for Nix to optimise the store every time it installs a new package.

`nix build`

nix build is the command you use to... build packages. This is useful if you want to start writing your own packages. I'm not going to explain how to write Nix packages, as there are better resources out there for any kind of packages you might want to build.
What I will say though, is that while Nix has a generic, low-level concept of how to define a package (which it calls a derivation, because it is derived from its inputs), Nixpkgs provides lots of helpers (and the wider community many more) that make building specific packages very easy. As such, building a standard C or C++ project, a Go module or a Rust crate only takes a few lines of Nix.

A prime candidate for writing a package would be some project for which you already have a devshell but keep building manually anyway. While nix build is not necessarily the best tool for iteration (it is slower than just calling the underlying build system and can bloat up your Nix store), it is a nice single command for getting the build dependencies of a package and do everything required to build it.

The resulting package will be available in the result directory, which is a symlink to the Nix store, where the package actually lives. A convenient side-effect of this is that Nix will not garbage collect anything that this package depends on, as long as this result symlink exists. This can be (ab)used to keep packages around if you're often using nix shell or nix develop and running garbage collection.

`nix edit`

At this point, if you're writing your own derivations[2], you're probably referencing documentation and examples. You can use nix edit to open the code for a specific derivation in an editor. For example you can nix edit nixpkgs#vim or nix edit nixpkgs#firefox (a fairly trivial package and a complex one respectively) to see how they are packaged.

Home manager

Installing packages

At this point, if you have used most things outlined above, you might want to learn how to actually install software, so that you don't need to constantly run Nix commands to run common programs. The solution is what Nix calls profiles, which are versioned sets of packages, which can be installed, upgraded and rolled-back.

There is a nix profile install command, which is similar to apt install (or dnf install, pacman -Sv9arstvh, etc.), but I will not talk about it, as it has its share of caveats and does not take advantage of the declarative nature of Nix.

Instead, a more declarative option would be to write down a list of packages and let Nix automatically get the state of the system to the contents of that list by manipulating a profile. This can be implemented manually, but a more popular option is to use Home Manager, which is a utility that manages user environments using profiles.

I recommend following the Home Manager manual to install it and get a basic configuration going: follow the instructions for a standalone setup that uses flakes. Once setup, you'll be able to add whatever packages you want to the home.packages list, and run the home-manager switch command to install/uninstall the packages as required to satisfy that list.

Understanding what is going on

Home Manager can seem a little magic at first: how does the Nix configuration and the home-manager command interact, and how does everything work under the hood?

In short, Home Manager builds a custom package which (for now) mostly contains a bash script. Running this script will run the appropriate nix profile commands so that the current user profile matches what is declared in the Nix configuration. In turn, running home-manager switch basically just runs nix build .#homeConfigurations.your-user.activationPackage and then runs the script at result/bin/activate. Try it for yourself if you want!

Managing package configuration

If Home Manager generates a script that manages Nix profiles, it can generate many more things, such as configuration files for the packages in that profile. From there, it is not a big ask for the activation script to symlink them to the appropriate place, which is precisely what Home Manager does, and why so many people like it. Specifically, Home Manager enables you to have a central place of configuration for all your packages, and (for the most part) to configure them using a single language (Nix), as opposed to a mix of JSON, YAML, TOML, etc. Home Manager can even manage user services for you.

You can read about all the supported packages and their available options in the Home Manager manual. For any unsupported packages (or packages that you just want to manage manually), you can keep installing them by putting them in the home.packages list and configuring them as you did before.

NixOS

Coming soon!

[1] I will announce updates to this series via my RSS feed. Jump back

[2] Again, derivation is just another name for package definition. Just repeating that as it's a common point of confusion for beginners. Jump back