Contributing to nixpkgs for the first time

I’ve been using NixOS for a couple years now but only in the last month have I had the opportunity to contribute something back. In this case it’s the packaging code for Hoarder, a bookmarking app I’ve been using the last couple of months as a Pocket replacement.

Although I’m not a regular contributor to any open source project, contributing to Nixpkgs has been, by far, the least daunting. I think a big reason is contributing doesn’t require much special knowledge for people that are already using Nixpkgs. While maintaining my Nix laptop, Nix server (which runs this website) and Nix Homelab I’m writing Nix code regularly and often have to create my own derivations, or modify other derivations. This includes using the Nixpkgs standard library and, sometimes, digging into certain internal things.

A downside of Nix that I often hear cited is the learning curve, which hopefully my previous article provides some help with. Using Nix, even just as a package manager, you quickly end up looking under the hood and writing Nix code. So when you actually want to contribute you already have the tools and knowledge to do so.

Compare this to Debian where creating .deb files is documented but non-obvious, even to someone who has spent years using and maintaining Debian systems. Becoming a maintainer requires more process than just creating a pull request, and the standards tend to be quite high in terms of what packages get accepted and how they are packaged.

Here are some takeaways from the process.

You don’t have to be affiliated with upstream. I am not affiliated with the Hoarder project, from what I can tell, there’s no expectation that package maintainers be affiliated. I have never contributed to Hoarder or interacted with its community. I’m just a user who also uses Nix.

The bar to contribute is low. You can just open a pull request with the code and add yourself as a maintainer, even if you’ve never interacted with the community before. There is a contributing document everyone should read, but the guidelines are pretty lightweight by open source standards.

Split up derivations to make long builds faster, even if you’re submitting the combined one. This one’s a little more subtle. A nix derivation looks something like this,

{ stdenv }:

stdenv.mkDerivation {
  name = "large-package";
  src = pkgs.fetchgit {
    # ...
  };
  
  patchPhase = ''
    # .. lots of copying ...
  '';

  buildPhase = ''
    make build
    
    # ... complicated extra build steps ...
  '';

  installPhase = ''
    # .. lots of copying ..
  '';
}

Let’s say each of these stages is very expensive. As this package is written now each rebuild will require each stage to be re-run, even if you’re just changing the copying logic in the install phase. When developing you could write,

{ stdenv }:

stdenv.mkDerivation {
  name = "large-package";
  phases = [ "unpackPhase" "installPhase" ];
  src = stdenv.mkDerivation {
    name = "large-package-0";
    src = pkgs.fetchgit {
      # ...
    };
    patchPhase = ''
      # .. lots of copying ...
    '';
    buildPhase = ''
      make build
      
      # ... complicated extra build steps ...
    '';
    installPhase = "cp -pr --reflink=auto -- . $out";
  };
  installPhase = ''
    # .. lots of copying ..
  '';
}

And now you can just focus on the logic in the install phase, and your work in other phases is effectively cached. This derivation would be inappropriate to add to nixpkgs, but once you get it working you can modify it into a single mkDerivation.

Look for previous work. Hoarder is a NextJS application, and many of the quirks of packaging NextJS applications are already known. This knowledge can be obtained via grepping the codebase. For instance use of non-local fonts needs to be patched out because NextJS will try to download them during the install phase. Another case I ran into is running install scripts for better-sqlite3, for which there are many examples.

How to package might not be obvious. Hoarder is a monorepo that requires a number of services that need to be started to get a working web server. Additionally this monorepo has code for an iPhone and Android app, which are not packaged in the Nix package. One challenge was determining the appropriate output directory structure – I retained the build directory layout, even though it was quite messy. Because the package’s usage wasn’t immediately clear, I added helper scripts to start the necessary services and run migrations. The end result was quite different from the docker setup that’s recommended, and that Hoarder was clearly designed to run with.

People respect patience and clarity. Every non-obvious line should be documented as to why it’s there. The PR description should indicate that what you’re doing is actually useful. I think I was helped by the number of thumbs up for my PR – Hoarder is evidently a much-wanted package. Since I’ve been using a configuration on my homelab that used the package I was submitting, it was easy to provide examples and show that everything worked.