This list will be updated and expanded upon when the mood strikes.
preenix: pre-evaluated nix search and nix installSome of Consequence 1 and all of Consequence 2 of current Nixpkgs metrics can be solved by making a separate tool, or adding a command to nix, to search and install from pre-evaluated derivation files.
Basically, you do nix-env -f . -qaP --argstr system "x86_64-linux" on a snapshot of Nixpkgs, then nix-instantiate each of the attributes it produces take the closure of all the derivation files. Then you make a DB mapping from package attribute names to derivation file names and their .meta attributes. And then pack both the closure and the DB it into an archive you then distribute a-la nix-channel.
Then, on the client, you just use the mapping to immediately get the corresponding derivation file name without evaluating anything, use the closure from archive file as a binary cache, and just straight do nix-store --realise on the derivation in question.
Newbies in https://github.com/NixOS/nixpkgs/issues/38635 complain about Consequence 2 a lot, so solving it would be beneficial in lowering the entry barrier to NixOS, but I’m sure this would be that interesting otherwise. If you are using NixOps or similar, you probably have all your packages in environment.systemPackages and don’t really care about this at all.
callPackages-splicingCompletely solving the whole thing (including Consequence 1) in upstream Nixpkgs without an incompatible fork is probably not realistic as upstream (frustratingly) seems to move in the opposite direction, most recently: https://github.com/NixOS/nixpkgs/pull/237439#pullrequestreview-1535393840
And, even if it did not, adding more abstraction layers to NixOS will not help.
But, for the pkgs part of Nixpkgs, the best approach to cut evaluation memory requirements by about 60% is to replace overlays and overrides with callPackage-splicing https://github.com/NixOS/nixpkgs/pull/56227 and https://github.com/NixOS/nixpkgs/pull/56110.
But it got a lot of push-back when I proposed it… And it would still take quite a bit of memory to evaluate configuration.nix files of NixOS as that won’t get any more efficient…
I have some of the patches required to do it in my SLNOS (which is a private Suckless desktop-oriented NixOS fork I participate in) branch, but this would need a lot of maintenance to support, as they conflict with NixOS/master all the time. So probably not viable yet.
stdenv.mkDerivation hackingAlso, I feel like a lot can be gained by making stdenv.mkDerivation more efficient. Memory consumption was much better before cross-compilation infra was added to stdenv.mkDerivation.
Some ideas how to improve things:
I suspect that adding strategic strict-evaluation in some places there, stripping cross-compilation stuff when it is not used, and ensuring the resulting attrset does not contain superfluous nulls, empty strings, and empty lists could help a lot.
I did some very simple experimental changes and they tend to give 1-3% performance improvements immediately, so there is certainly stuff to gain here. For instance:
diff --git a/lib/customisation.nix b/lib/customisation.nix
index a9281b1ab698..e21ff4dc517e 100644
--- a/lib/customisation.nix
+++ b/lib/customisation.nix
@@ -195,12 +195,11 @@ rec {
the derivation itself and check a given condition when evaluating. */
extendDerivation = condition: passthru: drv:
let
outputs = drv.outputs or [ "out" ];
- commonAttrs = drv // (builtins.listToAttrs outputsList) //
- ({ all = map (x: x.value) outputsList; }) // passthru;
+ commonAttrs = drv // (builtins.listToAttrs outputsList) // passthru;
outputToAttrListElement = outputName:
{ name = outputName;
value = commonAttrs // {
inherit (drv.${outputName}) type outputName;But I have no idea how much time it would take, nor how much is to be gained there.
I can do some experiments if somebody is interested to sponsor this, however.
From https://sta.li/:
stali is a static Linux distribution based on the original pre-2010 plans of the suckless.org project
Basically, it is a Linux distro built statically with musl libc. This results in tiny very fast binaries. See https://sta.li/faq/ for more info.
Upstream has pkgsMusl, but it uses plain pkgs with a some work (and then a lot of work trying to push the changes upstream, I suspect) it is be possible to allow Nixpkgs to be optionally built with musl by default, thus getting the combined advantages of stali and Nixpkgs/NixOS.
With some more work put into splitting packages into multiple outputs, adding automatic disk compression for all documentation, and etc, 128MiB and less on-disk and in-RAM web server instances should totally be possible.
NixOS re-generates bootloader (GRUB, etc) menus on configuration switches, meaning it is easy to have it become outdated w.r.t. /nix/var/nix/profiles, like after a garbage collection.
Also, NixOS wants to install its GRUB config to /boot, meaning you can’t use NixOS in combination with other distros.
What NixOS should do instead is just generate a JSON file in /run/current-system describing available boot paths (kernel + initrd, xen, etc) and bootloader generation should be split into a separate thing (with backward-compatibility layer for old NixOS configs), a-la update-grub of Debian, thus keeping grub.cfg always up to date.
This would also make it easy to integrate NixOS into other distros by generating NixOS GRUB menu into a separate file included from target distro’s grub.cfg.
Combined with NixOS-install-via this also provides a non-destructive alternative to destructively installing NixOS over an existing host system. Basically, you could just install nix on your host distro, do deploy-via-ssh from your evaluation machine into a chroot on the target, generate a grub.cfg for it, and include it in your host one.
You can now boot into it.
This will also allow simplify the use of NixOS under things like Heads firmware.
NixOS needs a service abstraction layer. A-la https://github.com/NixOS/nixpkgs/pull/5246 and https://github.com/NixOS/nixpkgs/issues/26067
It just needs to be implemented.
It would allow you to run NixOS services on other distros: similarly to how you can install and use a package installed via nix in Debian, you could then also install a NixOS service under Debian and then symlink it to Debian’s systemd service folder without the need for third-party tools like https://github.com/numtide/system-manager. Magic!
If the abstraction layer is abstract enough, replacing systemd with another init system in NixOS would also be fairly easy.
Also, with a good abstraction layer, it would be easy to essentially erase the differences between normal NixOS and containered NixOS configs (the user will just ask a different service build output type), simplyfing things for complex configuration.nix’es greatly.
NixOS service configurations are not really that composable.
Like, in the very first NixOps example above, if you are just starting out and/or if your app has periods of very low and very high usage, you probably want to only rent a one or two instances on “low tide” and the above config will require at least four.
Well, you could make it more modular by splitting the service definitions away from NixOS configs and then generate less fine-grained NixOS configs for low tides, but then the problem is that NixOS can’t do multi-instance services, so nginx will have to be configured differently, or you will need to set up NixOS containers…
This is a bit messy.
As described in https://nixos.org/nix-dev/2014-December/015362.html and https://nixos.org/nix-dev/2014-December/015381.html this just needs to be implemented.
See the hypothetical NixOS config examples there, they are very cute.