Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aristanetworks/nix-serve-ng

A drop-in replacement for nix-serve that is faster and more reliable
https://github.com/aristanetworks/nix-serve-ng

Last synced: about 2 months ago
JSON representation

A drop-in replacement for nix-serve that is faster and more reliable

Awesome Lists containing this project

README 

        
# `nix-serve-ng`



`nix-serve-ng` is a faster, more reliable, drop-in replacement for `nix-serve`.



## Quick start



There are three main approaches you can use to configure a NixOS system to replace

the old `nix-serve` with `nix-serve-ng`:



- **A**: Set `services.nix-serve.package = pkgs.nix-serve-ng;` in your NixOS configuration

  - `nix-serve-ng` is [packaged in nixpkgs](https://search.nixos.org/packages) already

  - There is no need to consume this repository directly

- **B**: Include `nix-serve-ng.nixosModules.default` in your NixOS configuration

  - `nix-serve-ng` refers to this repository being a flake input

  - Requires consume this repository / this flake

  - Overlays `pkgs.nix-serve` with `pkgs.nix-serve-ng`

- **C**: Like **B** but not requiring a flake



We recommend approach **A**. Only use **B** or **C** if you need a bleeding edge

upstream version of the project.



### Variant A: 



_The code snippet below shows a `flake.nix`._



```nix

{ 

  inputs.nixpkgs.url = "github:NixOS/nixpkgs";



  outputs = { nixpkgs, ... }: {

    nixosConfigurations.default = nixpkgs.lib.nixosSystem {

      modules = [

        /* ... */

        { 

          services.nix-serve.enable = true;

          services.nix-serve.package = pkgs.nix-serve-ng;

          /* ... */

        }

        /* ... */

      ];

    };

  };

}

```



### Variant B: 



_The code snippet below shows a `flake.nix`._



```nix

{ 

  inputs.nixpkgs.url = "github:NixOS/nixpkgs";

  inputs.nix-serve-ng.url = "aristanetworks/nix-serve-ng";



  outputs = { nixpkgs, nix-serve-ng, ... }: {

    nixosConfigurations.default = nixpkgs.lib.nixosSystem {

      modules = [

        nix-serve-ng.nixosModules.default

        /* ... */

        { 

          services.nix-serve.enable = true;

          /* ... */

        }

        /* ... */

      ];

    };

  };

}

```



### Variant C: 



_The code snippet below shows a NixOS module file._



```nix

{ config, pkgs, lib, ... }:



let 

  nix-serve-ng-src = builtins.fetchTarball {

    # Replace the URL and hash with whatever you actually need

    url    = "https://github.com/aristanetworks/nix-serve-ng/archive/1937593598bb1285b41804f25cd6f9ddd4d5f1cb.tar.gz";



    sha256 = "1lqd207gbx1wjbhky33d2r8xi6avfbx4v0kpsvn84zaanifdgz2g";

  };



  nix-serve-ng = import nix-serve-ng-src;

in

{ 

  /* ... */

  imports = [ nix-serve-ng.nixosModules.default ];

  

  config = {

    services.nix-serve.enable = true;

  };

  /* ... */

}

```



## Motivation



Our requirements for this project were:



* Improve reliability



  … since `nix-serve` would intermittently hang and require restarts



* Improve efficiency



  … since `nix-serve` was doing some obviously inefficient things which we

  felt we could improve upon



* Be backwards-compatible



  Our replacement would need to be a drop-in replacement for the original

  `nix-serve`, supporting the same command-line options and even sharing the

  same executable name



  The only exception is logging: we provide more detailed logging than before



Did we satisfy those requirements?



## Results



* Reliability



  We have test-driven this internally under heavy load with stable memory

  usage and without any failures but it's probably premature to declare victory.



  In particular, we have *not* done the following things:



  * Memory leak detection



    In other words, we haven't put our `nix-serve` through, say, `valgrind`



  * Exploit detection



    In other words, we haven't attempted to crash or sabotage the service with

    maliciously-crafted payload



* Performance



  We have improved significantly on efficiency, not only compared to `nix-serve`

  but also compared to other `nix-serve` rewrites.  We are more efficient than:



  * The original `nix-serve`



  * [`eris`](https://github.com/thoughtpolice/eris) - A Perl rewrite of

    `nix-serve`



  * [`harmonia`](https://github.com/nix-community/harmonia) - A Rust rewrite

    of `nix-serve`



  See the Benchmarks section below for more details



* Backwards-compatibility



  We have excellent backwards-compatibility, so in the vast majority of cases,

  you can simply replace `pkgs.nix-serve` with `pkgs.nix-serve-ng` and make no

  other changes.



  * Our executable shares the same name (`nix-serve`) as the original program



  * We support most the original command-line options



    The options that we're aware of that we do not currently support fall into

    two categories:



    * Useless options which are only relevant to `starman`:



      Upon request, we can still parse and ignore the following irrelevant

      options for extra backwards compatibility:



      * `--workers`



        We do not use worker subprocess like `starman` does.  Instead we use

        `warp` which internally uses Haskell green threads to service a much

        larger number of requests with less overhead and lower footprint when

        idle.



      * `--preload-app`



        This optimization is meaningless for a compiled Haskell executable.



      * `--disable-proctitle`



    * Useful options



      We might accept requests to support the following options, but we might

      explore other alternatives first before supporting them:



      * `--max-requests`



        `warp` itself is unlikely to be a bottleneck to servicing a large number

        of requests but there may still be Nix-specific or disk-specific

        reasons to cap the number of requests.



      * `--disable-keepalive`



      * `--keepalive-timeout`



      * `--read-timeout`



      * `--user`



      * `--group`



      * `--pid`



      * `--error-log`



  Because of this backwards-compatibility you only need to replace the old

  `nix-serve` executable with the `nix-serve` executable built by this package

  (which is what the included NixOS module does).



  You don't need to define or use any new NixOS options.  You continue to use

  the old `services.nix-serve` options hierarchy to configure the upgraded

  service.



## Benchmarks



The test environment is a large server machine:



* CPU: 24 × Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz

* RAM: 384 GB (24 × 16 GB @ 2133 MT/s)

* Disk (`/nix/store`): ≈4 TB SSD



Legend:



* **Fetch present NAR info ×10**: Time to fetch the NAR info for 10 different files that are present

* **Fetch absent NAR info ×1**: Time to fetch the NAR info a single file that is absent

* **Fetch empty NAR ×10**: Time to fetch the NAR for the same empty file 10 times

* **Fetch 10 MB NAR ×10**: Time to fetch the NAR for the same 10 MB file 10 times



Raw numbers:



| Benchmark                  | `nix-serve`      | `eris`           | `harmonia`       | `nix-serve-ng`   |

|----------------------------|------------------|------------------|------------------|------------------|

| Fetch present NAR info ×10 | 2.09 ms ± 66  μs | 41.5 ms ± 426 μs | 1.57 ms ±  91 μs | 1.32 ms ±  33 μs |

| Fetch absent NAR info ×1   | 212  μs ± 18  μs | 3.42 ms ± 113 μs | 139  μs ±  11 μs | 115  μs ± 6.2 μs |

| Fetch empty NAR ×10        | 164  ms ± 8.5 ms | 246  ms ±  20 ms | 279  ms ±  10 ms | 5.16 ms ± 368 μs |

| Fetch 10 MB NAR ×10        | 291  ms ± 8.7 ms | 453  ms ±  19 ms | 487  ms ±  41 ms | 86.9 ms ± 3.0 ms |



Speedups (compared to `nix-serve`):



| Benchmark                  | `nix-serve`      | `eris`           | `harmonia`       | `nix-serve-ng`   |

|----------------------------|------------------|------------------|------------------|------------------|

| Fetch present NAR info ×10 | 1.0              | 0.05             | 1.33             |  1.58            |

| Fetch absent NAR info ×1   | 1.0              | 0.06             | 1.53             |  1.84            |

| Fetch empty NAR ×10        | 1.0              | 0.67             | 0.59             | 31.80            |

| Fetch 10 MB NAR ×10        | 1.0              | 0.64             | 0.60             |  3.35            |



We can summarize `nix-serve-ng`'s performance like this:



* Time to handle a NAR info request: ≈ 100 μs 

* Time to serve a NAR: ≈ 500 μs + 800 μs / MB



You can reproduce these benchmarks using the benchmark suite.  See the

instructions in [`./benchmark/Main.hs`](./benchmark/Main.hs) for running your

own benchmarks.



Caveats:



* We haven't used any of these services' tuning options, including:

  * Tuning garbage collection (for `nix-serve-ng`)

  * Tuning concurrency/parallelism/workers

* We haven't benchmarked memory utilization