{"id":35827656,"url":"https://github.com/rackslab/FireHPC","last_synced_at":"2026-01-14T03:00:49.526Z","repository":{"id":105074479,"uuid":"429770403","full_name":"rackslab/FireHPC","owner":"rackslab","description":"Instantly fire up container-based emulated HPC cluster","archived":false,"fork":false,"pushed_at":"2026-01-02T17:01:28.000Z","size":421,"stargazers_count":20,"open_issues_count":13,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-01-09T02:47:30.305Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rackslab.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-19T11:17:21.000Z","updated_at":"2026-01-04T05:58:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"f3f61e11-2e81-481b-b0e9-5d87295c8568","html_url":"https://github.com/rackslab/FireHPC","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/rackslab/FireHPC","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rackslab%2FFireHPC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rackslab%2FFireHPC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rackslab%2FFireHPC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rackslab%2FFireHPC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rackslab","download_url":"https://codeload.github.com/rackslab/FireHPC/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rackslab%2FFireHPC/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T01:52:23.358Z","status":"online","status_checked_at":"2026-01-14T02:00:06.678Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-07T21:00:32.148Z","updated_at":"2026-01-14T03:00:49.510Z","avatar_url":"https://github.com/rackslab.png","language":"Python","funding_links":[],"categories":["Resources"],"sub_categories":["Other/Wikis"],"readme":"# FireHPC: Instantly fire up container-based emulated HPC cluster\n\n## Description\n\nFireHPC is a tool designed to quickly deploy a tiny emulated HPC cluster based\non Linux containers, ready to run non-intensive MPI jobs with\n[Slurm](https://slurm.schedmd.com/overview.html).\n\nObviously, FireHPC does not aim at performances as you get better performances\nout of your computer without containers overhead.\n\nThe purposes are the following:\n\n- Setup development, CI and tests environments\n- Learn tools and software in breakable environment\n- Testing and discovering new technologies in isolated environment\n\nFireHPC aims to emulate HPC clusters with multiple distributions. It supports\nrunning multiple emulated HPC clusters in parallel on the same host, each\ncluster running in its dedicated virtual network.\n\nThe following services are automatically deployed in the emulated cluster:\n\n- Slurm workload manager\n- SlurmDBD accounting with MariaDB backend\n- Slurmrestd REST API\n- LDAP directory with TLS\n- SSH with public keys\n- OpenMPI\n\nAdditional components can be also be deployed, such as:\n\n- [Slurm-web](https://slurm-web.com)\n- Metrics stack with Prometheus, Grafana and Grafana Alloy\n\n## Architecture\n\nFireHPC requires Python \u003e= 3.9.\n\nFireHPC relies on:\n\n- [RacksDB](https://github.com/rackslab/RacksDB) to get emulated clusters\n  description.\n- [`systemd-nspawn`](https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html)\n  containers controlled with `systemd-{networkd,machined}`\n- [Ansible](https://docs.ansible.com/ansible/latest/index.html) automation tool\n\n## Quickstart\n\nDownload and install Rackslab packages repository signing keyring:\n\n```console\n$ curl -sS https://pkgs.rackslab.io/keyring.asc | gpg --dearmor | sudo tee /usr/share/keyrings/rackslab.gpg \u003e /dev/null\n```\n\nCreate `/etc/apt/sources.list.d/rackslab.sources`:\n\n* For Debian 12 _Bookworm_:\n\n```\nTypes: deb\nURIs: https://pkgs.rackslab.io/deb\nSuites: bookworm\nComponents: main\nArchitectures: amd64\nSigned-By: /usr/share/keyrings/rackslab.gpg\n```\n\n* For Debian 13 _Trixie_:\n\n```\nTypes: deb\nURIs: https://pkgs.rackslab.io/deb\nSuites: trixie\nComponents: main\nArchitectures: amd64\nSigned-By: /usr/share/keyrings/rackslab.gpg\n```\n\n* For Debian 14 _Forky_:\n\n```\nTypes: deb\nURIs: https://pkgs.rackslab.io/deb\nSuites: forky\nComponents: main\nArchitectures: amd64\nSigned-By: /usr/share/keyrings/rackslab.gpg\n```\n\n* For Debian _sid_:\n\n```\nTypes: deb\nURIs: https://pkgs.rackslab.io/deb\nSuites: sid\nComponents: main\nArchitectures: amd64\nSigned-By: /usr/share/keyrings/rackslab.gpg\n```\n\nUpdate packages database and install `firehpc`:\n\n```console\n$ sudo apt update \u0026\u0026 sudo apt install firehpc\n```\n\nAdd your user in `firehpc` system group:\n\n```console\n$ sudo usermod -a -G firehpc ${USERNAME}\n```\n\nImport [hpck.it](https://hpck.it/) repository keyring to verify container images\nsignatures:\n\n```console\n$ curl -s https://hpck.it/keyring.asc | \\\n  sudo gpg --no-default-keyring --keyring=/etc/systemd/import-pubring.gpg --import\ngpg: key F2EB7900E8151A0D: public key \"HPCk.it team \u003ccontact@hpck.it\u003e\" imported\ngpg: Total number processed: 1\ngpg:               imported: 1\n```\n\nEnable and start `systemd-networkd` service:\n\n```\n$ sudo systemctl enable --now systemd-networkd.service\n```\n\nIt is also recommended to increase maximum inotify instances from default 128 to\n1024 for instance to avoid weird issues when starting a large number of\ncontainers:\n\n```console\n# sysctl fs.inotify.max_user_instances=1024\n```\n\nWithout this modification, the `mymachines` service is basically ignored by the\n_return_ action on `resolve` service. For reference, see `nss-mymachines(8)`.\n\nThis can be made persistent with:\n\n```console\n# echo fs.inotify.max_user_instances=1024 \u003e /etc/sysctl.d/99-firehpc.conf\n```\n\n## Usage\n\n### Bootstrap\n\nFirst, bootstrap deployment environments:\n\n```console\n$ firehpc bootstrap\n```\n\n\u003e [!NOTE]\n\u003e This command creates multiple Python virtual environments with all versions of\n\u003e Ansible required for all supported target OS.\n\n### Deploy\n\nFor a quick start, copy the simple example RacksDB database:\n\n```console\n$ cp /usr/share/doc/firehpc/examples/db/racksdb.yml racksdb.yml\n```\n\nThis file can optionally be modified to add nodes or change hostnames.\n\nWith your regular user, run FireHPC with a cluster name and an OS in arguments.\nFor example:\n\n```\n$ firehpc deploy --db racksdb.yml --cluster hpc --os debian13\n```\n\nThe available OS are reported by this command:\n\n```\n$ firehpc images\n```\n\n### Status\n\nWhen it is deployed, check the status of the emulated cluster:\n\n```\n$ firehpc status --cluster hpc\n```\n\nThis reports the started containers and the randomly generated user accounts.\n\n### MPI\n\nYou can connect to cluster with this command:\n\n```\n$ firehpc ssh hpc\n```\n\nConnect with a generated user account on the login node:\n\n```\n$ firehpc ssh \u003cuser\u003e@login.hpc\n```\n\nThen run MPI job in Slurm job:\n\n```\n[\u003cuser\u003e@login ~]$ curl --silent https://raw.githubusercontent.com/mpitutorial/mpitutorial/gh-pages/tutorials/mpi-hello-world/code/mpi_hello_world.c -o helloworld.c\n[\u003cuser\u003e@login ~]$ export PATH=$PATH:/usr/lib64/openmpi/bin  # required on rocky8, not on debian11\n[\u003cuser\u003e@login ~]$ mpicc -o helloworld helloworld.c\n[\u003cuser\u003e@login ~]$ salloc -N 2\nsalloc: Granted job allocation 2\n[\u003cuser\u003e@login ~]$ mpirun helloworld\nHello world from processor cn1.hpc, rank 0 out of 4 processors\nHello world from processor cn1.hpc, rank 1 out of 4 processors\nHello world from processor cn2.hpc, rank 2 out of 4 processors\nHello world from processor cn2.hpc, rank 3 out of 4 processors\n```\n\nYou can also try Slurm REST API:\n\n```\n[\u003cuser\u003e@login ~]$ export $(scontrol token)\n[\u003cuser\u003e@login ~]$ curl -H \"X-SLURM-USER-NAME: ${USER}\" -H \"X-SLURM-USER-TOKEN: ${SLURM_JWT}\" http://admin:6820/slurm/v0.0.39/nodes\n```\n\n### Slurm-web\n\nWhen Slurm-web is enabled, it is available at: http://admin.hpc/\n\n### Metrics\n\nWhen the metrics stack is enabled, Grafana is available at:\nhttp://admin.hpc:3000/\n\nGrafana is setup with a _Slurm_ dashboard showing diagrams of nodes states and\njob queue by default.\n\n### Clean\n\nWhen you are done, you can clean up everything for a cluster with this command:\n\n```\n$ firehpc clean --cluster hpc\n```\n\n## Authors\n\nFireHPC is developed by [Rackslab](https://rackslab.io).\n\n## License\n\nFireHPC is distributed under the terms of the GNU General Public License v3.0 or\nlater (GPLv3+).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frackslab%2FFireHPC","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frackslab%2FFireHPC","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frackslab%2FFireHPC/lists"}