Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-institutional-hpc

List of tools you can use to get the most from your institutional HPC resources
https://github.com/frankier/awesome-institutional-hpc

Last synced: 17 minutes ago
JSON representation

  • Tools provided for you

  • Workflow managers and distributed computing frameworks

  • Fancy shell sessions

    • mosh - Drop free, low latency faked SSH sessions
    • kitty - Terminal supporting image display
    • EternalTerminal - Another approach to drop free SSH sessions
    • xxh - Bring your own shell and config to shared computing environments
    • Euporie - Jupyter notebooks in the terminal including support for image display with kitty
    • tmux - Terminal multiplexers, which allow running multiple sessions and detatching and reattaching to terminal sessions

  • Containers

    • singreqrun - Shim to allow running programs from the host from within a Singularity container
    • Singularity - Container runtime which must be installed by your cluster administrator. It can run convert Docker images.
    • udocker - Docker compatible CLI, which does not require action from your cluster administrator, but can use Singularity as a backend if it's available.
    • singreqrun - Shim to allow running programs from the host from within a Singularity container

  • SLURM wrappers and libraries

    • yaspi - A Python-based SLURM wrapper based on job arrays
    • SEML - SEML: Slurm Experiment Management Library. Integration of the sacred experiment manager and SLURM.
    • pyslurm - Python library for interfacing with SLURM
    • slurmpy - Small Python library for starting SLURM jobs
    • stubl - Utilities for SLURM including wrappers for job launching and reporting

  • Networking

    • sshuttle - This program can help if you need to connect from outside of campus and the VPN doesn't get you where you want.
    • lsyncd - A program to sync your files to a remote server, e.g. a HPC login node. This allows a workflow where you can develop and run smoke tests locally, and then seemlessly switch to HPC for full runs.

  • Guides

Programming Languages
Python 9 Shell 3 C 2 C++ 1 Lua 1 Cython 1 Groovy 1 HTML 1
Categories
Workflow managers and distributed computing frameworks 11 Fancy shell sessions 6 SLURM wrappers and libraries 5 Containers 4 Tools provided for you 2 Networking 2 Guides 1
Sub Categories
Keywords
slurm 3 snakemake 3 python 2 ssh 2 terminal 2 workflow-management 2 hpc 2 batch 1 cython 1 cluster 1 chroot 1 containers 1 utility 1 slurm-workload-manager 1 orchestration 1 hyperparameter-optimization 1 experiment-tracking 1 experiment-manager 1 lesson 1 english 1 carpentries-incubator 1 alpha 1 snakemake-profile 1 deep-hybrid-datacloud 1 bioinformatics 1 docker 1 docker-containers 1 reproducibility 1 user 1 runc 1 root-privileges 1 proot 1 indigo 1 emulation 1 grid 1 fakechroot 1 eosc-hub 1 powerlevel9k 1 shell 1 xonsh 1 xxh 1 zsh 1 euporie 1 ipynb 1 ipython 1 ipython-notebook 1 jupyter 1 jupyter-notebook 1 kitty 1 prompt-toolkit 1