Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-institutional-hpc

List of tools you can use to get the most from your institutional HPC resources
https://github.com/frankier/awesome-institutional-hpc

Last synced: 5 days ago
JSON representation

  • Tools provided for you

  • Workflow managers and distributed computing frameworks

  • Fancy shell sessions

    • mosh - Drop free, low latency faked SSH sessions
    • kitty - Terminal supporting image display
    • EternalTerminal - Another approach to drop free SSH sessions
    • xxh - Bring your own shell and config to shared computing environments
    • Euporie - Jupyter notebooks in the terminal including support for image display with kitty
    • tmux - Terminal multiplexers, which allow running multiple sessions and detatching and reattaching to terminal sessions
  • Containers

    • singreqrun - Shim to allow running programs from the host from within a Singularity container
    • Singularity - Container runtime which must be installed by your cluster administrator. It can run convert Docker images.
    • udocker - Docker compatible CLI, which does not require action from your cluster administrator, but can use Singularity as a backend if it's available.
    • singreqrun - Shim to allow running programs from the host from within a Singularity container
  • SLURM wrappers and libraries

    • yaspi - A Python-based SLURM wrapper based on job arrays
    • SEML - SEML: Slurm Experiment Management Library. Integration of the sacred experiment manager and SLURM.
    • pyslurm - Python library for interfacing with SLURM
    • slurmpy - Small Python library for starting SLURM jobs
    • stubl - Utilities for SLURM including wrappers for job launching and reporting
  • Networking

    • sshuttle - This program can help if you need to connect from outside of campus and the VPN doesn't get you where you want.
    • lsyncd - A program to sync your files to a remote server, e.g. a HPC login node. This allows a workflow where you can develop and run smoke tests locally, and then seemlessly switch to HPC for full runs.
  • Guides