awesome-high-performance-computing
A curated list of awesome high performance computing resources
https://github.com/trevor-vincent/awesome-high-performance-computing
Last synced: 12 days ago
JSON representation
-
Resources
-
Other/Wikis
- Setting the record straight: What is HPC?
- Atomic operations and contention
- A concurrency cost hiearchy
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklog
- Best practices for machine learning with HPC
- How to pick the right hardware for AI - Gigabyte - Part 1
- A practitioner's guide to testing and running large GPU clusters for training generative AI models
- Hardware Acceleration of LLMs: A comprehensive survey and comparison
- The Utralscale Playbook - Training LLMs on GPU Clusters
- The Landscape of Exascale Research: A Data-Driven Literature Analysis (2020)
- On the Memory Underutilization: Exploring Disaggregated Memory on HPC Systems (2020)
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Abstract Machine Models and Proxy Architectures for Exascale Computing, 2014, Sandia National Laboratories and Lawrence Berkeley National Laboratory
- Argonne lectures on Extreme Scale Computing 2022
- Warewulf HPC Youtube Channel
- Scott Meyers: Cpu Caches and Why You Car
- Task based Parallelism and why it's awesome - Pedro Gonnet
- Optimizing a Math Expression Parser in Rust
- Slurm for dummies guide
- Build a Beowulf cluster
- Turing PI - mini PI cluster off the shelf
- VirtualOrfeo - Virtual HPC Cluster
- Building a NVIDIA Jetson Cluster
- Building your own HPC using ebay parts
- Magic Castle - Terraform modules to replicate HPC in cloud
- HPC University Careers search
- HPC wire job postings
- Gigabyte HPC Blog
- Energy HPC Conference
- Society of Research Software Engineering
- Women In HPC
- HPC Hallway
- The High Performance Computing Special Interest Group
- SigHPC
- Reddit HPC Admin Interview help
- The HPC.NRW competence network of North-Rhine-Westphalia
- Benchmarking data science: Twelve ways to lie with statistics and performance on parallel computers.
- libsc - Supercomputing library
- xbyak jit assembler
- HPCInfo by Jeff Hammond
- Rust in HPC
- Top500 Supercomputer Data Analysis
- How to build your LEGO Scafell Pike Supercomputer
- Deadlock empire - practice concurrency
- Vim Adventures
- High Performance Computing in Biomimetics Modeling, Architecture and Applications
- Supercomputers for Linux SysAdmins
- Raspberry Pi Supercomputing and Scientific Programming
- MPI with Python (free)
- HLRS Stuttgart HPC Courses
- An Introduction to Parallel Programming with MPI and Python
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Reinventing High Performance Computing: Challenges and Opportunities (2022)
- CConfidential High-Performance Computing in the Public Cloud
- Heterogeneous Computing Systems (2023)
- Myths and Legends in High-Performance Computing
- Tools for scientific computing
- High Performance Computing in Biomimetics Modeling, Architecture and Applications
- Supercomputers for Linux SysAdmins
- Interactive and Urgent HPC Challenges (2024)
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- High Performance Computing in Biomimetics Modeling, Architecture and Applications
- Supercomputers for Linux SysAdmins
- Designs, Lessons and Advice from Building Large Distributed Systems - Jeff Dean (Google)
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- High Performance Computing in Biomimetics Modeling, Architecture and Applications
- Supercomputers for Linux SysAdmins
- Performance Hints
- Performance Hints of the Week
- 1024 Cores - Dmitry Vyukov
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- High Performance Computing in Biomimetics Modeling, Architecture and Applications
- Supercomputers for Linux SysAdmins
- Raspberry Pi Supercomputing and Scientific Programming
- The Engineers Guide to C++
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- FireHPC
- hpc-wiki.info - Tutorials and articles for HPC users, developers, administrators and specific HPC systems
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Want speed? Pass by value
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Performance engineering blog
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- The Register
- CINECA Summer HPC School for Heterogeneous Computing
- CSC Summer School in High-Performance Computing
- Programming with POSIX threads
- Raspberry Pi Supercomputing and Scientific Programming
- Grokking concurrency
- Livermore training materials
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Journal of Parallel and Distributed Computing
- Rust programming language in the high-performance computing environment
- Energy vs. performance: Introducing the Z-plot - Georg Hager on visualizing energy-to-solution vs. performance to analyze core count and clock speed tradeoffs.
- International Workshop on OpenMP
- Raspberry Pi Supercomputing and Scientific Programming
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Raspberry Pi Supercomputing and Scientific Programming
- Guide into OpenMP: Easy multithreading programming for C++
- Advances in Parallel & Distributed Processing, and Applications (conference proceedings)
- Nvidia HPC benchmarks
-
-
Software
-
Trends
- Chapel - A Programming Language for Productive Parallel Computing on Large-scale Systems
- Charm++ - Parallel Programming with Migratable Objects
- Cilk Plus - C/C++ Extension for Data and Task Parallelism
- CUDA - High performance NVIDIA GPU acceleration
- DeterminedAI - Distributed deep learning
- Halide - A language for fast, portable computation on images and tensors
- HPC-X - Nvidia implementation of MPI
- ISPC - An open-source compiler for high-performance SIMD programming on the CPU and GPU
- joblib - Data-flow programming for performance (python)
- MAGMA - Next generation linear algebra (LA) GPU accelerated libraries
- Merlin - A distributed task queuing system, designed to allow complex HPC workflows to scale to large numbers of simulations
- mpi4py - Python bindings for MPI
- MPI - OpenMPI implementation of the Message passing interface
- MPI - MPICH implementation of the Message passing interface
- MPI Standardization Forum - Forum for MPI standardization
- MPAVICH - Implementation of MPI
- NCCL - The NVIDIA Collective Communication Library for multi-GPU and multi-node communication
- cuNumeric - GPU drop-in for numpy
- stdpar - GPU accelerated C++ from NVIDIA
- numba - A JIT compiler that translates a subset of Python into fast machine code
- oneAPI - A unified, multiarchitecture, multi-vendor programming model
- OpenACC - "OpenMP for GPUs"
- OpenCilk - MIT continuation of Cilk Plus
- OpenMP - Multi-platform Shared-memory Parallel Programming in C/C++ and Fortran
- PVM - Parallel Virtual Machine: A predecessor to MPI for distributed computing
- PMIX - Standard for process management
- ray - Scale AI and Python workloads from reinforcement learning to deep learning
- ROCM - First open-source software development platform for HPC/Hyperscale-class GPU computing
- RS MPI - Rust bindings for MPI
- Simgrid - Simulate cluster/HPC environments
- SkelCL - A Skeleton Library for Heterogeneous Systems
- STAPL - Standard Template Adaptive Parallel Programming Library in C++
- STLab - High-level Constructs for Implementing Multicore Algorithms with Minimized Contention
- SYCL - C++ Abstraction layer for heterogeneous devices
- The Open Community Runtime - Specification for Asynchronous Many Task systems
- Tuplex - Blazing fast python data science
- LIKWID.jl - Julia wrapper for LIKWID.
- cpuid - A software instruction available on Intel, AMD, and other processors that can be used to determine processor type and features.
- cpuid instruction note
- intel cpuinfo - Intel tool providing information about the characteristics of Intel CPUs.
- openmpi hwloc
- Flux framework - A framework for high-performance computing clusters.
- E4S - The Extreme Scale HPC Scientific Stack - A collection of open-source software packages for HPC environments.
- RADIUSS - Rapid Application Development via an Institutional Universal Software Stack.
- OpenHPC - A community-led set of HPC components.
- Slurm - A cluster management and job scheduling system for Linux clusters.
- SGE - A resource management software for large clusters of computers.
- OpenPBS - A software for workload management and job scheduling.
- Lustre Parallel File System - A high-performance distributed filesystem for large-scale cluster computing.
- GPFS - A high-performance parallel file system developed by IBM.
- Spack - A package manager for HPC/supercomputers.
- Guix - A package manager for HPC/supercomputers.
- Easybuild - A package manager for HPC/supercomputers.
- Lmod - A Lua-based module system for software environment management on HPC systems.
- xCat - A distributed computing management and provisioning tool.
- Warewulf - An operating system provisioning system and cluster management tool.
- OpenXdMod - A tool for managing high-performance computing resources.
- LSF - A batch system for HPC and distributed computing environments.
- BeeGFS - A parallel file system designed for performance-critical environments.
- fpsync - A tool for fast parallel data transfer using fpart and rsync.
- moosefs - A fault-tolerant, highly available, distributed file system.
- rocks - An open-source Linux cluster distribution.
- OpenOnDemand - A web portal for accessing supercomputing resources.
- XDMoD - An open-source tool for managing high-performance computing resources.
- Globus Connect - A fast data transfer tool between supercomputers.
- Starfish - Unstructured data management and metadata solution for files and objects.
- mOS - A specialized operating system for high-performance computing, designed to support large-scale, manycore processors.
- Kitten - A lightweight kernel designed for high-performance computing. It focuses on providing low noise and predictable performance for HPC applications.
- Apptainer (formerly Singularity) - Container platform designed for scientific and high-performance computing (HPC) environments.
- Docker - A set of platform as a service products that use OS-level virtualization to deliver software in packages called containers.
- Kubernetes - An open-source system for automating deployment, scaling, and management of containerized applications.
- HTCondor - An open-source high-throughput computing software framework.
- grpc - A high-performance, open-source universal RPC framework.
- Jacamar-ci - CI/CD tool designed for HPC and scientific computing workflows.
- Prefect - A workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine.
- Apache Airflow - A platform to programmatically author, schedule, and monitor workflows.
- snakemake - A workflow management system that reduces the complexity of creating reproducible and scalable data analyses.
- Prometheus - An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.
- nextflow - A workflow framework to deploy data-driven computational pipelines.
- Summary of C/C++ debugging tools
- ddt
- marmot MPI checker - A tool for detecting and reporting issues in MPI (Message Passing Interface) applications.
- python debugging tools - A collection of tools for debugging Python applications, including pdb and other utilities.
- Summary of profiling tools - A comprehensive list of profiling tools for performance analysis in HPC.
- Summary of code performance analysis tools
- papi - Provides standard APIs for accessing hardware performance counters available on modern microprocessors.
- scalasca - A software tool that supports performance analysis of large-scale parallel applications.
- tau - TAU (Tuning and Analysis Utilities) is a profiling and tracing toolkit for performance analysis of parallel programs.
- vampir - A tool for detailed analysis of MPI program executions by visualizing their event traces.
- NASA parallel benchmark suite - A set of benchmarks designed to evaluate the performance of parallel supercomputers.
- HPL benchmark - The High Performance Linpack Benchmark for measuring floating-point computing power of systems.
-
Programming Languages
Sub Categories
Other/Wikis
609
Trends
247
Cloud
19
Most Recent List of the Top500 Supercomputers
18
GPU
13
Interconnects/Topology
11
CPU
10
History
6
Vendors (Work in Progress)
5
Student Opportunities / Workshops
4
Custom/FPGA/ASIC/APU
4
A Few Upcoming Supercomputers
3
TPU/Tensor Cores
2
Many integrated core processor (MIC)
1
Certification
1
Keywords
hpc
16
gpu
9
python
9
mpi
7
parallel-computing
7
parallel-programming
7
cuda
7
linux
7
cpp
7
c-plus-plus
7
gpu-programming
6
machine-learning
6
rust
5
profiling
5
c
4
benchmarking
4
parallel
4
high-performance-computing
4
heterogeneous-parallel-programming
4
deep-learning
4
benchmark
4
intel
3
openmp
3
hip
3
data-science
3
compiler
3
performance-analysis
3
performance
3
cli
3
pytorch
3
slurm
3
distributed-computing
3
profiler
3
scientific-computing
3
gpu-computing
3
disk
2
simd
2
macos
2
radiuss
2
shmem
2
jit
2
instrumentation
2
header-only
2
awesome-list
2
awesome
2
heterogeneous-computing
2
terminal
2
pgas
2
nvidia-gpu
2
command-line
2