{"id":21650673,"url":"https://github.com/engineeringsoftware/python-hpc-frameworks","last_synced_at":"2025-03-20T03:33:23.462Z","repository":{"id":263813927,"uuid":"860487947","full_name":"EngineeringSoftware/python-hpc-frameworks","owner":"EngineeringSoftware","description":"List of Python frameworks for developing HPC applications","archived":false,"fork":false,"pushed_at":"2024-11-23T22:42:07.000Z","size":8705,"stargazers_count":10,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-25T05:27:22.260Z","etag":null,"topics":["hpc","python"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EngineeringSoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-20T14:25:46.000Z","updated_at":"2024-11-23T22:42:11.000Z","dependencies_parsed_at":"2024-11-20T21:52:17.116Z","dependency_job_id":null,"html_url":"https://github.com/EngineeringSoftware/python-hpc-frameworks","commit_stats":null,"previous_names":["engineeringsoftware/python-hpc-frameworks"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fpython-hpc-frameworks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fpython-hpc-frameworks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fpython-hpc-frameworks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Fpython-hpc-frameworks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EngineeringSoftware","download_url":"https://codeload.github.com/EngineeringSoftware/python-hpc-frameworks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244544068,"owners_count":20469604,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hpc","python"],"created_at":"2024-11-25T07:41:57.526Z","updated_at":"2025-03-20T03:33:23.444Z","avatar_url":"https://github.com/EngineeringSoftware.png","language":null,"readme":"## Can Python Do for HPC What It Did for Machine Learning?\n\nThis repository accopanies a [Birds of a\nFeather](https://sc24.conference-program.com/presentation/?id=bof213\u0026sess=sess633)\nsession at Supercomputing Conference 2024.\n\n```\nCan Python Do for HPC What It Did for Machine Learning?\nWednesday, 20 November 2024 12:15pm - 1:15pm EST\nLocation B212\n```\n\n### Description\n\nPython is now one of the most popular programming languages. In HPC,\nit has predominantly been used to coordinate coarse-grain library\ncomponents or workflows. However, it is increasingly being used to\ndevelop and coordinate applications with dynamic finer-grain\ncomponents that are challenging to map efficiently onto heterogeneous\nresources. In this BoF, we discuss this challenge and efforts to\ndesign Python-based HPC, production quality codes for HPC leadership\nplatforms. We will discuss issues such as multithreading, GPU kernel\ndevelopment, task-based coordination on heterogeneous systems with a\nmix of CPUs and GPUs, inter-node interoperability, scalability,\nportability, and reproducibility.\n\n\n### Program\n\n* Introduction\n* Lightening talks\n    * [Parla: HPC tasks for shared-memory heterogeneous nodes in Python](slides/Parla.pdf) (Mattan Erez)\n    * [PyCOMPSs support to HPC + AI workflows](slides/PyCOMPSs.pdf) (Rosa M. Badia)\n    * [PyKokkos: A Performance Portability Framework for Python](slides/PyKokkos.pdf) (Milos Gligoric)\n    * [Distributed Tasking in Python with Legion](slides/Legion.pdf) (Elliott Slaughter)\n    * [cuPyNumeric, Zero code change scaling of NumPy code](slides/cuPyNumeric.pdf) (Manolis Papadakis)\n* Discussion (Q\u0026A)\n\n\n### Frameworks for HPC Python development\n\nBelow is an incomplete list of framework for developing HPC\napplications in Python and brief descriptions.\n\n* [Arkouda](https://arkouda-www.github.io/) - A numpy/pandas inspired Python library backed by [Chapel](https://chapel-lang.org/)\n* [Charm4py](https://charm4py.readthedocs.io) - Charm++ programming model in Python\n* [CuPy](https://cupy.dev) - NumPy/SciPy-compatible Array Library for GPU-accelerated Computing with Python\n* [cuPyNumeric](https://developer.nvidia.com/cupynumeric) - Write NumPy, run automatically on clusters of CPUs and GPUs\n* [Dask](https://www.dask.org) - Easy parallel Python that does what you need\n* [DaCe](https://github.com/spcl/dace) - Data Centric Parallel Programming\n* [FlexFlow](https://flexflow.ai) - Drop-in PyTorch, Keras, ONNX interface\n* [lbmpy](https://pypi.org/project/lbmpy) - Run fast fluid simulations based on the lattice Boltzmann method in Python on CPUs and GPUs\n* [loopy](https://documen.tician.de/loopy) - A code generator for array-based code in the OpenCL/CUDA execution model\n* [mpi4py](https://mpi4py.readthedocs.io) - MPI for Python\n* [Numba](https://numba.pydata.org) - JIT compiler that translates a subset of Python and NumPy code into fast machine code\n* [Pallas](https://jax.readthedocs.io/en/latest/pallas/index.html) - An extension to JAX that enables writing custom kernels for GPU and TPU\n* [Parla](https://github.com/ut-parla/Parla.py) - A task-parallel programming library for Python\n* [Parsl](https://parsl-project.org) - Productive parallel programming in Python\n* [PyCOMPS](https://docs.idmod.org/projects/pycomps/en/latest) - Workflow orchestration in Python\n* [PyCUDA](https://documen.tician.de/pycuda) - Pythonic access to Nvidia's CUDA parallel computation API\n* [Pygion](https://legion.stanford.edu/pygion) - A task-based framework for Python based on the Legion programming system\n* [PyKokkos](https://github.com/kokkos/pykokkos) - Framework for writing performance portable HPC kernels in Python\n* [PyOMP](https://github.com/Python-for-HPC/PyOMP) - OpenMP for Python in Numba for CPU/GPU parallel programming\n* [PyOpenCL](https://developer.nvidia.com/pyopencl) - Lets you access GPUs and other massively parallel compute devices from Python\n* [pystencils](https://pypi.org/project/pystencils) - Run blazingly fast stencil codes on numpy arrays\n* [PyTorch](https://pytorch.org) - An open-source machine learning library based on the Torch library\n* [Ray](https://www.ray.io) - AI Compute Engine\n* [Taichi Lang](https://github.com/taichi-dev/taichi) - Imperative, parallel programming language for high-performance numerical computation\n\n### Contact\n\nIf you would like to make any addition, feel free to create a PR with\nsuggested changes or email Milos Gligoric \u003cgligoric@utexas.edu\u003e.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineeringsoftware%2Fpython-hpc-frameworks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fengineeringsoftware%2Fpython-hpc-frameworks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineeringsoftware%2Fpython-hpc-frameworks/lists"}