https://github.com/zonca/python_hpc_tutorial
https://github.com/zonca/python_hpc_tutorial
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/zonca/python_hpc_tutorial
- Owner: zonca
- Created: 2018-12-12T21:09:23.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2022-11-19T06:20:36.000Z (over 2 years ago)
- Last Synced: 2025-03-01T04:41:37.781Z (3 months ago)
- Language: Jupyter Notebook
- Size: 4.87 MB
- Stars: 50
- Watchers: 2
- Forks: 12
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Webinar recording
https://www.sdsc.edu/Events/training/webinars/distributed_parallel_computing_with_python_2019/recording/
# Python for HPC
This webinar provides an introduction to distributed computing with Python, we will show how to modify a standard Python script to use multiple CPU cores using the concurrent.futures module from the Python standard library and then the dask package. Then we will leverage dask workers running on other machines to distribute a data processing task and monitor its execution through the live dask dashboard. You will understand the difference between threads and processes, how the Global Interpreter Lock works and principles of distributed computing. All material will be available as Jupyter Notebooks.
* `notebook_singularity.slrm`: SLURM script to launch a Jupyter Notebook job through Singularity
* `digits_of_pi/`: example of parallel programming estimating the digits of pi, using `concurrent.futures` and `dask`
* `dask` scripts: `launch_workers.sh` and `launch_scheduler.sh` and `dask_workers.slrm` launch all the components of `dask` `distributed`
* [Slides on Google Docs](https://docs.google.com/presentation/d/1hcgwy6S7QXVCIZHI0_Rb7_9FPHQP8HH6iWmaqCKpdIU/edit?usp=sharing)