https://github.com/open-atmos/pypartmc
Python (and C++) interface to PartMC with Jupyter/Python, Julia and Matlab examples
https://github.com/open-atmos/pypartmc
aerosol-modelling atmospheric-modelling atmospheric-physics monte-carlo-simulation particle-system pybind11 python research simulation sundials
Last synced: about 1 month ago
JSON representation
Python (and C++) interface to PartMC with Jupyter/Python, Julia and Matlab examples
- Host: GitHub
- URL: https://github.com/open-atmos/pypartmc
- Owner: open-atmos
- License: gpl-3.0
- Created: 2021-10-26T16:44:30.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-05-22T22:13:53.000Z (12 months ago)
- Last Synced: 2024-05-22T22:26:16.420Z (12 months ago)
- Topics: aerosol-modelling, atmospheric-modelling, atmospheric-physics, monte-carlo-simulation, particle-system, pybind11, python, research, simulation, sundials
- Language: C++
- Homepage: https://open-atmos.github.io/PyPartMC/
- Size: 1.47 MB
- Stars: 21
- Watchers: 3
- Forks: 8
- Open Issues: 59
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
README
# PyPartMC
PyPartMC is a Python interface to [PartMC](https://lagrange.mechse.illinois.edu/partmc/),
a particle-resolved Monte-Carlo code for atmospheric aerosol simulation.
Development of PyPartMC has been intended to remove limitations to the use of Fortran-implemented PartMC.
PyPartMC facilitates the dissemination of computational research results by streamlining independent execution
of PartMC simulations (also during peer-review processes).
Additionally, the ability to easily package examples, simple simulations, and results in a web-based notebook
allows PyPartMC to support the efforts of many members of the scientific community, including researchers,
instructors, and students, with nominal software and hardware requirements.
Documentation of PyPartMC is hosted at https://open-atmos.github.io/PyPartMC.
PyPartMC is implemented in C++ and it also constitutes a C++ API to the PartMC Fortran internals.
The Python API can facilitate using PartMC from other environments - see, e.g., Julia and Matlab examples below.
For an outline of the project, rationale, architecture, and features, refer to: [D'Aquino et al., 2024 (SoftwareX)](https://doi.org/10.1016/j.softx.2023.101613) (please cite if PyPartMC is used in your research).
For a list of talks and other relevant resources, please see [project Wiki](https://github.com/open-atmos/PyPartMC/wiki/).
If interested in contributing to PyPartMC, please have a look a the [notes for developers](https://github.com/open-atmos/PyPartMC/tree/main/CONTRIBUTING.md).
[](https://asr.science.energy.gov/) [](https://www.ncn.gov.pl/?language=en)
[](https://www.gnu.org/licenses/gpl-3.0.html)
[](https://atmos.illinois.edu/)
[](https://github.com/open-atmos/PyPartMC/actions/workflows/tests+pypi.yml)
[](https://open-atmos.github.io/PyPartMC/)
[](https://codecov.io/gh/open-atmos/PyPartMC)
[](https://doi.org/10.5281/zenodo.7662635)
[](https://pypi.org/p/PyPartMC)
[](https://www.repostatus.org/#active)
[](https://github.com/pyOpenSci/software-review/issues/179)
[](https://www.python.org/)
[](https://en.wikipedia.org/wiki/Linux)
[](https://en.wikipedia.org/wiki/macOS)
[](https://en.wikipedia.org/wiki/Windows)
[](https://jupyter.org/)
## Installation
### Using the command-line `pip` tool (also applies to conda environments)
```bash
pip install PyPartMC
```
Note that, depending on the environment (OS, hardware, Python version), the pip-install invocation
may either trigger a download of a pre-compiled binary, or trigger compilation of PyPartMC.
In the latter case, a Fortran compiler and some development tools includiong CMake, m4 and perl
are required (while all non-Python dependencies are included in the PyPartMC source archive).
In both cases, all Python dependencies will be resolved by pip.
### In a Jupyter notebook cell (also on Colab or jupyter-hub instances)
```python
! pip install PyPartMC
import PyPartMC
```
#### Jupyter notebooks with examples
Note: clicking the badges below redirects to cloud-computing platforms. The mybinder.org links allow anonymous execution, Google Colab requires logging in with a Google account, ARM JupyerHub requires logging in with an ARM account (and directing Jupyter to a particular notebook within the `examples` folder).
The example notebooks feature additional dependencies that can be installed with:
```bash
pip install PyPartMC[examples]
```
- Urban plume scenario demo (as in [PartMC](https://github.com/compdyn/partmc/tree/master/scenarios/1_urban_plume)):
[](https://github.com/open-atmos/PyPartMC/blob/main/examples/particle_simulation.ipynb)
[](https://colab.research.google.com/github/open-atmos/PyPartMC/blob/main/examples/particle_simulation.ipynb)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC.git/main?urlpath=lab/tree/examples/particle_simulation.ipynb)
[](https://jupyterhub.arm.gov/hub/user-redirect/git-pull?repo=https%3A//github.com/open-atmos/PyPartMC&branch=main&urlPath=)
- Dry-Wet Particle Size Equilibration with PartMC and PySDM:
[](https://github.com/open-atmos/PyPartMC/blob/main/examples/lognorm_ex.ipynb)
[](https://colab.research.google.com/github/open-atmos/PyPartMC/blob/main/examples/lognorm_ex.ipynb)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC.git/main?urlpath=lab/tree/examples/lognorm_ex.ipynb)
[](https://jupyterhub.arm.gov/hub/user-redirect/git-pull?repo=https%3A//github.com/open-atmos/PyPartMC&branch=main&urlPath=)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC/main?urlpath=voila%2Frender%2Fexamples%2Flognorm_ex.ipynb)
- Simulation output processing example (loading from netCDF files using PyPartMC):
[](https://github.com/open-atmos/PyPartMC/blob/main/examples/process_simulation_output.ipynb)
[](https://colab.research.google.com/github/open-atmos/PyPartMC/blob/main/examples/process_simulation_output.ipynb)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC.git/main?urlpath=lab/tree/examples/process_simulation_output.ipynb)
[](https://jupyterhub.arm.gov/hub/user-redirect/git-pull?repo=https%3A//github.com/open-atmos/PyPartMC&branch=main&urlPath=)
- Optical properties calculation using external Python package ([PyMieScatt](https://pymiescatt.readthedocs.io/en/latest/)):
[](https://github.com/open-atmos/PyPartMC/blob/main/examples/mie_optical.ipynb)
[](https://colab.research.google.com/github/open-atmos/PyPartMC/blob/main/examples/mie_optical.ipynb)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC.git/main?urlpath=lab/tree/examples/mie_optical.ipynb)
[](https://jupyterhub.arm.gov/hub/user-redirect/git-pull?repo=https%3A//github.com/open-atmos/PyPartMC&branch=main&urlPath=)
- Cloud parcel example featuring supersaturation-evolution-coupled CCN activation and drop growth:
[](https://github.com/open-atmos/PyPartMC/blob/main/examples/cloud_parcel.ipynb)
[](https://colab.research.google.com/github/open-atmos/PyPartMC/blob/main/examples/cloud_parcel.ipynb)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC.git/main?urlpath=lab/tree/examples/cloud_parcel.ipynb)
[](https://jupyterhub.arm.gov/hub/user-redirect/git-pull?repo=https%3A//github.com/open-atmos/PyPartMC&branch=main&urlPath=)
- Coagulation Model Intercomparison with PySDM, Droplets.jl:
[](https://github.com/open-atmos//PyPartMC/blob/main/examples/additive_coag_comparison.ipynb)
[](https://colab.research.google.com/github/open-atmos/PyPartMC/blob/main/examples/additive_coag_comparison.ipynb)
[](https://mybinder.org/v2/gh/open-atmos/PyPartMC.git/main?urlpath=lab/tree/examples/additive_coag_comparison.ipynb)
[](https://jupyterhub.arm.gov/hub/user-redirect/git-pull?repo=https%3A//github.com/open-atmos/PyPartMC&branch=main&urlPath=)
## Features
- works on Linux, macOS and Windows (compatibility assured with [CI builds](https://github.com/open-atmos/PyPartMC/blob/main/.github/workflows/tests.yml))
- hassle-free installation using `pip` (prior PartMC installation **not needed**)
- works out of the box on [mybinder.org](https://mybinder.org/), [Google Colab](colab.research.google.com/) and alike
- ships with [a set of examples](https://github.com/open-atmos/PyPartMC/tree/main/examples) maintained in a form of Jupyter notebooks
- Pythonic API (but retaining PartMC jargon) incl. Python GC deallocation of Fortran objects
- specification of parameters using native Python datatypes (lists, dicts) in place of PartMC spec files
- code snippets in README depicting how to use PyPartMC from Julia and Matlab (also executed on CI)
- auto-generated [API docs on the web](https://open-atmos.github.io/PyPartMC/)
- support for [de]serialization of selected wrapped structures using JSON
- based on [unmodified PartMC code](https://github.com/open-atmos/PyPartMC/tree/main/gitmodules)
- does not use or require shell or any pre-installed libraries
- aiming at 100% [unit test coverage](https://github.com/open-atmos/PyPartMC/tree/main/tests)
## Usage examples
The listings below depict how the identical task of randomly sampling particles from an aerosol size distribution in PartMC can be
done in different programming languages.
For a Fortran equivalent of the Python, Julia and Matlab programs below, see the [`readme_fortran` folder](https://github.com/open-atmos/PyPartMC/tree/main/readme_fortran).
#### Python
```Python
import numpy as np
import PyPartMC as ppmc
from PyPartMC import si
aero_data = ppmc.AeroData((
# [density, ions in solution, molecular weight, kappa]
{"OC": [1000 *si.kg/si.m**3, 0, 1e-3 *si.kg/si.mol, 0.001]},
{"BC": [1800 *si.kg/si.m**3, 0, 1e-3 *si.kg/si.mol, 0]},
))
aero_dist = ppmc.AeroDist(
aero_data,
[{
"cooking": {
"mass_frac": [{"OC": [1]}],
"diam_type": "geometric",
"mode_type": "log_normal",
"num_conc": 3200 / si.cm**3,
"geom_mean_diam": 8.64 * si.nm,
"log10_geom_std_dev": 0.28,
},
"diesel": {
"mass_frac": [{"OC": [0.3]}, {"BC": [0.7]}],
"diam_type": "geometric",
"mode_type": "log_normal",
"num_conc": 2900 / si.cm**3,
"geom_mean_diam": 50 * si.nm,
"log10_geom_std_dev": 0.24,
}
}],
)
n_part = 100
aero_state = ppmc.AeroState(aero_data, n_part, "nummass_source")
aero_state.dist_sample(aero_dist)
print(np.dot(aero_state.masses(), aero_state.num_concs), "# kg/m3")
```
#### Julia (using [PyCall.jl](https://github.com/JuliaPy/PyCall.jl))
```Julia
using Pkg
Pkg.add("PyCall")
using PyCall
ppmc = pyimport("PyPartMC")
si = ppmc["si"]
aero_data = ppmc.AeroData((
# (density, ions in solution, molecular weight, kappa)
Dict("OC"=>(1000 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0.001)),
Dict("BC"=>(1800 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0))
))
aero_dist = ppmc.AeroDist(aero_data, (
Dict(
"cooking" => Dict(
"mass_frac" => (Dict("OC" => (1,)),),
"diam_type" => "geometric",
"mode_type" => "log_normal",
"num_conc" => 3200 / si.cm^3,
"geom_mean_diam" => 8.64 * si.nm,
"log10_geom_std_dev" => .28,
),
"diesel" => Dict(
"mass_frac" => (Dict("OC" => (.3,)), Dict("BC" => (.7,))),
"diam_type" => "geometric",
"mode_type" => "log_normal",
"num_conc" => 2900 / si.cm^3,
"geom_mean_diam" => 50 * si.nm,
"log10_geom_std_dev" => .24,
)
),
))
n_part = 100
aero_state = ppmc.AeroState(aero_data, n_part, "nummass_source")
aero_state.dist_sample(aero_dist)
print(aero_state.masses()'aero_state.num_concs, "# kg/m3")
```
#### Matlab (using [Matlab's built-in Python interface](https://www.mathworks.com/help/matlab/python-language.html))
notes (see the [PyPartMC Matlab CI workflow](https://github.com/open-atmos/PyPartMC/blob/main/.github/workflows/readme_listings.yml) for an example on how to achieve it on Ubuntu 20):
- Matlab ships with convenience copies of C, C++ and Fortran runtime libraries which are `dlopened()` by default; one way to make PyPartMC OK with it is to [pip-]install by compiling from source using the very same version of GCC that Matlab borrowed these libraries from (e.g., [GCC 9 for Matlab R2022a, etc](https://www.mathworks.com/support/requirements/supported-compilers-linux.html));
- Matlab needs to [use the same Python interpretter/venv](https://www.mathworks.com/support/requirements/python-compatibility.html) as the pip invocation used to install PyPartMC;
- a single-line `pybind11_builtins.py` file with just `pybind11_type=type` inside needs to be placed within Matlab's `PYTHONPATH` to sort out a [Matlab-pybind11 incompatibility](https://github.com/pybind/pybind11/issues/3945).
````Matlab
ppmc = py.importlib.import_module('PyPartMC');
si = py.importlib.import_module('PyPartMC').si;
aero_data = ppmc.AeroData(py.tuple({ ...
py.dict(pyargs("OC", py.tuple({1000 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0.001}))), ...
py.dict(pyargs("BC", py.tuple({1800 * si.kg/si.m^3, 0, 1e-3 * si.kg/si.mol, 0}))) ...
}));
aero_dist = ppmc.AeroDist(aero_data, py.tuple({ ...
py.dict(pyargs( ...
"cooking", py.dict(pyargs( ...
"mass_frac", py.tuple({py.dict(pyargs("OC", py.tuple({1})))}), ...
"diam_type", "geometric", ...
"mode_type", "log_normal", ...
"num_conc", 3200 / si.cm^3, ...
"geom_mean_diam", 8.64 * si.nm, ...
"log10_geom_std_dev", .28 ...
)), ...
"diesel", py.dict(pyargs( ...
"mass_frac", py.tuple({ ...
py.dict(pyargs("OC", py.tuple({.3}))), ...
py.dict(pyargs("BC", py.tuple({.7}))), ...
}), ...
"diam_type", "geometric", ...
"mode_type", "log_normal", ...
"num_conc", 2900 / si.cm^3, ...
"geom_mean_diam", 50 * si.nm, ...
"log10_geom_std_dev", .24 ...
)) ...
)) ...
}));
n_part = 100;
aero_state = ppmc.AeroState(aero_data, n_part, "nummass_source");
aero_state.dist_sample(aero_dist);
masses = cell(aero_state.masses());
num_concs = cell(aero_state.num_concs);
fprintf('%g # kg/m3\n', dot([masses{:}], [num_concs{:}]))
````
#### usage in other projects
PyPartMC is used within the [test workflow of the PySDM project](https://github.com/atmos-cloud-sim-uj/PySDM/tree/main/tests/smoke_tests/box/partmc).
## Other packages with relevant feature scope
- [aerosolGDEFoam](https://openaerosol.sourceforge.io/): OpenFOAM CFD-coupled aerosol dynamics including nucleation, coagulation, and surface growth
- [AIOMFAC and AIOMFAC-web](http://www.aiomfac.caltech.edu/): Fortran-implemented aerosol thermodynamic model for calculation of activity coefficients in organic-inorganic mixtures – from simple binary solutions to complex multicomponent systems
- [DustPy](https://stammler.github.io/dustpy/): Python package for modelling dust evolution in protoplanetary disks (differences: focus on astrophysical applications vs. atmospheric aerosol)
- [multilayerpy](https://github.com/tintin554/multilayerpy): kinetic multi-layer model for aerosol particles and films
- [PyBox](https://pybox.readthedocs.io): aerosol simulation model featuring gas and particle chamistry (differences: PyBox focuses on chemical mechanisms; PyPartMC is an interface to PartMC which focuses on physics - e.g., collisions of aerosol particles - while chemical processes are handled with external software, e.g., CAMP or MOSAIC)
- [PyCHAM](https://github.com/simonom/PyCHAM): CHemistry with Aerosol Microphysics in Python Box Model for modelling of indoor environments, including aerosol chambers
- [PySDM](https://open-atmos.github.io/PySDM): particle-based Monte-Carlo aerosol-cloud simulation package (differences: PySDM focuses on growth and breakup processes relevant to cloud droplets; PyPartMC focuses on processes relevant to air pollutants and their chemical and physical transformations)
- [SSH-aerosol](https://github.com/sshaerosol/ssh-aerosol): C++/Fortran package for simulating evolution of primary and secondary atmospheric aerosols
## FAQ
- Q: How to install PyPartMC with MOSAIC enabled?
A: Installation can be done using `pip`, however, `pip` needs to be instructed not to use binary packages available at pypi.org but rather to compile from source (pip will download the source from pip.org), and the path to compiled MOSAIC library needs to be provided at compile-time; the following command should convey it:
```bash
MOSAIC_HOME=<> pip install --force-reinstall --no-binary=PyPartMC PyPartMC
```
- Q: Why `pip install PyPartMC` triggers compilation on my brand new Apple machine, while it quickly downloads and installs binary packages when executed on older Macs, Windows or Linux?
A: We are providing binary wheels on PyPI for Apple-silicon (arm64) machines for selected macOS version made available by Github. In case the macOS version you are using is newer, compilation from source is triggered.
- Q: Why some of the constructors expect data to be passed as **lists of single-entry dictionaries** instead of multi-element dictionaries?
A: This is intentional and related with PartMC relying on the order of elements within spec-file input; while Python dictionaries preserve ordering (insertion order), JSON format does not, and we intend to make these data structures safe to be [de]serialized using JSON.
- Q: How to check the version of PartMC that PyPartMC was compiled against?
A: Version numbers of compile-time dependencies of PyPartMC, including PartMC, can be accessed as follows:
```Python
import PyPartMC
PyPartMC.__versions_of_build_time_dependencies__['PartMC']
```
- Q: Why m4 and perl are required at compile time?
A: PyPartMC includes parts of netCDF and HDF5 codebases which depend on m4 and perl, respectively, for generating source files before compilation.
## Troubleshooting
#### Common installation issues
```
error: [Errno 2] No such file or directory: 'cmake'
```
Try rerunning after installing CMake, e.g., using `apt-get install cmake` (Ubuntu/Debian), `brew install cmake` (homebrew on macOS) or using [MSYS2](https://www.msys2.org/docs/cmake/) on Windows.
```
No CMAKE_Fortran_COMPILER could be found.
```
Try installing a Fortran compiler (e.g., `brew reinstall gcc` with Homebrew on macOS or using [MSYS2](https://packages.msys2.org/package/mingw-w64-x86_64-gcc-fortran?repo=mingw64) on Windows).
```
Could not find NC_M4 using the following names: m4, m4.exe
```
Try installing `m4` (e.g., using [MSYS2](https://packages.msys2.org/package/m4?repo=msys&variant=x86_64) on Windows).
## Acknowledgement and citations
We would greatly appreciate citation of the PartMC model description paper (Riemer et al., 2009)
and the PyPartMC description paper (D’Aquino et al., 2024) if PyPartMC was used in your study.
The citations are:
- Riemer, N., M. West, R. A. Zaveri, R. C. Easter: Simulating the evolution of soot
mixing-state with a particle-resolved aerosol model
J. Geophys. Res., 114, D09202, 2009, DOI: [10.1029/2008JD011073](https://doi.org/10.1029/2008JD011073)
- D’Aquino, Z., S. Arabas, J. H. Curtis, A. Vaishnav, N. Riemer, M. West: PyPartMC: A
pythonic interfact to a particle-resolved, Monte Carlo aerosol simulation framework
SoftwareX, 25, 101613, 2024, DOI: [10.1016/j.softx.2023.101613](https://doi.org/10.1016/j.softx.2023.101613)
The following paragraph provides a more substantial description of PartMC (text released into the public domain and can be freely copied by anyone for any purpose):
> PartMC is a stochastic, particle-resolved aerosol box model. It tracks the
composition of many computational particles (104 to 106) within a well-mixed air
volume, each represented by a composition vector that evolves based on physical
and chemical processes. The physical processes—including Brownian coagulation,
new particle formation, emissions, dilution, and deposition—are simulated using a
stochastic Monte Carlo approach via a Poisson process while chemical processes are
simulated deterministically for each computational particle. The weighted flow
algorithm (DeVille, Riemer, and West, 2011, 2019) enhances efficiency and reduces
ensemble variance. Detailed numerical methods are described in Riemer et al.
(2009), DeVille et al. (2011, 2019), and Curtis et al. (2016). PartMC is open-source
under the GNU GPL v2 and available at
[github.com/compdyn/partmc](https://github.com/compdyn/partmc).
>
> References:
> - Curtis, J. H., M. D. Michelotti, N. Riemer, M. T. Heath, M. West: Accelerated
simulation of stochastic particle removal processes in particle-resolved aerosol
models, J. Computational Phys., 322, 21-32, 2016, DOI: [10.1016/j.jcp.2016.06.029](https://doi.org/10.1016/j.jcp.2016.06.029)
> - DeVille, L., N. Riemer, M. West, Convergence of a generalized weighted flow
algorithm for stochastic particle coagulation, J. Computational Dynamics, 6, 69-94,
2019, DOI: [10.3934/jcd.2019003](https://doi.org/10.3934/jcd.2019003)
> - DeVille, R. E. L., N. Riemer, M. West, The Weighted Flow Algorithm (WFA) for
stochastic particle coagulation, J. Computational Phys., 230, 8427-8451, 2011,
DOI: [10.1016/j.jcp.2011.07.027](https://doi.org/10.1016/j.jcp.2011.07.027)
> - Riemer, N., M. West, R. A. Zaveri, R. C. Easter, Simulating the evolution of soot
mixing-state with a particle-resolved aerosol model, J. Geophys. Res., 114, D09202,
2009., DOI: [10.1029/2008JD011073](https://doi.org/10.1029/2008JD011073)
## Credits
#### PyPartMC:
authors: [PyPartMC developers](https://github.com/open-atmos/PyPartMC/graphs/contributors)
funding: [US Department of Energy Atmospheric System Research programme](https://asr.science.energy.gov/), [Polish National Science Centre](https://ncn.gov.pl/en)
copyright: [University of Illinois at Urbana-Champaign](https://atmos.illinois.edu/)
licence: [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html)
#### PartMC:
authors: [Nicole Riemer](https://www.atmos.uiuc.edu/~nriemer/), [Matthew West](https://lagrange.mechse.illinois.edu/mwest/), [Jeff Curtis](https://publish.illinois.edu/jcurtis2/) et al.
licence: [GPL v2](https://www.gnu.org/licenses/old-licenses/gpl-2.0.html) or later