{"id":22668224,"url":"https://github.com/lanl/phoebus","last_synced_at":"2025-05-07T21:30:16.088Z","repository":{"id":37893115,"uuid":"356070762","full_name":"lanl/phoebus","owner":"lanl","description":"Phifty One Ergs Blows Up A Star","archived":false,"fork":false,"pushed_at":"2025-04-24T09:51:07.000Z","size":16141,"stargazers_count":48,"open_issues_count":53,"forks_count":3,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-04-24T10:42:24.871Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lanl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-04-08T23:05:44.000Z","updated_at":"2025-04-10T20:17:35.000Z","dependencies_parsed_at":"2024-01-14T23:01:22.124Z","dependency_job_id":"fe23c0ef-6b1a-42b7-9700-66de528ba8f9","html_url":"https://github.com/lanl/phoebus","commit_stats":{"total_commits":1831,"total_committers":16,"mean_commits":114.4375,"dds":0.5188421627525942,"last_synced_commit":"532be0f43549b5abc903d20f6961ceeb5c61ddb4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lanl%2Fphoebus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lanl%2Fphoebus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lanl%2Fphoebus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lanl%2Fphoebus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lanl","download_url":"https://codeload.github.com/lanl/phoebus/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252956960,"owners_count":21831411,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-09T15:14:10.294Z","updated_at":"2025-05-07T21:30:16.006Z","avatar_url":"https://github.com/lanl.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"phoebus\n===\n![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/lanl/phoebus/tests.yml?branch=main)\n[![Documentation](https://github.com/lanl/phoebus/actions/workflows/docs.yml/badge.svg?branch=main)](https://lanl.github.io/phoebus/main/index.html)\n[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n\nPhifty One Ergs Blows Up a Star\n\n# Obtaining the Source Code\n\n`phoebus` uses submodules extensively. To make sure you get them all, clone it as\n```bash\ngit clone --recursive git@github.com:lanl/phoebus.git\n```\nor as\n```bash\ngit clone git@github.com:lanl/phoebus.git\ncd phoebus\ngit submodule update --init --recursive\n```\n\n# Building\n\n## Prerequisites\n\nHere are a few tips and tricks for using and building phoebus. The first\nstep is creating a build directory, as in-source builds are\nforbidden. Assuming you just cloned phoebus,\n```bash\ncd phoebus\nmkdir bin\ncd bin\n```\n\nYou also need to make sure you have appropriate dependencies in your\npath. On Darwin, loading the following modules (in this order) will\nwork:\n```bash\nmodule load gcc/7.4.0 cuda/10.2 openmpi/4.0.1-gcc_7.4.0 cmake/3.17.3\n```\n(The default build for `parthenon` requires Python3 loaded. This is\nnot the case for `phoebus`.)\n\nIf you want output, make sure `hdf5` is available as well. On\n`Darwin`, this likely means a spack environment. On `snow`, just load\nthe appropriate module. If you are compiling with mpi, use\n`hdf5-parallel`, otherwise `hdf5-serial`.\n\nIf `cmake` can't find your hdf5 installation, You may have luck\nsetting `-DHDF5_ROOT=/path/to/hdf5` at configure time.\n\n### Installing hdf5 through spack\n\nIf hdf5 isn't available on your target machine, here is one way to build it, using [Spack](https://spack.readthedocs.io/en/latest/getting_started.html).\n```bash\n# download spack\ngit clone https://github.com/spack/spack.git\n# Activate the spack bash aliases. This typically needs to be done every time you want to use spack. I have it in my bashrc\n. spack/share/spack/setup-env.sh\n# load your modules\nmodule load...\n# Tell spack to learn about compilers and loaded modules\nspack compiler find\nspack external find\n# Install parallel hdf5 with high-level libraries\nspack install hdf5+mpi+hl\n# load it\nspack load hdf5\n```\n\nInstalling from source, or just building without hdf5 will also both probably work.\n\n## Example builds\n\nBelow are some example build commands\n\n### MPI-parallel only\n\nThe following will build `phoebus` with MPI parallelism but no shared memory parallelism\n```bash\ncmake ..\nmake -j\n```\n\n### OpenMP-parallel\n\nThe following will build `phoebus` with OpenMP parallelism only\n```bash\ncmake -DPHOEBUS_ENABLE_MPI=Off -DPHOEBUS_ENABLE_OPENMP=ON ..\nmake -j\n```\n\n### Cuda\n\nThe following will build `phoebus` with no MPI or OpenMP parallelism.\n```bash\ncmake -DPHOEBUS_ENABLE_CUDA=On -DCMAKE_CXX_COMPILER=${HOME}/phoebus/external/singularity-eos/utils/kokkos/bin/nvcc_wrapper -DKokkos_ARCH_HSW=ON -DKokkos_ARCH_VOLTA70=ON -DPHOEBUS_ENABLE_MPI=OFF ..\n```\nA few notes for this one:\n- Note here the `-DCMAKE_CXX_COMPILER` flag. This is necessary. You *must* set the compiler to `nvcc_wrapper` provided by Kokkos.\n- Note the `-DKokkos_ARCH_*` flags. Those set the host and device microarchitectures and are required. The choice here is on an `x86_64` machine with a `volta` GPU.\n\n## Build Options\n\nThe build options explicitly provided by `phoebus` are:\n\n| Option             | Default | Comment                                            |\n| ------------------ | ------- | -------------------------------------------------- |\n| PHOEBUS_ENABLE_CUDA   | OFF     | Cuda                                               |\n| PHOEBUS_ENABLE_HDF5   | ON      | HDF5. Required for output and restarts.            |\n| PHOEBUS_ENABLE_MPI    | ON      | MPI. Required for distributed memory parallelism.  |\n| PHOEBUS_ENABLE_OPENMP | OFF     | OpenMP. Required for shared memory parallelism.    |\n| MACHINE_CFG        | None    | Machine-specific config file, optional.            |\n\nSome relevant settings from Parthenon and Kokkos you may need to play with are:\n\n| Option                   | Default | Comment                                                   |\n| ------------------------ | ------- | --------------------------------------------------------- |\n| CMAKE_CXX_COMPILER       | None    | Must be set to `nvcc_wrapper` with cuda backend           |\n| Kokkos_ARCH_XXXX         | OFF     | You must set the GPU architecture when compiling for Cuda |\n\nYou can see all the Parthenon build options [here](https://github.com/lanl/parthenon/blob/develop/docs/building.md) and all the Kokkos build options [here](https://github.com/kokkos/kokkos/wiki/Compiling)\n\n## Cmake machine configs\n\nIf you are proficient with `cmake` You can optionally write a `cmake`\nfile that sets the configure parameters that you like on a given\nmachine. Both `phoebus` and `parthenon` can make use of it. You can point to the file with\n```\n-DMACHINE_CFG=path/to/machine/file\n```\nat config time or by setting the environment variable `MACHINE_CFG` to point at it, e.g.,\n```bash\nexport MACHINE_CFG=path/to/machine/file\n```\n\nAn example machine file might look like\n```\n# Machine file for x86_64-volta on Darwin\nmessage(STATUS \"Loading machine configuration for Darwin x86-volta node\")\nmessage(STATUS \"Assumes: module load module load gcc/7.4.0 cuda/10.2 openmpi/4.0.3-gcc_7.4.0 anaconda/Anaconda3.2019.10 cmake \u0026\u0026 spack load hdf5\")\nmessage(STATUS \"Also assumes you have a valid spack installation loaded.\")\n\nset(PHOEBUS_ENABLE_CUDA ON CACHE BOOL \"Cuda backend\")\nset(PHOEBUS_ENABLE_MPI OFF CACHE BOOL \"No MPI\")\nset(Kokkos_ARCH_HSW ON CACHE BOOL \"Haswell target\")\nset(Kokkos_ARCH_VOLTA70 ON CACHE BOOL \"volta target\")\nset(CMAKE_CXX_COMPILER /home/jonahm/phoebus/external/parthenon/external/Kokkos/bin/nvcc_wrapper CACHE STRING \"nvcc wrapper\")\n```\n\nyou could then configure and compile as\n\n```bash\ncmake -DMACHINE_CFG=path/to/machine/file ..\nmake -j\n```\n\n# Running\n\nRun phoebus from the `build` directory as\n```bash\n./src/phoebus -i path/to/input/file.pin\n```\nThe input files are in `phoebus/inputs/*`. There's typically one input file per problem setup file.\n\n## Submodules\n\n- `parthenon` asynchronous tasking and block-AMR infrastructure\n- `singularity-eos` provides performance-portable equations of state and PTE solvers\n- `singularity-opac` provides performance-portable opacities and emissivities\n- `Kokkos` provides performance portable shared-memory parallelism. It allows our loops to be\n  CUDA, OpenMP, or something else. By default we use the `Kokkos` shipped with `parthenon`.\n\n## External (Required)\n\n- `cmake` for building\n\n## Optional\n\n- `hdf5` for output\n- `MPI` for distributed memory parallelism\n- `python3` for visualization\n\n## Clang-format\n\nWe use clang format for code cleanliness. The current version of\n`clang-format` is pinned to `clang-format-12`, however, you can try\nformatting with other versions at your own risk.\n\nYou can format all files in git history automatically by calling\n`scripts/bash/format.sh` from anywhere within the Phoebus repo (but\noutside of submodules).\n\nYou can specify which clang-format you use in the script with the CFM\nenvironment variable. For example:\n```bash\nCFM=clang-format-12 ./scripts/bash/format.sh\n```\n\n# Contribute\nWe are always happy to have users contribute to `phoebus`.\nTo contribute, create a fork and issue a pull request against the main branch. \nFor more details on how to contribute to `phoebus`, please see [CONTRIBUTING.md](CONTRIBUTING.md).\nWe adhere to a [code of conduct](CODE_OF_CONDUCT.md).\n\n# Copyright\n\n© 2021-2024. Triad National Security, LLC. All rights reserved.  This\nprogram was produced under U.S. Government contract 89233218CNA000001\nfor Los Alamos National Laboratory (LANL), which is operated by Triad\nNational Security, LLC for the U.S.  Department of Energy/National\nNuclear Security Administration. All rights in the program are\nreserved by Triad National Security, LLC, and the U.S. Department of\nEnergy/National Nuclear Security Administration. The Government is\ngranted for itself and others acting on its behalf a nonexclusive,\npaid-up, irrevocable worldwide license in this material to reproduce,\nprepare derivative works, distribute copies to the public, perform\npublicly and display publicly, and to permit others to do so.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flanl%2Fphoebus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flanl%2Fphoebus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flanl%2Fphoebus/lists"}