{"id":32178573,"url":"https://github.com/pyiron/pyiron_base","last_synced_at":"2026-03-17T04:05:46.463Z","repository":{"id":37431348,"uuid":"294009556","full_name":"pyiron/pyiron_base","owner":"pyiron","description":"Core components of the pyiron integrated development environment (IDE) for computational materials science","archived":false,"fork":false,"pushed_at":"2026-01-06T07:07:09.000Z","size":36608,"stargazers_count":24,"open_issues_count":78,"forks_count":14,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-01-07T21:30:42.461Z","etag":null,"topics":["workflow","workflow-automation","workflow-engine","workflow-management"],"latest_commit_sha":null,"homepage":"https://pyiron-base.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyiron.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-09-09T05:14:32.000Z","updated_at":"2026-01-06T07:07:12.000Z","dependencies_parsed_at":"2024-01-01T05:28:32.267Z","dependency_job_id":"f27c8273-edff-493c-b8ca-fb0156791aed","html_url":"https://github.com/pyiron/pyiron_base","commit_stats":{"total_commits":4039,"total_committers":42,"mean_commits":96.16666666666667,"dds":0.7135429561772716,"last_synced_commit":"c1a02fe54b64080215e971c3d8ab0cba2b8d7cf0"},"previous_names":[],"tags_count":231,"template":false,"template_full_name":null,"purl":"pkg:github/pyiron/pyiron_base","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyiron%2Fpyiron_base","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyiron%2Fpyiron_base/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyiron%2Fpyiron_base/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyiron%2Fpyiron_base/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyiron","download_url":"https://codeload.github.com/pyiron/pyiron_base/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyiron%2Fpyiron_base/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28303474,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T11:18:18.743Z","status":"ssl_error","status_checked_at":"2026-01-11T11:07:56.842Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["workflow","workflow-automation","workflow-engine","workflow-management"],"created_at":"2025-10-21T20:58:02.193Z","updated_at":"2026-01-12T14:33:46.447Z","avatar_url":"https://github.com/pyiron.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pyiron_base\n\n[![codecov](https://codecov.io/gh/pyiron/pyiron_base/graph/badge.svg?token=qjRvhGVij2)](https://codecov.io/gh/pyiron/pyiron_base)\n[![Release_Date](https://anaconda.org/conda-forge/pyiron_base/badges/latest_release_date.svg)](https://anaconda.org/conda-forge/pyiron_base/)\n[![Pipeline](https://github.com/pyiron/pyiron_base/actions/workflows/pipeline.yml/badge.svg)](https://github.com/pyiron/pyiron_base/actions/workflows/pipeline.yml)\n[![Downloads](https://anaconda.org/conda-forge/pyiron_base/badges/downloads.svg)](https://anaconda.org/conda-forge/pyiron_base/)\n[![Documentation Status](https://readthedocs.org/projects/pyiron-base/badge/?version=latest)](https://pyiron-base.readthedocs.io/en/latest/?badge=latest)\n\nThe `pyiron_base` workflow manager provides the data storage and job management for the [pyiron](https://pyiron.org)\nproject. As part of the modularization of the [pyiron](https://pyiron.org) project in 2018, the monolithic code base\nwhich started as `pyCMW` back in 2011 was split in `pyiron_base` and `pyiron_atomistics`. This split highlights the \nseparation of the technical complexity and workflow management in `pyiron_base` and the physics modelling for atomistic \nsimulation in `pyiron_atomistics`.\n\n## Features:\n\n* Calculation which can be either simple python functions or external executables written in any programming language \n  can be wrapped in `pyiron_base` to enable parameter studies with thousands or millions of calculation.\n* The calculation can either be executed locally on the same computer or on high performance computing (HPC) resources.\n  The python simple queuing system adapter [pysqa](https://pysqa.readthedocs.io) is used to interface with the HPC \n  queuing systems directly from python and the [executorlib](https://executorlib.readthedocs.io) package is employed to \n  assign dedicated resources like multiple CPU cores and GPUs to individual python functions.\n* Scientific data is efficiently stored using the [hierarchical data format (HDF)](https://www.hdfgroup.org) via the \n  [h5py](https://www.h5py.org) python library and more specifically the [h5io](https://github.com/h5io) packages to \n  match the python datatypes to the HDF5 data types.\n\nWith this functionality the `pyiron_base` workflow manager enables the rapid prototyping and up-scaling of parameter \nstudies for a wide range of scientific application. Starting from simulation codes written in Fortran without any Python\nbindings, over more modern modelling codes written in C or C++ with Python bindings up to machine learning models \nrequiring GPU acceleration, the approach follows the same three steps:\n\n* Implement a wrapper for the simulation code, which takes a set of input parameters calls the simulation code and\n  returns a set of output parameters. For a simulation code with python bindings this is achieved with the \n  `wrap_python_function()` function and for any external executable which requires file-based communication this is \n  achieved with the `create_job_class()` function which requires only a `write_input()` function and a `collect_output()`\n  function to parse the input and output files of the external executable. Both functions return a `job` object. This is\n  the central building block of the `pyiron_base` workflow manager.\n* Following the map-reduce pattern a series of `job` objects are created and submitted to the available computing \n  resources. When the `pyiron_base` workflow manager is executed directly on the login node of a HPC cluster, the\n  calculation are directly submitted to the queuing system. Alternatively, the `pyiron_base` workflow manager also \n  supports submission via an secure shell (SSH) connection to the HPC cluster. Still in contrast to many other workflow \n  managers, the `pyiron_base` workflow manager does not require constant connection to the remote computing resources. \n  Once the `job` objects are submitted the workflow can be shutdown.\n* Finally, after the execution of the individual `job` objects is completed the `pyiron_table` object gathers the data \n  of the individual `job` objects in a single table. The table is accessible as `pandas.DataFrame` so it is compatible \n  to most machine learning and plotting libraries for further analysis.\n\n## Example:\nAs the `pyiron_base` workflow manager was developed as part of the [pyiron](https://pyiron.org) project the\nimplementation of the [quantum espresso](https://www.quantum-espresso.org) density functional theory (DFT) simulation\ncode in the `pyiron_base` workflow manager is chosen as example. Still the same steps apply for any kind of simulation \ncode:\n```python\n import os\n import matplotlib.pyplot as plt\n import numpy as np\n from ase.build import bulk\n from ase.calculators.espresso import Espresso\n from ase.io import write\n from pwtools import io\n\n\n def write_input(input_dict, working_directory=\".\"):\n     filename = os.path.join(working_directory, 'input.pwi')\n     os.makedirs(working_directory, exist_ok=True)\n     write(\n         filename=filename,\n         images=input_dict[\"structure\"],\n         Crystal=True,\n         kpts=input_dict[\"kpts\"],\n         input_data={\"calculation\": input_dict[\"calculation\"]},\n         pseudopotentials=input_dict[\"pseudopotentials\"],\n         tstress=True,\n         tprnfor=True\n     )\n\n\n def collect_output(working_directory=\".\"):\n     filename = os.path.join(working_directory, 'output.pwo')\n     try:\n         return {\"structure\": io.read_pw_md(filename)[-1].get_ase_atoms()}\n     except TypeError:\n         out = io.read_pw_scf(filename)\n         return {\n             \"energy\": out.etot,\n             \"volume\": out.volume,\n         }\n\n\n def workflow(project, structure):\n     # Structure optimization\n     job_qe_minimize = pr.create.job.QEJob(job_name=\"qe_relax\")\n     job_qe_minimize.input[\"calculation\"] = \"vc-relax\"\n     job_qe_minimize.input.structure = structure\n     job_qe_minimize.run()\n     structure_opt = job_qe_minimize.output.structure\n\n     # Energy Volume Curve\n     energy_lst, volume_lst = [], []\n     for i, strain in enumerate(np.linspace(0.9, 1.1, 5)):\n         structure_strain = structure_opt.copy()\n         structure_strain = structure.copy()\n         structure_strain.set_cell(\n             structure_strain.cell * strain**(1/3),\n             scale_atoms=True\n         )\n         job_strain = pr.create.job.QEJob(\n             job_name=\"job_strain_\" + str(i)\n         )\n         job_strain.input.structure = structure_strain\n         job_strain.run(delete_existing_job=True)\n         energy_lst.append(job_strain.output.energy)\n         volume_lst.append(job_strain.output.volume)\n\n     return {\"volume\": volume_lst, \"energy\": energy_lst}\n\n\n from pyiron_base import Project\n pr = Project(\"test\")\n pr.create_job_class(\n     class_name=\"QEJob\",\n     write_input_funct=write_input,\n     collect_output_funct=collect_output,\n     default_input_dict={  # Default Parameter\n         \"structure\": None,\n         \"pseudopotentials\": {\"Al\": \"Al.pbe-n-kjpaw_psl.1.0.0.UPF\"},\n         \"kpts\": (3, 3, 3),\n         \"calculation\": \"scf\",\n     },\n     executable_str=\"mpirun -np 1 pw.x -in input.pwi \u003e output.pwo\",\n )\n\n job_workflow = pr.wrap_python_function(workflow)\n job_workflow.input.project = pr\n job_workflow.input.structure = bulk('Al', a=4.15, cubic=True)\n job_workflow.run()\n\n plt.plot(job_workflow.output.result[\"volume\"], job_workflow.output.result[\"energy\"])\n plt.xlabel(\"Volume\")\n plt.ylabel(\"Energy\")\n```\n\nAfter the definition of the `write_input()` and `collect_output()` function for the quantum espresso DFT simulation code\nthe `workflow()` function is defined to combine multiple quantum espresso DFT simulation. First the structure is \noptimized to identify the equilibrium volume and afterwards five strains ranging from 90% to 110% are applied to \ndetermine the bulk modulus. Finally, in the last few lines all the individual pieces are put together, by creating \n`QEJob` the quantum espresso job class based on the `write_input()` and `collect_output()` function and then wrapping \nthe `workflow()` function using the `wrap_python_function()`. The whole workflow is executed when the `run()` function \nis called. Afterwards the results are plotted using the `matplotlib` library.\n\n## Disclaimer\nWhile we try to develop a stable and reliable software library, the development remains a opensource project under the\nBSD 3-Clause License without any warranties:\n```\nBSD 3-Clause License\n\nCopyright (c) 2018, Max-Planck-Institut für Eisenforschung GmbH - Computational Materials Design (CM) Department\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n\n* Redistributions of source code must retain the above copyright notice, this\nlist of conditions and the following disclaimer.\n\n* Redistributions in binary form must reproduce the above copyright notice,\nthis list of conditions and the following disclaimer in the documentation\nand/or other materials provided with the distribution.\n\n* Neither the name of the copyright holder nor the names of its\ncontributors may be used to endorse or promote products derived from\nthis software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\nFOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\nDAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\nCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\nOR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n```\n\n## Documentation\n\n* [Installation](https://pyiron-base.readthedocs.io/en/latest/installation.html)\n  * [Try pyiron_base online](https://pyiron-base.readthedocs.io/en/latest/installation.html#try-pyiron-base-online)\n  * [Workstation Installation](https://pyiron-base.readthedocs.io/en/latest/installation.html#workstation-installation)\n  * [HPC Cluster Installation](https://pyiron-base.readthedocs.io/en/latest/installation.html#hpc-cluster-installation)\n* [Tutorial](https://pyiron-base.readthedocs.io/en/latest/tutorial.html)\n  * [Jupyter Lab](https://pyiron-base.readthedocs.io/en/latest/tutorial.html#jupyter-lab)\n  * [Project](https://pyiron-base.readthedocs.io/en/latest/tutorial.html#project)\n  * [Job](https://pyiron-base.readthedocs.io/en/latest/tutorial.html#job)\n  * [Table](https://pyiron-base.readthedocs.io/en/latest/tutorial.html#table)\n* [Example](https://pyiron-base.readthedocs.io/en/latest/example.html)\n* [Command Line](https://pyiron-base.readthedocs.io/en/latest/commandline.html)\n* [Developers](https://pyiron-base.readthedocs.io/en/latest/developer.html)\n  * [Install from Source](https://pyiron-base.readthedocs.io/en/latest/developer.html#install-from-source)\n  * [HDF5 Serialization](https://pyiron-base.readthedocs.io/en/latest/developer.html#hdf5-serialization)\n  * [Run function](https://pyiron-base.readthedocs.io/en/latest/developer.html#run-function)\n  * [Queuing System](https://pyiron-base.readthedocs.io/en/latest/developer.html#queuing-system)\n  * [Command Line](https://pyiron-base.readthedocs.io/en/latest/developer.html#command-line)\n* [FAQ](https://pyiron-base.readthedocs.io/en/latest/faq.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyiron%2Fpyiron_base","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpyiron%2Fpyiron_base","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyiron%2Fpyiron_base/lists"}