{"id":13446798,"url":"https://github.com/getpopper/popper","last_synced_at":"2025-12-14T15:45:48.880Z","repository":{"id":41160295,"uuid":"43576029","full_name":"getpopper/popper","owner":"getpopper","description":"Container-native task automation engine.","archived":false,"fork":false,"pushed_at":"2022-03-29T22:02:27.000Z","size":1501,"stargazers_count":304,"open_issues_count":23,"forks_count":61,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-20T16:58:01.391Z","etag":null,"topics":["automation-engine","cli","containers","devops","docker","podman","reproducibility","sciops","singularity","workflows"],"latest_commit_sha":null,"homepage":"https://getpopper.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/getpopper.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-02T20:58:39.000Z","updated_at":"2025-03-13T12:57:25.000Z","dependencies_parsed_at":"2022-09-01T15:24:47.825Z","dependency_job_id":null,"html_url":"https://github.com/getpopper/popper","commit_stats":null,"previous_names":["systemslab/popper"],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getpopper%2Fpopper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getpopper%2Fpopper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getpopper%2Fpopper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getpopper%2Fpopper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/getpopper","download_url":"https://codeload.github.com/getpopper/popper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244656693,"owners_count":20488638,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation-engine","cli","containers","devops","docker","podman","reproducibility","sciops","singularity","workflows"],"created_at":"2024-07-31T05:01:00.162Z","updated_at":"2025-12-14T15:45:48.839Z","avatar_url":"https://github.com/getpopper.png","language":"Python","readme":"# \u003cimg src=\"https://raw.githubusercontent.com/getpopper/website/bcba4c8/assets/images/popper_logo_just_jug.png\" width=\"64\" valign=\"middle\" alt=\"Popper\"/\u003e Popper\n\n[![Downloads](https://pepy.tech/badge/popper)](https://pepy.tech/project/popper)\n[![Build Status](https://travis-ci.org/getpopper/popper.svg?branch=master)](https://travis-ci.org/getpopper/popper)\n[![codecov](https://codecov.io/gh/getpopper/popper/branch/master/graph/badge.svg)](https://codecov.io/gh/getpopper/popper)\n[![Join the chat at https://gitter.im/systemslab/popper](https://badges.gitter.im/systemslab/popper.svg)](https://gitter.im/falsifiable-us/popper?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n[![slack](https://img.shields.io/badge/chat-on_slack-C03C20.svg?logo=slack)](https://join.slack.com/t/getpopper/shared_invite/zt-dtn0se2s-c50myMHNpeoikQXDeNbPew)\n[![CROSS](https://img.shields.io/badge/supported%20by-CROSS-green)](https://cross.ucsc.edu)\n\nPopper is a tool for defining and executing container-native testing \nworkflows in Docker. With Popper, you define a workflow in a YAML \nfile, and then execute it with a single command. A workflow file looks \nlike this:\n\n```yaml\nsteps:\n- id: dev-init\n  uses: docker://rikorose/gcc-cmake:gcc-9\n  runs: [sh, -uexc]\n  args:\n  - |\n    rm -rf build/\n    cmake -DCMAKE_BUILD_TYPE=Release -S . -B build\n\n- id: build\n  uses: docker://rikorose/gcc-cmake:gcc-9\n  runs: [cmake, --build, build/, --parallel, '4']\n\n- id: test\n  uses: docker://rikorose/gcc-cmake:gcc-9\n  dir: /workspace/build/\n  runs: [ctest]\n```\n\nAssuming the above is stored in a `ci.yml` file in the root of your \nproject folder, this entire workflow gets executed by running:\n\n```bash\npopper run -f ci.yml\n```\n\nRunning a single step:\n\n```bash\npopper run -f ci.yml build\n```\n\nStarting a shell inside the `build` step container:\n\n```bash\npopper run -f ci.yml build\n```\n\nRunning on another engine (Podman):\n\n```bash\npopper run -f ci.yml -e podman build\n```\n\nSee the [`examples/`](./examples) folder for examples for tests for \nother languages, as well as other types of tests (integration, \nregresssion, etc.).\n\n## Installation\n\nTo install or upgrade Popper, run the following in your terminal:\n\n```bash\ncurl -sSfL https://raw.githubusercontent.com/getpopper/popper/master/install.sh | sh\n```\n\n[Docker][docker] is required to run Popper and the installer will \nabort if the `docker` command cannot be invoked from your shell. For \nother installation options, including installing for use with the \nother supported engines (Singularity and Podman), or for setting up a \ndeveloping environment for Popper, [read the complete installation \ninstructions][installation].\n\nOnce installed, you can get an overview and list of available \ncommands:\n\n```bash\npopper help\n```\n\nRead the [Quickstart Guide][getting_started] to learn the basics of \nhow to use Popper. Or browse the [Official documentation][docs].\n\n## Features\n\n  * **Lightweight workflow and task automation syntax.** Defining a list of \n    steps is as simple as writing file in a [lightweight YAML syntax][cnwf] and \n    invoking `popper run` (see demo above). If you're familiar with \n    [Docker Compose][compose], you can think of Popper as Compose but \n    for end-to-end tasks instead of services.\n\n  * **An abstraction over container runtimes**. In addition to Docker, \n    Popper can seamlessly execute workflows in other runtimes by \n    interacting with distinct container engines. Popper currently \n    supports [Docker][docker], [Singularity][sylabs] and \n    [Podman][podman].\n\n  * **An abstraction over CI services**. Define a pipeline once and \n    then instruct Popper to generate configuration files for distinct \n    CI services, allowing users to run the exact same workflows they \n    run locally on Travis, Jenkins, Gitlab, Circle and others. See the \n    [`examples/`](./examples/) folder for examples on how to automate \n    CI tasks for multiple projects (Go, C++, Node, etc.).\n\n  * **An abstraction over resource managers**. Popper can also execute \n    workflows on a variety of resource managers and schedulers such as \n    Kubernetes and SLURM, without requiring any modifications to a \n    workflow YAML file. We currently support SLURM and are working on \n    adding support for Kubernetes.\n\n  * **Aid in workflow development**. Aid in the implementation and \n    [debugging][pp-sh] of workflows, and provide with an extensive \n    list of [example \n    workflows](https://github.com/getpopper/popper-examples) that can \n    serve as a starting point.\n\n## What Problem Does Popper Solve?\n\nPopper is a container-native workflow execution and task automation \nengine. In practice, when we work following the \n[container-native](docs/sections/concepts.md) paradigm, we end up \ninteractively executing multiple Docker commands (`docker pull`, \n`docker build`, `docker run`, etc.) so that we can build containers, \ncompile code, test applications, deploy software, among others. \nKeeping track of which `docker` commands we have executed, in which \norder, and which flags were passed to each, can quickly become \nunmanageable, difficult to document (think of outdated README \ninstructions) and error prone.\n\nOn top of this, having the same workflow work in other environments \n(CI, K8S, etc.) is time-consuming and defeats the purpose of using \ncontainers (portability). The goal of Popper is to bring order to this \nchaotic scenario by providing a framework for clearly and explicitly \ndefining container-native tasks. You can think of Popper as tool for \nwrapping all these manual tasks in a lightweight, machine-readable, \nself-documented format (YAML).\n\nWhile this sounds simple at first, it has significant implications: \nresults in time-savings, improves communication and in general unifies \ndevelopment, testing and deployment workflows. As a developer or user \nof \"Popperized\" container-native projects, you only need to learn one \ntool, and leave the execution details to Popper, whether is to build \nand tests applications locally, on a remote CI server or a Kubernetes \ncluster.\n\n## Contributing\n\nAnyone is welcome to contribute to Popper! To get started, take a look \nat our [contributing guidelines](CONTRIBUTING.md), then dive in with \nour [list of good first issues][gfi].\n\n## Participation Guidelines\n\nPopper adheres to the code of conduct [posted in this \nrepository](CODE_OF_CONDUCT.md). By participating or contributing to \nPopper, you're expected to uphold this code. If you encounter unacceptable \nbehavior, please immediately [email us](mailto:ivotron@ucsc.edu).\n\n## How to Cite Popper\n\n\u003e Ivo Jimenez, Michael Sevilla, Noah Watkins, Carlos Maltzahn, Jay \n\u003e Lofstead, Kathryn Mohror, Andrea Arpaci-Dusseau and Remzi \n\u003e Arpaci-Dusseau. _The Popper Convention: Making Reproducible Systems \n\u003e Evaluation Practical_. In 2017 IEEE International Parallel and \n\u003e Distributed Processing Symposium Workshops (IPDPSW), 1561–70, 2017. \n\u003e (https://doi.org/10.1109/IPDPSW.2017.157)\n\nPDF for a pre-print version [available here](https://raw.githubusercontent.com/systemslab/popper-paper/master/paper/paper.pdf). \nFor BibTeX, [click here](https://raw.githubusercontent.com/systemslab/popper-paper/master/popper.bib).\n\n[gfi]: https://github.com/getpopper/popper/issues?utf8=%E2%9C%93\u0026q=is%3Aissue+label%3A%22good+first+issue%22+is%3Aopen\n[docker]: https://docs.docker.com/get-docker/\n[getting_started]: https://popper.readthedocs.io/en/latest/sections/getting_started.html\n[docs]: https://popper.readthedocs.io/en/latest/\n[sylabs]: https://sylabs.io/\n[compose]: https://docs.docker.com/compose/\n[podman]: https://podman.io\n[pp-sh]: docs/sections/cli_features.md#executing-a-step-interactively\n[installation]: docs/installation.md\n[cnwf]: ./docs/sections/cn_workflows.md#syntax\n","funding_links":[],"categories":["Development with Docker","Python"],"sub_categories":["CI/CD"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetpopper%2Fpopper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgetpopper%2Fpopper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetpopper%2Fpopper/lists"}