{"id":13534477,"url":"https://github.com/ploomber/ploomber","last_synced_at":"2025-04-29T18:49:48.911Z","repository":{"id":36964158,"uuid":"235190220","full_name":"ploomber/ploomber","owner":"ploomber","description":"The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️","archived":false,"fork":false,"pushed_at":"2024-09-18T19:11:09.000Z","size":7059,"stargazers_count":3559,"open_issues_count":108,"forks_count":238,"subscribers_count":30,"default_branch":"master","last_synced_at":"2025-04-08T21:18:10.463Z","etag":null,"topics":["data-engineering","data-science","jupyter","jupyter-notebooks","machine-learning","mlops","notebooks","papermill","pipelines","pycharm","vscode","workflow"],"latest_commit_sha":null,"homepage":"https://docs.ploomber.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ploomber.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-20T20:13:06.000Z","updated_at":"2025-04-06T02:37:47.000Z","dependencies_parsed_at":"2023-01-17T09:01:49.539Z","dependency_job_id":"26f165f5-2017-46e6-85d4-3ee0258dd94a","html_url":"https://github.com/ploomber/ploomber","commit_stats":{"total_commits":2963,"total_committers":83,"mean_commits":35.69879518072289,"dds":0.503206209922376,"last_synced_commit":"b230fa672f2fa3afb2820b4e242b9cec211e2078"},"previous_names":[],"tags_count":116,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ploomber%2Fploomber","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ploomber%2Fploomber/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ploomber%2Fploomber/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ploomber%2Fploomber/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ploomber","download_url":"https://codeload.github.com/ploomber/ploomber/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251564294,"owners_count":21609876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","data-science","jupyter","jupyter-notebooks","machine-learning","mlops","notebooks","papermill","pipelines","pycharm","vscode","workflow"],"created_at":"2024-08-01T07:01:33.957Z","updated_at":"2025-04-29T18:49:48.888Z","avatar_url":"https://github.com/ploomber.png","language":"Python","readme":"\u003cp align=\"center\" width=\"100%\"\u003e\n  \u003cimg src=\"_static/logo.png\" height=\"250\"\u003e\n\u003c/p\u003e\n\n[![CI Linux](https://github.com/ploomber/ploomber/actions/workflows/ci-unit-linux.yml/badge.svg)](https://github.com/ploomber/ploomber/actions/workflows/ci-unit-linux.yml/badge.svg)\n[![CI macOS](https://github.com/ploomber/ploomber/actions/workflows/ci-unit-macos.yml/badge.svg)](https://github.com/ploomber/ploomber/actions/workflows/ci-unit-macos.yml/badge.svg)\n[![Documentation Status](https://readthedocs.org/projects/ploomber/badge/?version=latest)](https://docs.ploomber.io/en/latest/?badge=latest)\n[![PyPI](https://badge.fury.io/py/ploomber.svg)](https://badge.fury.io/py/ploomber)\n[![Conda (channel only)](https://img.shields.io/conda/vn/conda-forge/ploomber)](https://anaconda.org/conda-forge/ploomber)\n[![Conda](https://img.shields.io/conda/pn/conda-forge/ploomber)](https://anaconda.org/conda-forge/ploomber)\n[![Coverage](https://coveralls.io/repos/github/ploomber/ploomber/badge.svg?branch=master)](https://coveralls.io/github/ploomber/ploomber?branch=master)\n[![Twitter](https://img.shields.io/twitter/follow/ploomber?label=Follow\u0026style=social)](https://twitter.com/intent/user?screen_name=ploomber)\n[![Downloads](https://pepy.tech/badge/ploomber)](https://pepy.tech/project/ploomber)\n\n\u003e [!TIP]\n\u003e Deploy AI apps for free on [Ploomber Cloud!](https://ploomber.io/?utm_medium=github\u0026utm_source=ploomber)\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://ploomber.io/community\"\u003eJoin our community\u003c/a\u003e\n  |\n  \u003ca href=\"https://share.hsforms.com/1E7Qa_OpcRPi_MV-segFsaAe6c2g\"\u003eNewsletter\u003c/a\u003e\n  |\n  \u003ca href=\"mailto:contact@ploomber.io\"\u003eContact us\u003c/a\u003e\n  |\n  \u003ca href=\"https://docs.ploomber.io/\"\u003eDocs\u003c/a\u003e\n  |\n  \u003ca href=\"https://ploomber.io/blog/\"\u003eBlog\u003c/a\u003e\n  |  \n  \u003ca href=\"https://www.ploomber.io\"\u003eWebsite\u003c/a\u003e\n  |\n  \u003ca href=\"https://www.youtube.com/channel/UCaIS5BMlmeNQE4-Gn0xTDXQ\"\u003eYouTube\u003c/a\u003e\n\u003c/p\u003e\n\n\nPloomber is the fastest way to build data pipelines ⚡️. Use your favorite editor (**[Jupyter](https://docs.ploomber.io/en/latest/user-guide/jupyter.html), [VSCode](https://docs.ploomber.io/en/latest/user-guide/editors.html), [PyCharm](https://docs.ploomber.io/en/latest/user-guide/editors.html)**) to develop interactively and deploy ☁️ without code changes (**[Kubernetes](https://soopervisor.readthedocs.io/en/latest/tutorials/kubernetes.html), [Airflow](https://soopervisor.readthedocs.io/en/latest/tutorials/airflow.html), [AWS Batch](https://soopervisor.readthedocs.io/en/latest/tutorials/aws-batch.html), and [SLURM](https://soopervisor.readthedocs.io/en/latest/tutorials/slurm.html)**). Do you have legacy notebooks? Refactor them into modular pipelines with a single command.\n\n\n## Installation\n\n*Compatible with Python 3.7 and higher.*\n\nInstall with `pip`:\n\n```sh\npip install ploomber\n```\n\nOr with `conda`:\n\n```sh\nconda install ploomber -c conda-forge\n```\n\n## Getting started\n\n### Try the tutorial:\n\u003cp\u003e\n  \u003ca href=\"https://github.com/ploomber/projects/blob/master/guides/first-pipeline/README.md\"\u003e \u003c/a\u003e\n\u003c/p\u003e\n\n## Community\n\n* [Join us on Slack](https://ploomber.io/community)\n* [Newsletter](https://share.hsforms.com/1E7Qa_OpcRPi_MV-segFsaAe6c2g)\n* [YouTube](https://www.youtube.com/channel/UCaIS5BMlmeNQE4-Gn0xTDXQ)\n* [Contact the development team](mailto:contact@ploomber.io)\n\n## Main Features\n\n### ⚡️ Get started quickly\n\nA simple YAML API to get started quickly, a powerful Python API for total flexibility.\n\nhttps://user-images.githubusercontent.com/989250/150660813-fc289c6c-0ed5-432d-b6df-063ce98c0093.mp4\n\n### ⏱ Shorter development cycles\n\nAutomatically cache your pipeline’s previous results and only re-compute tasks that have changed since your last execution.\n\nhttps://user-images.githubusercontent.com/989250/150660820-9a3a0abd-5904-492b-97ff-5494285dfebf.mp4\n\n### ☁️ Deploy anywhere\n\nRun as a shell script in a single machine or distributively in [Kubernetes](https://soopervisor.readthedocs.io/en/latest/tutorials/kubernetes.html), [Airflow](https://soopervisor.readthedocs.io/en/latest/tutorials/airflow.html), [AWS Batch](https://soopervisor.readthedocs.io/en/latest/tutorials/aws-batch.html), or [SLURM](https://soopervisor.readthedocs.io/en/latest/tutorials/slurm.html).\n\nhttps://user-images.githubusercontent.com/989250/150660830-3f81c9a2-5392-49e5-976d-cb8a38441ecb.mp4\n\n\n### 📙 Automated migration from legacy notebooks\n\nBring your old monolithic notebooks, and we’ll automatically convert them into maintainable, modular pipelines.\n\nhttps://user-images.githubusercontent.com/989250/150660840-b0c12f85-504c-4233-8c3d-6724d291f1aa.mp4\n\n\n[I want to migrate my notebook.](https://docs.ploomber.io/en/latest/user-guide/refactoring.html)\n\n[Show me a demo.](https://www.youtube.com/watch?v=EJecqsZBr3Q)\n\n## Resources\n\n* [Documentation](https://docs.ploomber.io/)\n* [PyData Chicago talk (covers motivation and demo)](https://youtu.be/qUL7QabcKcw)\n* [Develop and deploy an ML pipeline in 30 minutes (EuroPython 2021)](https://youtu.be/O8tqiCkIWPs)\n* [Guest blog post on the official Jupyter blog](https://blog.jupyter.org/ploomber-maintainable-and-collaborative-pipelines-in-jupyter-acb3ad2101a7)\n* [Examples (Machine Learning pipeline, ETL, among others)](https://github.com/ploomber/projects)\n* [Blog](https://ploomber.io/)\n* [Comparison with other tools](https://ploomber.io/posts/survey)\n* [More videos](https://docs.ploomber.io/en/latest/videos.html)\n\n## About Ploomber\n\nPloomber is a big community of data enthusiasts pushing the boundaries of Data Science and Machine Learning tooling.\n\nWhatever your skillset is, you can contribute to our mission. So whether you're a beginner or an experienced professional, you're welcome to join us on this journey!\n\n[Click here to know how you can contribute to Ploomber.](https://github.com/ploomber/contributing/blob/main/README.md)\n","funding_links":[],"categories":["Rendering/Publishing/Conversion","Python","Large Scale Deployment","Ecosystem Projects","data-science","Task Management Tools","Workflow","Data Management \u0026 Processing","Data Pipeline","数据管道和流处理","其他__大数据","Workflow Tools","workflow","Data Pipelines \u0026 Streaming","Uncategorized"],"sub_categories":["Workflow","Tags: Object Classification `[ObjCls]`, Object Detection `[ObjDet]`, Object Segmentation `[ObjSeg]`, General Library `[GenLib]`, Text Reading / Object Character Recognition `[OCR]`, Action Recognition `[ActRec]`, Object Tracking `[ObjTrk]`, Data Augmentation `[DatAug]`, Simultaneous Localization and Mapping `[SLAM]`, Outlier/Anomaly/Novelty Detection `[NvlDet]`, Content-based Image Retrieval `[CBIR]`, Image Enhancement `[ImgEnh]`, Aesthetic Assessment `[AesAss]`, Explainable Artificial Intelligence `[XAI]`, Text-to-Image Generation `[TexImg]`, Pose Estimation `[PosEst]`, Video Matting `[VidMat]`, Eye Tracking `[EyeTrk]`","Database \u0026 Cloud Management","网络服务_其他","Uncategorized"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fploomber%2Fploomber","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fploomber%2Fploomber","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fploomber%2Fploomber/lists"}