{"id":18644293,"url":"https://github.com/benmizrahi/paperless","last_synced_at":"2026-04-24T23:32:30.685Z","repository":{"id":219761330,"uuid":"749848148","full_name":"benmizrahi/paperless","owner":"benmizrahi","description":"A papermill implementation to run notebooks inside dataproc serverless","archived":false,"fork":false,"pushed_at":"2024-06-12T18:58:00.000Z","size":187,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-13T12:52:03.128Z","etag":null,"topics":["dataproc","gcp","jupyter","jupyter-notebook","notebook","python","serverless"],"latest_commit_sha":null,"homepage":"https://medium.com/@benm-23166/paperless-dataproc-serverless-meets-jupyter-3c44c6458789","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benmizrahi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-29T14:17:30.000Z","updated_at":"2024-06-12T18:58:16.000Z","dependencies_parsed_at":"2024-06-13T01:59:05.936Z","dependency_job_id":null,"html_url":"https://github.com/benmizrahi/paperless","commit_stats":{"total_commits":18,"total_committers":1,"mean_commits":18.0,"dds":0.0,"last_synced_commit":"0bedb8bc05f7a43d05304c122380dcdf5869f3c2"},"previous_names":["benmizrahi/paperless"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/benmizrahi/paperless","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benmizrahi%2Fpaperless","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benmizrahi%2Fpaperless/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benmizrahi%2Fpaperless/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benmizrahi%2Fpaperless/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benmizrahi","download_url":"https://codeload.github.com/benmizrahi/paperless/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benmizrahi%2Fpaperless/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32245132,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataproc","gcp","jupyter","jupyter-notebook","notebook","python","serverless"],"created_at":"2024-11-07T06:11:12.601Z","updated_at":"2026-04-24T23:32:30.671Z","avatar_url":"https://github.com/benmizrahi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Paperless\n\n[Paperless](https://github.com/Plarium-Repo/paperless.git) is a tool that extends the capabilities of [Papermill](https://papermill.readthedocs.io/) by providing the ability to run Papermill via [Google Cloud Dataproc Serverless](https://cloud.google.com/dataproc-serverless/docs).   \n\n\nhttps://medium.com/@benm-23166/paperless-dataproc-serverless-meets-jupyter-3c44c6458789\n\n## Overview\n\nPapermill is a powerful tool for parameterizing and executing Jupyter Notebooks. However, by default papermill dosn't support [Jupyter Kernel Gateway](https://jupyter-kernel-gateway.readthedocs.io/en/latest/) - it was impossible to run spark notebook vs Google Cloud Dataproc Serverless environment with Papermill tool - this is where Paperless helps.\n\nPaperless bridges the gap between Papermill and Google Cloud Dataproc Serverless interactive mode, allowing you to seamlessly integrate the two and harness the power of serverless execution for your Jupyter Notebooks.\n\n## Features\n\n- **Serverless Execution:** Run Papermill on Google Cloud Dataproc without managing the underlying infrastructure.\n\n- **Scalability:** Leverage the scalability of Google Cloud Dataproc for processing multiple Notebooks concurrently.\n\n- **Cost-Effective:** Pay only for the resources you consume during the execution, optimizing costs for your notebook parameterization tasks.\n\nPricing model is much more cost-effective considering other platofrms and technologies \n\n![https://cloud.google.com/dataproc-serverless/pricing](./images/pricing_model.png)\n*resource*: https://cloud.google.com/dataproc-serverless/pricing\n\n## Getting Started\n\n### Prerequisites\n\nBefore using Paperless, make sure you have the following:\n\n- A Google Cloud Platform (GCP) project\n- Access to Google Cloud Dataproc  [Enable the API](https://console.cloud.google.com/flows/enableapi?apiid=dataproc)\n- Papermill installed locally or in your environment\n\n\n### Step 1: Install Google Cloud SDK\n\nTo authenticate your application using Application Default Credentials (ADC) with `gcloud` -\nIf you haven't already installed the Google Cloud SDK, you can download and install it from the [Google Cloud SDK documentation](https://cloud.google.com/sdk/docs/install).\n\n### Step 2: Authenticate with `gcloud`\n\nOpen a terminal and run the following command to authenticate your Google Cloud SDK with your Google Cloud Platform (GCP) account:\n\n```bash\ngcloud auth login\n\ngcloud auth application-default login \n\ngcloud config set project \u003cproject_id\u003e\n\ngcloud config set compute/region \u003cregion\u003e\n\ngcloud config set dataproc/region \u003cregion\u003e\n\n```\n\n### Step 3: Install Paperless \n\n```bash\npip install paperless\n```\n\n### Step 4: Create sessionTemplates For Paperless\n\nParameters and details can be found in [GCP Docs](https://cloud.google.com/sdk/gcloud/reference/compute/instance-templates/create).  \n\nThe default template name is:  *paperless-interactive*\n```bash\n gcloud compute instance-templates create paperless-interactive --\u003cextra params...\u003e\n``` \n\nYou can change parameters as you need based on the jobs needs - check the docs for that.\n\n\n### Step 5: Test Executtion:\n\nPaperless excepts \u0026\u0026 supports all list or arguments exists in original [Papermill](https://papermill.readthedocs.io/) package -\nthe minimum needed for testing:\n\n```bash\n paperless \u003cinput_path\u003e \u003coutput_path\u003e ...\n``` \nAn extra environment variable  that is special for Paperless:  TEMPLATE_NAME=paperless-interactive\nExample:\n\n```bash\n export TEMPLATE_NAME=paperless-interactive \u0026\u0026 \\\n     paperless ./resources/spark.ipynb ./resources/spark-out.ipynb\n\n```\n\nYou're all set, enjoy :)\n\n----\n\n### Local development:\n\n```bash\n\n# Create a new directory for your project\ngit clone https://github.com/Plarium-Repo/paperless.git \u0026\u0026 cd paperless\n\n# Create a virtual environment\npython3 -m venv .venv\n\n# Activate the virtual environment\n# On Windows\n.venv\\Scripts\\activate\n\n# On macOS and Linux\nsource venv/bin/activate\n\n# Install requirements\npip install -r requirements.txt\n\n# Install the command line\npython setup.py install \n\n# Execute example\nexport TEMPLATE_NAME=paperless-interactive \u0026\u0026 ./.venv/bin/paperless ./tests/resources/test.ipynb ./tests/resources/test-out.ipynb\n\n```\n\nMade With Love ❤️ from  :israel: :israel:      \n\n---\n[MIT License](LICENSE)\n\n[Contribution](CONTRIBUTION)\n\n[Code Of Conduct](CODE_OF_CONDUCT)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenmizrahi%2Fpaperless","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenmizrahi%2Fpaperless","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenmizrahi%2Fpaperless/lists"}