{"id":50111416,"url":"https://github.com/npow/metaflow-phala","last_synced_at":"2026-05-23T12:32:27.590Z","repository":{"id":340930371,"uuid":"1168231694","full_name":"npow/metaflow-phala","owner":"npow","description":"Run Metaflow steps inside Phala Cloud TEE CVMs with one decorator","archived":false,"fork":false,"pushed_at":"2026-02-27T06:44:49.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-27T12:50:22.802Z","etag":null,"topics":["confidential-computing","metaflow","mlops","phala","privacy","pypi","python","tee","trusted-execution-environment"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/npow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-27T06:36:50.000Z","updated_at":"2026-02-27T06:44:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/npow/metaflow-phala","commit_stats":null,"previous_names":["npow/metaflow-phala"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/npow/metaflow-phala","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/npow%2Fmetaflow-phala","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/npow%2Fmetaflow-phala/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/npow%2Fmetaflow-phala/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/npow%2Fmetaflow-phala/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/npow","download_url":"https://codeload.github.com/npow/metaflow-phala/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/npow%2Fmetaflow-phala/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33396574,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T04:15:53.637Z","status":"ssl_error","status_checked_at":"2026-05-23T04:15:53.242Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["confidential-computing","metaflow","mlops","phala","privacy","pypi","python","tee","trusted-execution-environment"],"created_at":"2026-05-23T12:32:23.831Z","updated_at":"2026-05-23T12:32:27.585Z","avatar_url":"https://github.com/npow.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# metaflow-phala\n\n\u003e **Work in progress** — not yet production-ready. Integration tests pending.\n\n[![CI](https://github.com/npow/metaflow-phala/actions/workflows/ci.yml/badge.svg)](https://github.com/npow/metaflow-phala/actions/workflows/ci.yml)\n[![PyPI](https://img.shields.io/pypi/v/metaflow-phala)](https://pypi.org/project/metaflow-phala/)\n[![License: Apache-2.0](https://img.shields.io/badge/License-Apache--2.0-blue.svg)](LICENSE)\n[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/) [![Docs](https://img.shields.io/badge/docs-mintlify-18a34a?style=flat-square)](https://mintlify.com/npow/metaflow-phala)\n\nRun any Metaflow step inside a hardware-attested confidential VM with one decorator.\n\n## The problem\n\nYou need to process sensitive data — PII, health records, financial models — in the cloud, but you can't fully trust the infrastructure. Remote workers like `@batch` and `@kubernetes` run in plain VMs where cloud providers or insiders can read your memory. Phala Cloud's TEE CVMs provide hardware-enforced memory encryption and remote attestation, but wiring them into a Metaflow pipeline requires rewriting your execution layer from scratch.\n\n## Quick start\n\n```bash\npip install metaflow-phala\nexport PHALA_API_KEY=your-api-key\nexport METAFLOW_DEFAULT_DATASTORE=s3\nexport METAFLOW_DATASTORE_SYSROOT_S3=s3://your-bucket/metaflow\n```\n\n```python\nfrom metaflow import FlowSpec, step\nfrom metaflow_extensions.phala.plugins.phala_decorator import PhalaDecorator as phala\n\nclass MyFlow(FlowSpec):\n    @phala(cpu=2, memory=4096)\n    @step\n    def train(self):\n        self.result = 42\n        self.next(self.end)\n\n    @step\n    def end(self):\n        print(self.result)\n\nif __name__ == \"__main__\":\n    MyFlow()\n```\n\n```bash\npython flow.py run\n```\n\n## Install\n\n```bash\npip install metaflow-phala\n```\n\nFrom source:\n\n```bash\ngit clone https://github.com/npow/metaflow-phala\ncd metaflow-phala\npip install -e .\n```\n\n## Usage\n\n### Basic step with custom resources\n\n```python\n@phala(cpu=4, memory=8192, disk=40)\n@step\ndef train(self):\n    import sklearn\n    ...\n```\n\n### Pass extra environment variables\n\n```python\n@phala(cpu=2, memory=4096, env={\"HF_TOKEN\": \"hf_...\", \"MY_SECRET\": \"...\"})\n@step\ndef inference(self):\n    ...\n```\n\n### Set a step timeout\n\n```python\n@phala(cpu=2, memory=4096, timeout=1800)  # 30-minute cap\n@step\ndef preprocess(self):\n    ...\n```\n\n### Available parameters\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `image` | `python:3.11-slim` | Docker image |\n| `cpu` | `2` | vCPUs |\n| `memory` | `2048` | Memory in MB |\n| `disk` | `20` | Disk size in GB |\n| `timeout` | `3600` | Max seconds to wait |\n| `env` | `{}` | Extra environment variables |\n\nAll defaults can be overridden via environment variables: `METAFLOW_PHALA_IMAGE`, `METAFLOW_PHALA_CPU`, `METAFLOW_PHALA_MEMORY`, `METAFLOW_PHALA_DISK`, `METAFLOW_PHALA_TIMEOUT`.\n\n## How it works\n\nWhen Metaflow runs a `@phala`-decorated step, instead of running locally it:\n\n1. Uploads the code package to S3 (once per flow run)\n2. Provisions a Phala Cloud CVM via the REST API\n3. Starts the CVM with a docker-compose spec that embeds all env vars and a base64-encoded bootstrap script\n4. The container downloads the code package, installs dependencies, and runs the Metaflow step\n5. On completion, the exit code is written to an S3 sentinel key\n6. The local process polls the sentinel, retrieves the result, and deletes the CVM\n\nArtifacts are stored in your S3 datastore as usual — identical to `@batch` or `@kubernetes`.\n\n## Development\n\n```bash\ngit clone https://github.com/npow/metaflow-phala\ncd metaflow-phala\npip install -e \".[dev]\"\npytest -v  # unit tests\n```\n\nIntegration tests require a Phala API key and S3:\n\n```bash\nPHALA_API_KEY=... METAFLOW_DEFAULT_DATASTORE=s3 METAFLOW_DATASTORE_SYSROOT_S3=s3://... \\\npytest tests/ -m integration -s\n```\n\n## License\n\n[Apache 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnpow%2Fmetaflow-phala","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnpow%2Fmetaflow-phala","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnpow%2Fmetaflow-phala/lists"}