{"id":50476227,"url":"https://github.com/frankxai/arcanea-claw","last_synced_at":"2026-06-01T13:30:51.313Z","repository":{"id":349240588,"uuid":"1183817950","full_name":"frankxai/arcanea-claw","owner":"frankxai","description":"ArcaneaClaw — AI-powered Creator Media Engine. Scans, classifies, scores, and publishes creative media 24/7. Works with OpenClaw, Railway, Docker, Claude Code.","archived":false,"fork":false,"pushed_at":"2026-04-04T22:59:23.000Z","size":193,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-05T00:25:07.336Z","etag":null,"topics":["ai-agent","arcanea","creative-tools","image-processing","mcp","media-processing","openclaw","railway"],"latest_commit_sha":null,"homepage":"https://arcanea.ai/claw","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frankxai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-17T01:24:57.000Z","updated_at":"2026-04-04T22:59:29.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/frankxai/arcanea-claw","commit_stats":null,"previous_names":["frankxai/arcanea-claw"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/frankxai/arcanea-claw","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankxai%2Farcanea-claw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankxai%2Farcanea-claw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankxai%2Farcanea-claw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankxai%2Farcanea-claw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frankxai","download_url":"https://codeload.github.com/frankxai/arcanea-claw/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankxai%2Farcanea-claw/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33777967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agent","arcanea","creative-tools","image-processing","mcp","media-processing","openclaw","railway"],"created_at":"2026-06-01T13:30:50.663Z","updated_at":"2026-06-01T13:30:51.305Z","avatar_url":"https://github.com/frankxai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ArcaneaClaw\r\n\r\n24/7 AI-powered media processing engine for creative world-builders. Scan, classify, deduplicate, transform, score, and deploy creative assets — fully automated.\r\n\r\n[![Deploy on Railway](https://railway.com/button.svg)](https://railway.com/template/arcanea-claw?referralCode=arcanea)\r\n[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2Ffrankxai%2Farcanea-claw\u0026env=SUPABASE_URL,SUPABASE_SERVICE_KEY,GEMINI_API_KEY\u0026project-name=arcanea-claw\u0026repository-name=arcanea-claw)\r\n\r\n---\r\n\r\n## What is ArcaneaClaw?\r\n\r\nArcaneaClaw is a containerized daemon that watches source directories for new media files and runs them through an 8-skill processing pipeline: scan, AI classify (Gemini), deduplicate (perceptual hashing), process (resize/convert to WebP), aesthetic score (TASTE), upload (Supabase Storage + Vercel Blob), generate social variants, and notify (Discord/Slack webhooks). It runs on 2GB RAM, reports health via HTTP, and registers itself in Supabase so the Command Center dashboard can monitor it remotely.\r\n\r\n## Architecture\r\n\r\n```\r\n                        ArcaneaClaw Engine\r\n                        ==================\r\n\r\n  Source Files          8-Skill Pipeline            Destinations\r\n  ============         ==================          ==============\r\n\r\n  Google Drive    +--\u003e scan --------+\r\n  Local Folder   -+   classify      |  Gemini AI     Supabase Storage\r\n  S3 / Volume         dedup         |  (vision)      Vercel Blob\r\n                      process       |  WebP/resize   Social Platforms\r\n                      score         |  TASTE 0-100   Webhook Notify\r\n                      upload        |\r\n                      social_prep   |\r\n                      notify -------+\r\n\r\n  Health: GET /health (port 8080)\r\n  Heartbeat: Supabase agent_registry every 5 min\r\n  Pipeline: Full chain every 15 min (configurable)\r\n\r\n  ┌──────────────────────────────────────────────┐\r\n  │  Command Center (Vercel)                     │\r\n  │  Web UI for monitoring, config, asset browse │\r\n  └──────────────────────────────────────────────┘\r\n           |\r\n           | reads from\r\n           v\r\n  ┌──────────────────────────────────────────────┐\r\n  │  Supabase                                    │\r\n  │  agent_registry, asset_queue, media_assets   │\r\n  └──────────────────────────────────────────────┘\r\n           ^\r\n           | writes to\r\n           |\r\n  ┌──────────────────────────────────────────────┐\r\n  │  ArcaneaClaw Engine (Railway / Docker)       │\r\n  │  Python 3.11 + Node 20 + FFmpeg              │\r\n  │  Non-root \"claw\" user, /data persistent vol  │\r\n  └──────────────────────────────────────────────┘\r\n```\r\n\r\n## Quick Start\r\n\r\n### Option 1: Railway One-Click (Recommended)\r\n\r\nClick the button above. Railway will prompt you for three required variables:\r\n\r\n| Variable | Where to find it |\r\n|----------|-----------------|\r\n| `SUPABASE_URL` | Supabase Dashboard \u003e Settings \u003e API |\r\n| `SUPABASE_SERVICE_KEY` | Supabase Dashboard \u003e Settings \u003e API \u003e service_role |\r\n| `GEMINI_API_KEY` | [Google AI Studio](https://aistudio.google.com/apikey) |\r\n\r\nAdd a persistent volume mounted at `/data` in Railway settings. The engine starts automatically.\r\n\r\n### Option 2: Docker Compose\r\n\r\n```bash\r\ngit clone https://github.com/frankxai/arcanea-claw.git\r\ncd arcanea-claw\r\ncp .env.example .env\r\n# Edit .env with your keys\r\n\r\ndocker compose up -d\r\n```\r\n\r\nVerify it's running:\r\n\r\n```bash\r\ncurl http://localhost:8080/health\r\n```\r\n\r\n### Option 3: Native Python\r\n\r\n```bash\r\ngit clone https://github.com/frankxai/arcanea-claw.git\r\ncd arcanea-claw\r\n\r\npip install -r requirements.txt\r\nnpm install --production\r\n\r\n# Create data directories\r\nmkdir -p /data/{source,staging,processed,logs}\r\n\r\n# Set environment variables (or use .env file)\r\nexport SUPABASE_URL=\"https://your-project.supabase.co\"\r\nexport SUPABASE_SERVICE_KEY=\"your-service-role-key\"\r\nexport GEMINI_API_KEY=\"your-gemini-key\"\r\n\r\npython engine/daemon.py\r\n```\r\n\r\n## Skills Reference\r\n\r\nThe pipeline runs these skills sequentially every 15 minutes (configurable via `PIPELINE_INTERVAL`):\r\n\r\n| # | Skill | Module | Description |\r\n|---|-------|--------|-------------|\r\n| 1 | **Scan** | `media_scan` | Walk source directories, discover new files by extension |\r\n| 2 | **Classify** | `media_classify` | Gemini Vision AI identifies guardian, element, and tags |\r\n| 3 | **Dedup** | `media_dedup` | Perceptual hash comparison prevents duplicate processing |\r\n| 4 | **Process** | `media_process` | Resize to 6 variants, convert to WebP at 85% quality |\r\n| 5 | **Score** | `taste_score` | TASTE aesthetic scoring (0-100) for quality gating |\r\n| 6 | **Upload** | `media_upload` | Hero (score 80+) to Vercel Blob, gallery (60+) to Supabase |\r\n| 7 | **Social Prep** | `social_prep` | Generate platform-specific sizes (story, square, wide) |\r\n| 8 | **Notify** | `notify` | Send pipeline results to Discord/Slack webhooks |\r\n\r\n### Adding a Custom Skill\r\n\r\nCreate `engine/skills/\u003cname\u003e.py` with an async `run(config)` function, then add the module name to the `SKILL_CHAIN` list in `engine/daemon.py`.\r\n\r\n```python\r\nasync def run(config: dict) -\u003e dict:\r\n    \"\"\"Your custom skill logic.\"\"\"\r\n    # Access config values\r\n    output_dir = config.get(\"process\", {}).get(\"output_dir\", \"/data/processed\")\r\n\r\n    # Do work...\r\n    return {\"processed\": 42, \"status\": \"complete\"}\r\n```\r\n\r\n## Configuration\r\n\r\n### Environment Variables\r\n\r\n| Variable | Required | Default | Description |\r\n|----------|----------|---------|-------------|\r\n| `SUPABASE_URL` | Yes | - | Supabase project URL |\r\n| `SUPABASE_SERVICE_KEY` | Yes | - | Supabase service role key |\r\n| `GEMINI_API_KEY` | Yes | - | Google Gemini API key |\r\n| `VERCEL_TOKEN` | No | - | Vercel API token for Blob uploads |\r\n| `BLOB_READ_WRITE_TOKEN` | No | - | Vercel Blob read/write token |\r\n| `NOTIFY_WEBHOOK_URL` | No | - | Discord or Slack webhook URL |\r\n| `PIPELINE_INTERVAL` | No | `900` | Seconds between pipeline runs |\r\n| `HEARTBEAT_INTERVAL` | No | `300` | Seconds between heartbeat pings |\r\n| `ARCANEA_CLAW_CONFIG` | No | `/app/config.yaml` | Path to config file |\r\n\r\n### config.yaml\r\n\r\nThe config file controls scan paths, processing parameters, upload thresholds, and classification settings. See `config.yaml` for the full reference with comments.\r\n\r\nKey sections:\r\n\r\n- **scan.paths** — directories to watch for new media\r\n- **scan.extensions** — file types to process (.jpg, .png, .webp, .mp4, etc.)\r\n- **process.size_variants** — output dimensions (hero, gallery, social_square, etc.)\r\n- **upload.hero_threshold** — minimum TASTE score for Vercel Blob (default: 80)\r\n- **upload.gallery_threshold** — minimum TASTE score for Supabase Storage (default: 60)\r\n- **classify.model** — Gemini model for vision classification (default: gemini-2.0-flash)\r\n\r\n## API Reference\r\n\r\n### Health Endpoint\r\n\r\n```\r\nGET /health\r\n```\r\n\r\nReturns daemon state as JSON:\r\n\r\n```json\r\n{\r\n  \"status\": \"online\",\r\n  \"started_at\": 1710648000.0,\r\n  \"last_pipeline_at\": 1710648900.0,\r\n  \"last_heartbeat_at\": 1710648600.0,\r\n  \"pipeline_runs\": 12,\r\n  \"errors\": 0\r\n}\r\n```\r\n\r\n### MCP Tools\r\n\r\nArcaneaClaw includes an MCP server (`mcp-server/`) that gives Claude Code native tool access to the pipeline. Install it with:\r\n\r\n```bash\r\nclaude mcp add arcanea-claw -- node mcp-server/dist/index.js\r\n```\r\n\r\nAvailable tools:\r\n\r\n| Tool | Description |\r\n|------|-------------|\r\n| `claw_status` | Get engine health and pipeline statistics |\r\n| `claw_scan` | Trigger a scan of source directories |\r\n| `claw_classify` | Classify an asset with Gemini Vision |\r\n| `claw_score` | Get TASTE aesthetic score for an image |\r\n| `claw_process` | Process a single asset through the pipeline |\r\n| `claw_upload` | Upload a processed asset to storage |\r\n| `claw_queue` | View the asset processing queue |\r\n| `claw_config` | Read or update runtime configuration |\r\n\r\n## Skillpacks\r\n\r\nPre-built skill bundles for common workflows:\r\n\r\n| Skillpack | Description |\r\n|-----------|-------------|\r\n| `arcanea-media-pipeline` | Core 8-skill pipeline for media processing |\r\n| `arcanea-taste-scorer` | Standalone TASTE aesthetic scoring |\r\n| `arcanea-social-prep` | Social media variant generation |\r\n| `arcanea-command-center` | Web dashboard for monitoring and control |\r\n\r\nSee `skillpacks/README.md` for installation and configuration details.\r\n\r\n## Deployment\r\n\r\n### Railway\r\n\r\nThe `railway.json` template configures everything automatically: Dockerfile build, health checks, restart policy, and environment variable prompts. Add a persistent volume at `/data` after deploy.\r\n\r\n### Docker / Podman\r\n\r\nBoth `Dockerfile` (Docker) and `Containerfile` (Podman) are provided. The `docker-compose.yml` includes volume mounts for Google Drive integration and resource limits (2 CPU, 2GB RAM).\r\n\r\nFor Podman rootless:\r\n\r\n```bash\r\npodman-compose -f podman-compose.yml up -d\r\n```\r\n\r\n### Resource Requirements\r\n\r\n| Resource | Minimum | Recommended |\r\n|----------|---------|-------------|\r\n| CPU | 1 core | 2 cores |\r\n| RAM | 1 GB | 2 GB |\r\n| Disk | 1 GB | 10 GB (for /data volume) |\r\n| Network | Outbound HTTPS | Outbound HTTPS |\r\n\r\n## Project Structure\r\n\r\n```\r\narcanea-claw/\r\n  engine/\r\n    daemon.py              Main event loop (heartbeat + pipeline + health)\r\n    supabase_client.py     Supabase DB operations (agent registry, assets)\r\n    skills/                Skill modules (media_scan, media_classify, etc.)\r\n  skills/                  Skill packages (scan, classify, dedup, process...)\r\n  skillpacks/              Pre-built skill bundles\r\n  mcp-server/              MCP server for Claude Code integration\r\n  deploy/\r\n    railway-setup.sh       Post-deploy initialization script\r\n    vercel.json            Vercel config for Command Center UI\r\n  config.yaml              Pipeline configuration\r\n  Dockerfile               Multi-stage production build\r\n  Containerfile            Podman-compatible build\r\n  docker-compose.yml       Docker local deployment\r\n  podman-compose.yml       Podman rootless deployment\r\n  railway.json             Railway template configuration\r\n  .env.example             Environment variable template\r\n```\r\n\r\n## Contributing\r\n\r\n1. Fork the repository\r\n2. Create a feature branch (`git checkout -b feat/my-skill`)\r\n3. Add your skill to `engine/skills/` with an async `run(config)` function\r\n4. Add it to the `SKILL_CHAIN` in `engine/daemon.py`\r\n5. Test locally with `docker compose up`\r\n6. Open a pull request\r\n\r\n## License\r\n\r\nSee [LICENSE](../LICENSE) in the root Arcanea repository.\r\n\r\n---\r\n\r\nBuilt by [FrankX](https://arcanea.ai) as part of the [Arcanea](https://arcanea.ai) creative multiverse.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankxai%2Farcanea-claw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrankxai%2Farcanea-claw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankxai%2Farcanea-claw/lists"}