{"id":49223234,"url":"https://github.com/khamel83/penny","last_synced_at":"2026-04-24T05:04:13.055Z","repository":{"id":330481237,"uuid":"1120744476","full_name":"Khamel83/penny","owner":"Khamel83","description":"Voice middleware for Apple: speak naturally → items land in Reminders and Notes automatically. iPhone voice memos or Google Home → LLM classification → Apple's native apps.","archived":false,"fork":false,"pushed_at":"2026-04-23T18:15:16.000Z","size":2661,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-23T19:34:22.794Z","etag":null,"topics":["apple-notes","apple-reminders","automation","google-home","llm","macos","voice-assistant","whisper"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Khamel83.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-21T21:19:06.000Z","updated_at":"2026-04-23T18:15:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Khamel83/penny","commit_stats":null,"previous_names":["khamel83/penny"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Khamel83/penny","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Khamel83%2Fpenny","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Khamel83%2Fpenny/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Khamel83%2Fpenny/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Khamel83%2Fpenny/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Khamel83","download_url":"https://codeload.github.com/Khamel83/penny/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Khamel83%2Fpenny/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32209897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T03:15:14.334Z","status":"ssl_error","status_checked_at":"2026-04-24T03:15:11.608Z","response_time":64,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apple-notes","apple-reminders","automation","google-home","llm","macos","voice-assistant","whisper"],"created_at":"2026-04-24T05:03:58.994Z","updated_at":"2026-04-24T05:04:13.049Z","avatar_url":"https://github.com/Khamel83.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Penny\n\nPenny is voice capture middleware for Apple's native apps.\n\nYou speak naturally, Penny transcribes and routes it, and the result lands in Apple Reminders or Apple Notes. The primary flow is Apple Watch Voice Memos syncing to a Mac mini. The system is optimized for reliability over speed.\n\n```\nVoice front-end            →  Penny (middleware)  →  Apple back-end\n────────────────────────────────────────────────────────────────────\nApple Watch Voice Memo     →  transcribe + classify  →  Reminders / Notes\nGoogle Home                →  classify               →  Reminders / Notes\n```\n\n---\n\nCanonical documentation:\n- `docs/README.md`\n- `docs/reliability.md`\n- `docs/macmini-deployment.md`\n- `docs/troubleshooting.md`\n\n---\n\n## The architecture\n\n**Front-ends** (how you speak to it):\n- **Apple Watch Voice Memo** — primary input path, synced through Voice Memos/iCloud to the Mac mini\n- **Google Home** — add to \"My Tasks\", give any time when prompted\n\n**Middleware** (Penny, running on an always-on Apple Silicon Mac):\n- Transcribes audio locally with Whisper (no cloud, no cost)\n- Classifies free-form speech into actionable items using an LLM\n- Routes each item to the right Apple list\n\n**Back-end** (Apple's native apps, synced to all your devices via iCloud):\n- **Apple Reminders** — for actionable items, sorted by category\n- **Apple Notes** (Penny folder) — for everything else: thoughts, observations, things that aren't tasks\n\n## Repo Map\n\n- `watcher.py` — primary Apple Watch Voice Memos ingest path\n- `transcript_log.py` — SQLite persistence, dedup, ingest state, retry metadata\n- `core.py` — shared routing pipeline\n- `classifier.py` — content-type detection and item extraction\n- `reminders.py` — AppleScript bridge to Notes and Reminders\n- `tasks_poller.py` — Google Home / Google Tasks ingest path\n- `webhook/server.py` — optional direct-upload ingest path\n- `scripts/` — auth, export, and validation helpers\n- `launchd/` — launch agent templates for the Mac mini\n- `docs/` — canonical product and operations documentation\n\n---\n\n## Routing\n\n| What you say | Where it goes |\n|---|---|\n| \"get milk, eggs, sausages\" | Reminders → Groceries |\n| \"call dentist, pick up dry cleaning\" | Reminders → Health, Errands |\n| \"fix the leaky faucet\" | Reminders → Home |\n| \"expense report due Friday\" | Reminders → Work |\n| \"the weather today is beautiful\" | Notes → Penny folder |\n| Any pure thought or observation | Notes → Penny folder |\n| Short ambiguous memo | Notes → Penny folder + Reminders → Inbox |\n\nReminders lists: Groceries, Errands, Home, Health, Work, Kids, Inbox\n\nOne input can produce multiple routed items — \"get milk, call dentist, fix faucet\" becomes three reminders in three different lists.\n\nReliability rules:\n- Voice Memos on Apple Watch is the primary ingest path.\n- Memo duration is used as a soft routing signal, never a hard rule.\n- Short ambiguous memos always go to Notes and also create an Inbox reminder with a timestamped excerpt.\n- Reliability is prioritized over raw speed.\n\n---\n\n## Google Home constraint — read this before touching anything\n\n\u003e **⚠️ This is the only way it works. Do not change it.**\n\u003e\n\u003e Google Home will only write to the default Google Tasks list, which must be named **\"My Tasks\"**.\n\u003e Google locked down every other integration path between 2022 and 2023 — custom list names,\n\u003e third-party apps, IFTTT variable capture — all gone. \"My Tasks\" is the one shot.\n\u003e If you rename it, the integration breaks with no workaround.\n\nVoice command: **\"Hey Google, add [items] to my tasks\"** — give any time when prompted. The time is discarded; it's just Google's required prompt. Items are crossed off automatically after Penny processes them.\n\n---\n\n## Services (running on Mac Mini as launchd agents)\n\n| Service | File | What it does |\n|---------|------|-------------|\n| `com.penny.watcher` | `watcher.py` | Polls iCloud Voice Memos every 60s, transcribes via Whisper, classifies and routes |\n| `com.penny.tasks` | `tasks_poller.py` | Polls Google Tasks every 3 min, routes to Apple Reminders |\n| `com.penny.webhook` | `webhook/server.py` | HTTP server on port 5678 for direct uploads and text ingestion |\n| `com.penny.export` | `scripts/export_transcripts.py` | Dumps transcript history to JSON and rsyncs to homelab every 6h |\n\n---\n\n## Configuration\n\nNon-secret settings live in `config.toml`. The key ones:\n\n```toml\n[notifications]\ntelegram_enabled = false   # true to turn Telegram back on, false to silence it\n                           # The code and credentials stay either way\n\n[google_tasks]\nlist_name = \"My Tasks\"     # Do not change — see Google Home constraint above\n\n[apple_reminders]\nlists = [\"Groceries\", \"Errands\", \"Home\", \"Health\", \"Work\", \"Kids\", \"Inbox\"]\ndefault_list = \"Inbox\"\n\n[voice_memos]\nmax_file_size_mb = 50\nwhisper_model = \"mlx-community/whisper-large-v3-turbo\"\npoll_interval_seconds = 60\nstartup_process_limit = 5\n```\n\nAfter changing `config.toml`, rsync it to the Mac and restart the affected service:\n\n```bash\nrsync -av config.toml macmini:/Users/macmini/penny/\nssh macmini \"launchctl kickstart -k gui/\\$(id -u)/com.penny.watcher\"\n```\n\n---\n\n## Operations\n\n```bash\n# Check all services are running (exit code 0 = healthy)\nssh macmini \"launchctl list | grep penny\"\n\n# View logs\nssh macmini \"tail -f ~/.penny/logs/watcher.log\"\nssh macmini \"tail -f ~/.penny/logs/tasks.log\"\nssh macmini \"tail -f ~/.penny/logs/webhook.log\"\n\n# Restart a service\nssh macmini \"launchctl kickstart -k gui/\\$(id -u)/com.penny.SVCNAME\"\n```\n\n### Automated health monitoring\n\nA GitHub Actions workflow (`.github/workflows/health-check.yml`) runs daily at 9am UTC on a\nself-hosted runner (`oci-dev`). It SSHes into macmini and checks:\n- All services registered (count derived automatically from `launchd/*.plist.template` — no hardcoded number)\n- Persistent services (watcher, tasks, webhook) have a running PID\n- Watcher log updated in the last 15 minutes\n- Tasks poller connected to Google Tasks\n- VoiceMemos running (required for CloudKit sync)\n\nEach check is **self-healing**: before failing, the workflow attempts to fix the problem\n(restart the service via `launchctl kickstart`, relaunch VoiceMemos) and re-verifies.\nOnly if recovery fails does it exit non-zero and trigger a GitHub email.\n\nNo emails = everything is healthy.\n\nThe runner is installed as a systemd service on oci-dev:\n```bash\nsudo systemctl status actions.runner.Khamel83-penny.oci-dev.service\n```\n\n## Deploy from repo\n\n```bash\npython3 scripts/trust_check.py\n\nrsync -av --exclude='.git' --exclude='__pycache__' --exclude='venv' \\\n  /path/to/penny/ macmini:/Users/macmini/penny/\n\nssh macmini \"for svc in watcher webhook tasks; do\n  launchctl kickstart -k gui/\\$(id -u)/com.penny.\\${svc}\ndone\"\n```\n\n`scripts/trust_check.py` is the pre-deploy sanity gate — 7 checks: compile, duplicate entrypoints, sqlite3 leak detection, config invariants, launchd template keys, health-check.yml sync (verifies no hardcoded service count), and unit tests.\n\n---\n\n## Requirements\n\n- **Mac** — required for AppleScript access to Reminders and Notes. This repo uses MLX-based Whisper (Apple Silicon only), but any Whisper backend would work on Intel if you swap it out. An always-on Mac Mini is the natural fit.\n- macOS with Homebrew\n- Python 3.11+\n- ffmpeg (`brew install ffmpeg`)\n- Always-on (Mac Mini recommended)\n\n```bash\npip install -r requirements.txt\n```\n\n---\n\n## Setup\n\n### Environment variables (set in launchd plists)\n\n| Variable | Description |\n|----------|-------------|\n| `OPENROUTER_API_KEY` | LLM classification via OpenRouter |\n| `TELEGRAM_BOT_TOKEN` | Telegram bot token (kept even when notifications off) |\n| `TELEGRAM_CHAT_ID` | Your Telegram chat ID |\n| `GOOGLE_CREDENTIALS_FILE` | Path to Google OAuth credentials JSON |\n| `GOOGLE_TOKEN_FILE` | Path to Google OAuth token JSON |\n\nPlist templates with placeholders: `launchd/*.plist.template`\n\n### One-time macOS permissions\n\nTwo approvals required, each permanent after the first:\n\n1. **System Settings → Privacy \u0026 Security → Automation → Python → Reminders** ✓\n2. **System Settings → Privacy \u0026 Security → Automation → Python → Notes** ✓\n\n### Google Tasks setup\n\nOAuth credentials and token live at `~/.penny/` on the Mac.\n\nGoogle Cloud project `neon-feat-488623-u3` (Penny), Tasks API enabled.\n**The app is published (Production mode)** — refresh tokens do not expire.\n\nTo re-authorize (should never be needed again): run `scripts/google_auth.py` on the Mac.\n\n\u003e **Why \"Testing\" mode kills the token every 7 days:** Google limits refresh token lifetime for apps\n\u003e in Testing mode. Publishing the app (APIs \u0026 Services → OAuth consent screen → Publish App) removes\n\u003e this restriction. The app does not need Google verification for personal use.\n\n---\n\n## Runtime state (Mac Mini, never commit these)\n\n| File | Purpose |\n|------|---------|\n| `~/.penny/transcripts.db` | SQLite transcript log — single source of truth for all transcriptions |\n| `~/.penny/transcript_history.json` | Periodic JSON export (backed up to homelab) |\n| `~/.penny/logs/` | Service logs |\n| `~/.penny/last_pk.txt` | Last processed voice memo PK — do not delete |\n| `~/.penny/health.txt` | Watcher health status (written every 5 min) |\n| `~/.penny/health_tasks.txt` | Tasks poller health status |\n| `~/.penny/google_token.json` | Google OAuth token (auto-refreshes) |\n| `~/.penny/google_credentials.json` | Google OAuth app credentials |\n\nLegacy dedup files (`processed.txt`, `processed_webhook.txt`, `synced_tasks.txt`) are still present but superseded by `transcripts.db`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhamel83%2Fpenny","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkhamel83%2Fpenny","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkhamel83%2Fpenny/lists"}