{"id":51951703,"url":"https://github.com/dubsopenhub/oss-radar","last_synced_at":"2026-07-29T06:01:28.303Z","repository":{"id":340738171,"uuid":"1167374576","full_name":"DUBSOpenHub/oss-radar","owner":"DUBSOpenHub","description":"Autonomous Python agent that monitors developer forums for open source maintainer pain points, ranks by signal strength, and delivers daily intelligence reports via email","archived":false,"fork":false,"pushed_at":"2026-04-10T17:22:56.000Z","size":288,"stargazers_count":2,"open_issues_count":122,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-07T22:21:29.461Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DUBSOpenHub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":"audit-report-task-003.md","citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-02-26T08:20:41.000Z","updated_at":"2026-05-21T19:51:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/DUBSOpenHub/oss-radar","commit_stats":null,"previous_names":["dubsopenhub/oss-radar"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/DUBSOpenHub/oss-radar","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Foss-radar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Foss-radar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Foss-radar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Foss-radar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DUBSOpenHub","download_url":"https://codeload.github.com/DUBSOpenHub/oss-radar/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Foss-radar/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36020486,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-29T02:00:04.910Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-29T06:01:27.061Z","updated_at":"2026-07-29T06:01:28.281Z","avatar_url":"https://github.com/DUBSOpenHub.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OSS Opportunities Radar\n\nAn autonomous Python agent that monitors four developer platforms, extracts genuine open source maintainer pain points, ranks them by signal strength, and delivers curated intelligence via email — requiring zero interaction after initial setup.\n\nSet it up once, point it at your inbox, and forget it exists. Every morning a dark-mode email lands with the 5 highest-signal OSS pain points. Fridays bring the weekly trends. No dashboard. No buttons. Just signal.\n\n---\n\n## Architecture\n\n```\nlaunchd / GitHub Actions (survives reboots, zero intervention)\n  └── radar schedule\n       ├── Daily job (2x) ── scrape → filter → rank → [LLM summarize] → store → email\n       ├── Weekly job (Fri) ── aggregate → trend analysis → email digest\n       ├── LLM Backend ───── GitHub Models API → Amplifier CLI fallback (opt-in)\n       └── State Store ────── SQLite catalog (WAL mode, 3 tables)\n\n┌──────────────────────────────────────────────────────────────┐\n│  Entry Points: CLI (Typer) │ APScheduler │ GitHub Actions    │\n└──────────────────────────┬───────────────────────────────────┘\n                           │\n               ┌───────────▼───────────┐\n               │    pipeline.py        │\n               │  PipelineOrchestrator │\n               └───────────┬───────────┘\n                           │\n         ┌─────────────────▼──────────────────┐\n         │           Scraping Layer            │\n         │  HN │ Dev.to │ Lobsters │ Reddit   │\n         └─────────────────┬──────────────────┘\n                           │ List[RawPost]\n         ┌─────────────────▼──────────────────┐\n         │  Ranking Layer                      │\n         │  1. KeywordFilter (100+ patterns)   │\n         │  2. MaintainerContextFilter         │\n         │  3. SentimentFilter (VADER+TextBlob)│\n         │  4. SignalScorer (batch-normalised) │\n         └─────────────────┬──────────────────┘\n                           │ List[ScoredPost]\n         ┌─────────────────▼──────────────────┐\n         │  LLM Summarizer (opt-in)           │\n         │  GitHub Models → Amplifier fallback │\n         └─────────────────┬──────────────────┘\n                           │\n         ┌─────────────────▼──────────────────┐\n         │  BackfillManager (Fallback Ladder)  │\n         │  live → archive-7d → archive-30d   │\n         │  → partial                         │\n         └─────────────────┬──────────────────┘\n                           │\n         ┌─────────────────▼──────────────────┐\n         │  Storage: SQLite WAL (3 tables)     │\n         └─────────────────┬──────────────────┘\n                           │\n         ┌─────────────────▼──────────────────┐\n         │  Email: Jinja2 → SMTP (STARTTLS)   │\n         └────────────────────────────────────┘\n```\n\n---\n\n## Quick Start\n\n### Path A: Local Daemon (macOS, one command)\n\n```bash\ncd ~/dev/oss-radar\nbash install.sh\n# Edit .env with your SMTP credentials\n# Done. Daemon scrapes hourly, emails daily + weekly digest. Check logs: tail -f logs/radar-stdout.log\n```\n\n### Path B: GitHub Actions (cloud, zero infra)\n\n1. Push this repo to GitHub\n2. Add secrets: `RADAR_SMTP_HOST`, `RADAR_SMTP_PORT`, `RADAR_SMTP_USER`, `RADAR_SMTP_PASSWORD`, `RADAR_EMAIL_FROM`, `RADAR_EMAIL_TO`\n3. Optionally: `RADAR_REDDIT_CLIENT_ID`, `RADAR_REDDIT_CLIENT_SECRET`, `RADAR_LLM_ENABLED=true`\n4. Hourly scrapes run every hour; daily email report at **6 AM PST** (14:00 UTC)\n5. Weekly digest runs **Friday 12 PM PST** (20:00 UTC)\n\n### Path C: Manual\n\n```bash\npip install -r requirements.txt\ncp .env.example .env \u0026\u0026 nano .env\nradar daily        # one-shot daily report\nradar schedule     # start daemon\n```\n\n---\n\n## CLI Reference\n\n| Command | Description | Key Flags |\n|---|---|---|\n| `radar synth` | Full pipeline with synthetic data (no API keys) | `--count N`, `--seed N`, `--dry-run`, `--no-email` |\n| `radar scrape` | Scrape all platforms, store raw posts | `--db-path` |\n| `radar daily` | Full daily pipeline: scrape→filter→rank→email | `--force`, `--dry-run`, `--no-email` |\n| `radar weekly` | Weekly digest from past 7 days of reports | `--dry-run`, `--no-email` |\n| `radar validate` | Check DB, HTTP, SMTP connectivity | `--db-path` |\n| `radar schedule` | Start APScheduler daemon | |\n| `radar report` | Display a specific report | `--id ID` |\n| `radar stats` | Show catalog statistics | `--db-path` |\n\n**Global flags** (all commands): `--db-path PATH`, `--log-level LEVEL`\n\n**Exit codes**:\n- `0` — full success\n- `1` — partial (\u003c 5 posts, or SMTP retry exhausted)\n- `2` — fatal failure (all scrapers failed, DB unwritable, unhandled exception)\n\n---\n\n## LLM Summarization (Opt-in)\n\nEnable AI-powered one-sentence summaries for each pain point in your email reports.\n\n```bash\n# In .env\nRADAR_LLM_ENABLED=true\nRADAR_LLM_MODEL=claude-sonnet-4.6  # or any GitHub Models / Amplifier model\n```\n\n**How it works:**\n- Primary: GitHub Models API via `gh api /models/chat/completions`\n- Fallback: Amplifier CLI via `uv run amplifier`\n- Graceful degradation: if both fail, falls back to 120-char body excerpt\n- Zero new pip dependencies (pure stdlib async subprocess)\n\n**Requirements:** Either `gh` CLI authenticated or Amplifier installed at `~/amplifier`.\n\n---\n\n## Synthetic Mode (No API Keys)\n\nRun the full pipeline with realistic fake data — perfect for development, demos, and CI:\n\n```bash\nradar synth --no-email           # 50 synthetic posts, full pipeline\nradar synth --seed 42 --count 100 --no-email  # reproducible\nradar synth --dry-run            # no DB writes, no email\n```\n\nSynthetic posts use real pain keywords from the ranking module, so they exercise the full filter → score → backfill → store pipeline. ~60% pass all 3 filter layers, ~40% are intentional noise.\n\n---\n\n## Scoring Formula\n\n```\ninfluence_norm  = log10(author_karma + 1) / log10(max_karma + 1)   [0, 1]\nengagement_norm = log10(score + comments + 1) / log10(max_eng + 1) [0, 1]\n\nbase_score     = 0.4 × influence_norm + 0.6 × engagement_norm\npain_factor    = 1.0 (1 keyword match) | 1.2 (2–3 matches) | 1.5 (4+ matches)\nmaintainer_boost = 1.0 (1 context signal) | 1.25 (2+ signals)\nsignal_score   = base_score × pain_factor × (1.0 + |sentiment_score|) × maintainer_boost\n```\n\n---\n\n## Signal Filtering\n\nPosts must pass **all three** layers:\n\n1. **Keyword Gate** — title+body must match ≥1 of 100+ compiled regex patterns across 15 `PainCategory` enums\n2. **Maintainer Context** — post must contain ≥1 signal: \"I maintain\", \"my project\", \"we merged\", \"released v…\", etc.\n3. **Sentiment** — `(VADER × 0.6) + (TextBlob × 0.4) \u003c −0.05` (only pain posts pass)\n\n---\n\n## Configuration Reference\n\nAll environment variables are prefixed `RADAR_`. See `.env.example` for the complete list.\n\n| Variable | Default | Description |\n|---|---|---|\n| `RADAR_DB_PATH` | `~/.radar/catalog.db` | SQLite catalog path |\n| `RADAR_EMAIL_ENABLED` | `false` | Enable email dispatch |\n| `RADAR_SMTP_HOST` | `localhost` | SMTP server hostname |\n| `RADAR_SMTP_PORT` | `587` | SMTP server port |\n| `RADAR_SMTP_USER` | `\"\"` | SMTP username |\n| `RADAR_SMTP_PASSWORD` | `\"\"` | SMTP password |\n| `RADAR_EMAIL_TO` | `\"\"` | Comma-separated recipients |\n| `RADAR_REDDIT_ENABLED` | `false` | Enable Reddit scraper |\n| `RADAR_REDDIT_CLIENT_ID` | `\"\"` | Reddit API client ID |\n| `RADAR_REDDIT_CLIENT_SECRET` | `\"\"` | Reddit API client secret |\n| `RADAR_INFLUENCE_WEIGHT` | `0.4` | Influence signal weight (must sum to 1.0 with engagement) |\n| `RADAR_ENGAGEMENT_WEIGHT` | `0.6` | Engagement signal weight |\n| `RADAR_SCRAPE_CRON` | `0 * * * *` | Hourly scrape cron (UTC) — collect only, no email |\n| `RADAR_DAILY_CRON` | `0 14 * * *` | Daily email report cron (UTC) — 6 AM PST |\n| `RADAR_WEEKLY_CRON` | `0 20 * * 5` | Weekly digest cron (UTC) |\n| `RADAR_DUPLICATE_RUN_HOURS` | `10` | Duplicate-run guard window for daily email |\n| `RADAR_LLM_ENABLED` | `false` | Enable LLM pain-point summaries |\n| `RADAR_LLM_MODEL` | `claude-sonnet-4.6` | Model for LLM summarization |\n| `RADAR_LOG_LEVEL` | `INFO` | Logging verbosity |\n| `RADAR_LOG_JSON` | `false` | JSON log format for CI |\n\n---\n\n## Deployment\n\n### launchd (macOS)\n\nThe installer creates a persistent `launchd` agent that:\n- Runs `radar schedule` as a KeepAlive daemon (survives reboots)\n- Executes hourly scrapes + daily email (6 AM PST) + Friday weekly digest\n- Logs to `logs/` in the project directory\n\n```bash\n# Install\nbash install.sh\n\n# View logs\ntail -f logs/radar-stdout.log\n\n# Manual run\n.venv/bin/radar daily --force\n\n# Stop agent\nlaunchctl bootout gui/$(id -u)/com.dubsopenhub.oss-radar\n\n# Restart agent\nlaunchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.dubsopenhub.oss-radar.plist\n```\n\n### GitHub Actions (cloud)\n\nTwo workflow files in `.github/workflows/`:\n- **`daily.yml`** — Hourly cron: `0 * * * *` (scrape every hour, email at 14:00 UTC)\n- **`weekly.yml`** — Friday cron: `0 20 * * 5`\n\nBoth persist the SQLite catalog as a GitHub Actions artifact (90-day retention) and auto-create Issues on failure.\n\n---\n\n## Development\n\n```bash\n# Run tests (185 tests, no API keys needed)\npytest -x\n\n# Run with coverage\npytest --cov=radar --cov-report=html\n\n# Lint\nruff check .\n```\n\n---\n\n🐙 Created with 💜 by [@DUBSOpenHub](https://github.com/DUBSOpenHub) with the [GitHub Copilot CLI](https://docs.github.com/copilot/concepts/agents/about-copilot-cli).\n\nLet's build! 🚀✨\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdubsopenhub%2Foss-radar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdubsopenhub%2Foss-radar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdubsopenhub%2Foss-radar/lists"}