{"id":50112511,"url":"https://github.com/core-euler/lead_finder","last_synced_at":"2026-05-23T13:05:49.833Z","repository":{"id":332637604,"uuid":"1131986309","full_name":"core-euler/lead_finder","owner":"core-euler","description":null,"archived":false,"fork":false,"pushed_at":"2026-03-30T08:53:04.000Z","size":51833,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-30T10:29:57.190Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/core-euler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-11T04:44:54.000Z","updated_at":"2026-03-30T08:53:10.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/core-euler/lead_finder","commit_stats":null,"previous_names":["okoloboga/lead_finder","core-euler/lead_finder"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/core-euler/lead_finder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/core-euler%2Flead_finder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/core-euler%2Flead_finder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/core-euler%2Flead_finder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/core-euler%2Flead_finder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/core-euler","download_url":"https://codeload.github.com/core-euler/lead_finder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/core-euler%2Flead_finder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33396642,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T04:15:53.637Z","status":"ssl_error","status_checked_at":"2026-05-23T04:15:53.242Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-23T13:05:47.424Z","updated_at":"2026-05-23T13:05:49.824Z","avatar_url":"https://github.com/core-euler.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LeadCore\n\nLeadCore is a Telegram bot for finding and qualifying B2B leads from Telegram chats.\nIt is designed for agencies/freelancers who sell automation services (Telegram bots, AI assistants, workflow integrations).\n\n## What It Does\n\n- Creates \"programs\" (niche + source chats + settings).\n- Parses chat activity and identifies potential lead candidates.\n- Qualifies candidates with LLM.\n- Stores and shows lead cards in Telegram.\n- Extracts pains from saved/qualified leads.\n- Clusters pains and generates draft posts for your content workflow.\n- Supports scheduled runs per program (daily time), with in-bot on/off toggle.\n\n## Current Architecture\n\n- `aiogram` bot UI and handlers\n- `Telethon` for Telegram parsing/auth session\n- `SQLAlchemy + PostgreSQL` for storage\n- `APScheduler` for scheduling trigger time (enqueue only)\n- `Celery + Redis` for background pipeline execution in worker\n- `CometAPI` (OpenAI-compatible) for LLM calls\n- `Google Custom Search API` for web enrichment/best-practice context\n\nMain entrypoint: `run_bot.py`\n\nExecution model:\n- bot process: receives updates and schedules runs\n- scheduler job: enqueues Celery task only (non-blocking)\n- worker process: executes parsing/qualification pipeline and sends lead cards\n\n## Repository Structure\n\n- `bot/` — bot app, handlers, scheduler, Celery tasks, DB models\n- `modules/` — parsing, qualification, pain clustering, content generation\n- `prompts/` — prompt templates\n- `docs/` — product specs and implementation notes\n- `run_bot.py` — bot launcher\n- `docker-compose.yml` — app + worker + redis + postgres services\n\n## Prerequisites\n\n- Docker + Docker Compose\n- Telegram app credentials (`api_id`, `api_hash`, phone)\n- Telegram Bot token\n- CometAPI key\n- Google Custom Search credentials\n\n## Environment Variables\n\nCreate `.env` from `.env.example` and fill required values.\n\nCore:\n\n- `COMET_API_KEY`\n- `COMET_API_BASE_URL` (default: `https://api.cometapi.com/v1`)\n- `COMET_API_MODEL` (general model for qualification, etc.)\n- `COMET_API_POST_MODEL` (dedicated model for post generation)\n- `GOOGLE_API_KEY`\n- `GOOGLE_CSE_ID`\n- `TELEGRAM_API_ID`\n- `TELEGRAM_API_HASH`\n- `TELEGRAM_PHONE`\n- `TELEGRAM_BOT_TOKEN`\n\nDatabase (used by Docker Compose service `db`):\n\n- `POSTGRES_HOST`\n- `POSTGRES_PORT`\n- `POSTGRES_USER`\n- `POSTGRES_PASSWORD`\n- `POSTGRES_DB`\n\nUseful runtime settings:\n\n- `MESSAGES_LIMIT`\n- `MESSAGE_MAX_AGE_DAYS`\n- `SAFETY_MODE` (`fast`, `normal`, `careful`)\n- `MAX_CONCURRENT_PIPELINES` (in-worker parallel pipelines, keep `1` for stability)\n- `CELERY_BROKER_URL`\n- `CELERY_RESULT_BACKEND`\n- `CELERY_WORKER_CONCURRENCY`\n\nWorker mode (important):\n- Celery worker is configured with `--pool=solo` for async SQLAlchemy/asyncpg stability.\n\n## Quick Start (Docker)\n\n1. Create and fill `.env`:\n\n```bash\ncp .env.example .env\n```\n\n2. Build and start services:\n\n```bash\ndocker compose up -d --build\n```\n\n3. Watch logs:\n\n```bash\ndocker compose logs -f app worker\n```\n\n## Telegram Session Authentication (Telethon)\n\nThe bot needs a valid Telethon user session to parse chats.\n\nOption A (recommended in-bot flow):\n\n- Start a program run.\n- If auth is required, bot asks for code/password.\n- Complete steps directly in Telegram.\n\nOption B (manual one-time local session generation):\n\n```bash\npython generate_session.py\n```\n\nThis creates/updates Telethon session file, then Dockerized app can reuse it.\n\n## Typical In-Bot Workflow\n\n1. Open bot and go to `My Programs`.\n2. Create a program (name, niche, source chats).\n3. Run manually (`Run`) or keep scheduled auto-collection enabled.\n4. Review lead cards.\n5. Open `Pains \u0026 Content` to inspect clusters and generate drafts.\n\n## Notes on Content Generation\n\n- Post generation uses `COMET_API_POST_MODEL`.\n- Before generation, system fetches web best practices for AI/business integration and injects them into prompt context.\n- Current UX uses one unified post mode (no scenario/insight/trend selection).\n\n## Troubleshooting\n\n- `TELEGRAM_BOT_TOKEN not found`: check `.env`.\n- No pains/clusters shown: ensure program has qualified leads and run completed successfully.\n- Callback timeout errors (`query is too old`): update to latest code (callback ack is handled early).\n- DB errors about schema/constraints: restart app after pulling latest updates so startup migrations run.\n- `(sqlalchemy...InterfaceError) cannot perform operation: another operation is in progress` in worker:\n  - Cause: asyncpg connection state reused across process boundaries or prefork pool issues.\n  - Fix in current code: process-bound SQLAlchemy engine rebind in Celery worker + `--pool=solo`.\n  - Action: rebuild/restart `worker` (`docker compose up -d --build worker`).\n\n## Development\n\nInstall dependencies locally if needed:\n\n```bash\npip install -r requirements.txt\n```\n\nRun bot locally:\n\n```bash\npython run_bot.py\n```\n\nRun tests:\n\n```bash\npip install -r requirements.txt -r requirements-dev.txt\nPYTHONPATH=. pytest\n\n# Unit tests used in CI:\nPYTHONPATH=. pytest -m unit --cov=bot.services.subscription --cov-report=term-missing\n```\n\nFor production-like usage, Docker Compose is recommended.\n\n## CI/CD Auto Deploy (GitHub Actions -\u003e Server)\n\nThis repository includes an auto-deploy workflow:\n- File: `.github/workflows/deploy.yml`\n- Trigger: every push to `main` (and manual `workflow_dispatch`)\n- Action: SSH to server -\u003e `git pull` -\u003e `docker compose up -d --build`\n\nRequired GitHub repository secrets:\n- `DEPLOY_HOST` (server IP/domain)\n- `DEPLOY_USER` (SSH user)\n- `DEPLOY_SSH_KEY` (private key in PEM/OpenSSH format)\n- `DEPLOY_PATH` (absolute path to project on server, e.g. `/opt/leadsense`)\n\nOptional secrets:\n- `DEPLOY_PORT` (default `22`)\n- `DEPLOY_BRANCH` (default `main`)\n\nServer prerequisites:\n- Repository cloned at `DEPLOY_PATH`\n- Docker + Docker Compose installed\n- SSH user has permissions to run `docker compose`\n- Server can run `git pull` for this repository (deploy key or machine user configured)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcore-euler%2Flead_finder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcore-euler%2Flead_finder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcore-euler%2Flead_finder/lists"}