{"id":50648867,"url":"https://github.com/Yurii201811/tasksignal","last_synced_at":"2026-06-24T12:00:31.558Z","repository":{"id":361691028,"uuid":"1254654751","full_name":"Yurii201811/tasksignal","owner":"Yurii201811","description":"AI-assisted problem discovery engine that turns public developer complaints into evidence-backed software opportunities and Codex-ready MVP prompts.","archived":false,"fork":false,"pushed_at":"2026-06-11T00:10:20.000Z","size":1569,"stargazers_count":2,"open_issues_count":0,"forks_count":4,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-11T02:09:52.205Z","etag":null,"topics":["ai","codex","developer-tools","fastapi","local-first","machine-learning","nextjs","pgvector","portfolio-project","postgresql","product-discovery","python"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Yurii201811.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-30T21:01:11.000Z","updated_at":"2026-06-11T00:10:23.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Yurii201811/tasksignal","commit_stats":null,"previous_names":["yurii201811/tasksignal"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/Yurii201811/tasksignal","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yurii201811%2Ftasksignal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yurii201811%2Ftasksignal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yurii201811%2Ftasksignal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yurii201811%2Ftasksignal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Yurii201811","download_url":"https://codeload.github.com/Yurii201811/tasksignal/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yurii201811%2Ftasksignal/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34731256,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-24T02:00:07.484Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","codex","developer-tools","fastapi","local-first","machine-learning","nextjs","pgvector","portfolio-project","postgresql","product-discovery","python"],"created_at":"2026-06-07T16:00:22.266Z","updated_at":"2026-06-24T12:00:31.552Z","avatar_url":"https://github.com/Yurii201811.png","language":"Python","funding_links":[],"categories":["What's Inside"],"sub_categories":[],"readme":"# TaskSignal - AI Problem Discovery Engine\n\nFrom Reddit/forum complaints → evidence-backed project ideas → build-ready Codex prompts.\n\nTaskSignal is an AI-assisted engine that mines public developer and community discussions, detects concrete repetitive tasks people complain about, clusters similar pain signals, scores software opportunities, and generates Codex-ready MVP prompts.\n\n![TaskSignal dashboard after processing demo data](docs/images/dashboard-browser-verified.png)\n\n## Project Status\n\nTaskSignal is a portfolio-ready MVP built by Yurii Bakurov. It is designed for one local operator on their own machine: fixture data works out of the box, a local workspace profile stores that user's research defaults, and repeatable API-backed workflows can be enabled for supported public sources when credentials are provided.\n\nCurrent public posture: TaskSignal is an early public application repository, not a widely adopted package. Its strongest evidence today is reproducibility, release hygiene, CI, security/privacy documentation, contributor issues, and a browser-verified demo flow. See the [demo evidence snapshot](docs/demo-evidence.md) and [Codex for OSS evidence](docs/codex-for-oss-application.md) for the current review package.\n\nUseful starting points:\n\n- [Product context](PRODUCT.md)\n- [Architecture](docs/architecture.md)\n- [API reference](docs/api.md)\n- [Demo evidence snapshot](docs/demo-evidence.md)\n- [Deployment notes](docs/deployment.md)\n- [Data ethics](docs/data-ethics.md)\n- [Source limits and terms](docs/source-limits.md)\n- [Model card](docs/model-card.md)\n- [Roadmap](docs/roadmap.md)\n- [Threat model](docs/threat-model.md)\n- [Maintainer automation plan](docs/maintainer-automation.md)\n- [Codex for OSS application evidence](docs/codex-for-oss-application.md)\n- [Changelog](CHANGELOG.md)\n- [Contributing guide](CONTRIBUTING.md)\n- [Security policy](SECURITY.md)\n\n## Why This Exists\n\nMost idea lists are generic. TaskSignal is a task-replacement radar: it looks for specific repeated workflows people hate doing, such as exporting Stripe data into a spreadsheet every Friday and turning it into a client report.\n\n## Who Should Use This\n\nTaskSignal is for maintainers, builders, indie hackers, developer-tool teams, and researchers who want a local-first way to review public pain signals before deciding what to build. It is not for scraping private communities, profiling individuals, spam, outreach automation, or replacing human product judgment.\n\n## What It Does\n\n- Loads demo fixture data with no API keys.\n- Stores one local workspace profile with owner/focus/default research settings.\n- Saves repeatable research projects with source, query, limit, labels, cadence, last run, next run, and run count.\n- Reports integration readiness without exposing secret values.\n- Records scan outcomes with found/saved items, detected signals, generated opportunities, and guidance when live data produces no ranked opportunity.\n- Normalizes Reddit, Hacker News, GitHub Issues, Stack Exchange, and fixture-style records.\n- Stores author hashes instead of raw usernames by default.\n- Detects complaints, manual workflows, tool requests, workarounds, buying intent, and confusion.\n- Generates local embeddings with `sentence-transformers/all-MiniLM-L6-v2` when available.\n- Falls back to deterministic local vectors when the model is unavailable.\n- Clusters signals with a local thematic fallback by default, with optional DBSCAN when `TASKSIGNAL_USE_SKLEARN_CLUSTERING=1`.\n- Scores opportunities using frequency, recency, pain, concreteness, buying intent, feasibility, and competition penalty.\n- Generates opportunity cards, full Codex-ready build prompts, and richer Codex task packs.\n- Optionally enhances generated prompts through OpenAI API or local Ollama when explicitly configured.\n\n## Architecture\n\n```mermaid\nflowchart TD\n  A[Public sources and fixtures] --\u003e B[Ingestion connectors]\n  B --\u003e C[Normalizer and deduplicator]\n  C --\u003e D[(PostgreSQL + pgvector)]\n  D --\u003e E[Pain and task detector]\n  E --\u003e F[Embedding service]\n  F --\u003e G[Thematic fallback clustering / optional DBSCAN]\n  G --\u003e H[Opportunity scoring]\n  H --\u003e I[Prompt generator]\n  I --\u003e J[FastAPI API]\n  J --\u003e K[Next.js dashboard]\n```\n\n## Tech Stack\n\nFrontend: Next.js, TypeScript, Tailwind CSS, TanStack Query, Recharts, React Markdown, Zod-ready types.\n\nBackend: FastAPI, Pydantic v2, SQLAlchemy 2, Alembic, PostgreSQL, pgvector, pytest, ruff, scikit-learn.\n\nML/NLP: sentence-transformers with local-only load when the model cache exists, deterministic fallback vectors, optional DBSCAN clustering, rule-based signal detector.\n\nInfra: Docker Compose, Makefile, GitHub Actions CI, scheduled ingestion template.\n\n## Quickstart\n\n```bash\ncp .env.example .env\nmake doctor\nmake up\n```\n\nOpen the frontend at [http://localhost:3000](http://localhost:3000), go to Projects, save a research workflow, then run it. For a first proof path, go to Dashboard and click **Process demo data**. To use live public data, choose a source, query, and limit in **Live source**, then click **Run scan**.\n\nIf setup fails or a fresh checkout looks incomplete, run:\n\n```bash\nmake doctor\n```\n\n`make doctor` checks the required files, local `.env`, Python, Node 20+, npm,\nrepo-local Python dev tools, fixture files, and whether generated files are\naccidentally tracked. Docker is only required for the Compose quickstart.\n\nAPI health check:\n\n```bash\ncurl http://localhost:8000/health\n```\n\n## Local Development\n\nRun the API and frontend separately:\n\n```bash\ncd apps/api\n../../.venv/bin/uvicorn app.main:app --reload\n```\n\n```bash\ncd apps/web\nnpm run dev\n```\n\nRun checks before publishing changes:\n\n```bash\nmake test\nmake lint\nmake verify\n```\n\nThe Makefile prefers repo-local Python tools in `.venv/bin`. On Apple Silicon\nmacOS it also prepends Homebrew Node 20 from `/opt/homebrew/opt/node@20/bin`\nwhen available, matching the runtime required by the Next.js web app.\n\nRun the release-readiness gate before tagging a release:\n\n```bash\nmake release-check\n```\n\nRun the first-run smoke check to verify the credential-free fixture path against\na temporary database, including dashboard route wiring and task-pack export:\n\n```bash\nmake smoke\n```\n\nTo also boot the Next.js dev server and request `/dashboard`, run:\n\n```bash\napps/api/.venv/bin/python -u scripts/first_run_smoke.py --with-web-server\n```\n\nUse the local CLI for headless operation:\n\n```bash\nscripts/tasksignal_cli.py readiness\nscripts/tasksignal_cli.py configure-workspace --owner \"Local Builder\" --goal \"Find developer-tool opportunities\" --source hackernews --query ask --cadence daily\nscripts/tasksignal_cli.py create-project --name \"Track CI/CD pain\" --source hackernews --query ask --cadence daily\nscripts/tasksignal_cli.py run-due\nscripts/tasksignal_cli.py task-pack \u003copportunity-id\u003e --output task-pack.md\n```\n\nTaskSignal does not require multi-user accounts for this local mode. The local\nworkspace profile is a singleton in the app database and is meant for the person\nrunning the app on that machine.\n\n## Distribution\n\nTaskSignal is currently an application repository, not a published Python or npm library. Use the source checkout or Docker Compose workflow above. Reusable packages may be split out later if a stable library boundary emerges.\n\n## Reviewer Quick Check\n\nFor a quick public review, inspect:\n\n- [Latest release](https://github.com/Yurii201811/tasksignal/releases/tag/v0.1.3)\n- [Open contributor issues](https://github.com/Yurii201811/tasksignal/issues)\n- [Release-readiness workflow](https://github.com/Yurii201811/tasksignal/actions/workflows/release-check.yml)\n- [Demo evidence snapshot](docs/demo-evidence.md)\n- [Threat model](docs/threat-model.md)\n\n## Repository Layout\n\n```text\napps/api      FastAPI backend, ML pipeline, database models, tests\napps/web      Next.js dashboard, opportunity views, prompt export UI\ndata          Demo fixtures for local-first processing\ndocs          Architecture, API, deployment, ethics, and model notes\nnotebooks     Classifier training and evaluation workbooks\n```\n\n## Fixture Demo Mode\n\nFixture mode is the default. It loads records from `data/fixtures`, processes them end to end, and should generate at least five opportunity cards:\n\n- AI-generated code audit tool\n- Early-stage SaaS lead/community signal radar\n- Simple onboarding drop-off analyzer\n- GitHub Actions workflow debugging assistant\n- Spreadsheet-to-report automation helper\n\n## API Connector Setup\n\nLive scans use official APIs and keep the same local-first scoring/generation pipeline as fixture mode. The unauthenticated `POST /api/scans` endpoint is restricted to public API-safe sources (`fixture` and `hackernews`) so network callers cannot spend server-side credentials or retrieve data visible to server-side tokens.\n\nTrusted operators can still configure the internal connector pipeline with source credentials when running controlled jobs outside the public endpoint:\n\n- `REDDIT_CLIENT_ID`, `REDDIT_CLIENT_SECRET`, `REDDIT_USER_AGENT`\n- `GITHUB_TOKEN`\n- `STACK_EXCHANGE_KEY`\n\nHacker News works without credentials through the public Firebase API. GitHub and Stack Exchange can run without keys at lower rate limits. Reddit requires OAuth credentials. No paid LLM key is required. `LLM_PROVIDER=none` is the default.\n\nConnector credentials belong in environment variables, not source registry\nrecords. Source registry write endpoints require `OPERATOR_SCAN_TOKEN`, reject\nsecret-like `config_json` keys, and read endpoints return redacted config so\nlocal rows cannot expose token values.\n\n`PUBLIC_SCAN_SOURCES` can narrow the public endpoint further, for example to `hackernews` only. Credentialed sources such as GitHub, Reddit, and Stack Exchange stay reserved for trusted internal scan jobs.\n\nBrowser-triggered runs of credentialed sources are available through saved\nresearch projects only when `OPERATOR_SCAN_TOKEN` is configured on the API and\nthe same token is entered locally in the Projects or Integrations page. This\nkeeps hosted deployments from silently spending server-side credentials while\nstill letting trusted local operators connect APIs.\n\nSaved projects support manual, hourly, daily, weekly, and custom-hour cadences.\nTaskSignal does not hide a scheduler inside the web process. Run due projects\nfrom the Projects page, `scripts/tasksignal_cli.py run-due`, cron, GitHub\nActions, or another explicit worker.\n\nOptional prompt enhancement uses `LLM_PROVIDER=openai` plus `OPENAI_API_KEY`, or\n`LLM_PROVIDER=ollama` plus a local Ollama server. Browser-triggered enhancement\nrequires `OPERATOR_SCAN_TOKEN` on the API and the matching\n`X-Operator-Scan-Token` request header so network callers cannot spend\nserver-side model credentials. ChatGPT/Codex subscriptions do not provide\nbackend API credentials; TaskSignal supports subscription users by exporting\ntask packs they can open in their own signed-in Codex app, CLI, IDE extension,\nor Codex web session.\n\nDestructive fixture resets require `DEMO_RESET_TOKEN` and the matching `X-Demo-Reset-Token` request header. The normal dashboard demo-processing action is non-destructive by default.\n\n## Codex And Agent Handoff\n\nEach opportunity can export:\n\n- A generated Codex prompt.\n- An evidence bundle.\n- A Codex task pack with objective, suggested MVP, score, evidence, acceptance\n  criteria, privacy constraints, and recommended Codex flow.\n\nTask packs are designed for users who want to use their own signed-in Codex app,\nCLI, IDE extension, or Codex web session. They do not spend ChatGPT/Codex plan\nusage from the TaskSignal backend. A repo-local skill package is available at\n`skills/tasksignal-opportunity-builder` for agents that can load Codex-style\nskills.\n\n## ML/NLP Approach\n\nThe MVP uses transparent rules first. It scores pain phrases, repetition phrases, tool requests, buying intent, and task concreteness hints. Embeddings use `sentence-transformers/all-MiniLM-L6-v2` only when locally available; otherwise deterministic vectors keep the demo working.\n\n## Scoring Formula\n\n```text\nopportunity_score =\n  0.25 * frequency_score\n+ 0.20 * recency_score\n+ 0.20 * pain_intensity_score\n+ 0.15 * task_concreteness_score\n+ 0.10 * buying_intent_score\n+ 0.10 * feasibility_score\n- 0.10 * competition_penalty\n```\n\n## Privacy And Ethics\n\nTaskSignal is designed for public-data research, product discovery, and learning. It does not store raw usernames by default, preserves source URLs for attribution, respects API boundaries, and should not be used for spam or harassment workflows.\n\nBefore enabling live connectors, review [Data ethics](docs/data-ethics.md), configure API credentials through environment variables or GitHub repository secrets, and avoid committing `.env` files or exported datasets.\n\n## Example Generated Opportunity\n\n**Developers need clearer GitHub Actions failure diagnosis**\n\nProblem: teams spend repetitive time reading noisy CI logs, searching YAML errors, and guessing root causes.\n\nSuggested MVP: a CI log summarizer and workflow linter that identifies likely YAML mistakes, dependency failures, and next fixes.\n\n## Example Generated Codex Prompt\n\n```markdown\n# Build Developers need clearer GitHub Actions failure diagnosis\n\nYou are a senior full-stack engineer. Build a working MVP...\n```\n\n## Portfolio Notes\n\nThis repository demonstrates full-stack engineering, API design, Python backend development, TypeScript frontend development, PostgreSQL/pgvector modeling, ML/NLP pipelines, clustering, product scoring, privacy-conscious design, Docker, CI/CD, tests, and technical writing.\n\n## Roadmap\n\n- Publish and maintain tagged releases with changelog entries.\n- Expand contributor-friendly fixtures, docs, and public issues.\n- Add richer source scheduling and rate-limit state after privacy review.\n- Add pgvector ANN search in production mode.\n- Add reviewer workflow for human labels.\n\nSee [Roadmap](docs/roadmap.md) for maintainer tasks, security milestones, and longer-term ideas.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYurii201811%2Ftasksignal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FYurii201811%2Ftasksignal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FYurii201811%2Ftasksignal/lists"}