https://github.com/feed-mob/omni-collect
Multi-platform topic intelligence API for AI agents. Install a Skill guide, auto-collect data from Xiaohongshu, GitHub, Twitter and more, then generate structured intel reports locally.
https://github.com/feed-mob/omni-collect
Last synced: 4 months ago
JSON representation
Multi-platform topic intelligence API for AI agents. Install a Skill guide, auto-collect data from Xiaohongshu, GitHub, Twitter and more, then generate structured intel reports locally.
- Host: GitHub
- URL: https://github.com/feed-mob/omni-collect
- Owner: feed-mob
- Created: 2026-03-25T08:07:42.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-03-25T08:30:33.000Z (5 months ago)
- Last Synced: 2026-03-26T12:13:59.760Z (5 months ago)
- Size: 7.81 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# OmniCollect
Multi-platform topic intelligence API for AI agents. Install a Skill guide, auto-collect data from Xiaohongshu, GitHub, Twitter and more, then generate structured intel reports locally.
## How It Works
```
┌──────────────┐ install skill ┌──────────────────┐
│ AI Agent │◄───────────────────────│ Skill (API Guide)│
│ Claude Code │ └──────────────────┘
│ Codex │── HTTP + secret key ──►┌──────────────────┐
│ Continue │◄── structured JSONL ───│ OmniCollect API │
│ OpenClaw │ │ (FastAPI+SQLite) │
│ ... │ └───────┬──────────┘
└──────┬───────┘ │
│ local AI analysis │ Agent-Reach
▼ ▼
┌──────────────┐ ┌─────────────────────┐
│ Intel Report│ │ Xiaohongshu│GitHub │
│ JSONL + MD │── sync report ─────►│ Twitter │ ... │
└──────────────┘ └─────────────────────┘
```
- **Platform (cloud)**: Data collection API, auth, report sync
- **Skill (local)**: A Markdown guide that teaches any AI agent how to call our APIs
- **AI analysis (local)**: Runs on the user's own model — we don't do inference
## Supported Platforms
| Platform | Method | Status |
|----------|--------|--------|
| Xiaohongshu | mcporter + Cookie | Planned |
| GitHub | REST API (HTTP) | Planned |
| Twitter/X | bird + Cookie | Planned |
| More (HN, Reddit, ...) | TBD | Future |
## Tech Stack
- **Python + FastAPI + Pydantic** — lightweight async API, auto-generated Swagger docs
- **SQLite + aiosqlite** — lightweight, zero-ops, async
- **SQLAlchemy 2.0** — async ORM with declarative models
- **Jinja2** — server-side page rendering (planned)
- **Agent-Reach** — native Python integration for multi-platform data fetching
## Auth (Planned — M2)
No human registration needed. The AI agent handles everything:
1. AI generates a key pair locally
2. AI calls `POST /api/v1/auth/register` with the public key
3. Server stores the key (max 3 registrations per IP)
4. Key is persisted in the skill directory — done
> **Note**: Auth endpoints are not yet implemented. Currently only `GET /health` and `GET /` are available. See [ARCHITECTURE.md](./ARCHITECTURE.md) for the full API roadmap.
## Quick Start
```bash
# Install
pip install -e ".[dev]"
# Run
python -m omni_collect.main
# Visit
# http://localhost:8000/docs — Swagger UI
# http://localhost:8000/health — Health check
```
## Project Structure
See [ARCHITECTURE.md](./ARCHITECTURE.md) for the full architecture document, module breakdown, and development schedule.
## License
TBD