{"id":51391895,"url":"https://github.com/ishan-parihar/igs-mcp","last_synced_at":"2026-07-04T00:06:24.171Z","repository":{"id":348386076,"uuid":"1197816358","full_name":"ishan-parihar/igs-mcp","owner":"ishan-parihar","description":"Intelligence Gathering System — MCP server for real-time news monitoring, Reddit search, and academic research across 223 RSS sources","archived":false,"fork":false,"pushed_at":"2026-05-27T12:40:28.000Z","size":1730,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-27T13:25:03.867Z","etag":null,"topics":["arxiv","intelligence","mcp","news-aggregator","nlp","reddit","rss","typescript"],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ishan-parihar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"AUDIT_REPORT.md","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-31T23:07:46.000Z","updated_at":"2026-05-27T12:40:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ishan-parihar/igs-mcp","commit_stats":null,"previous_names":["ishan-parihar/igs-mcp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ishan-parihar/igs-mcp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ishan-parihar%2Figs-mcp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ishan-parihar%2Figs-mcp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ishan-parihar%2Figs-mcp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ishan-parihar%2Figs-mcp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ishan-parihar","download_url":"https://codeload.github.com/ishan-parihar/igs-mcp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ishan-parihar%2Figs-mcp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35105543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arxiv","intelligence","mcp","news-aggregator","nlp","reddit","rss","typescript"],"created_at":"2026-07-04T00:06:23.447Z","updated_at":"2026-07-04T00:06:24.154Z","avatar_url":"https://github.com/ishan-parihar.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IGS MCP Server\n\n**Intelligence Gathering System** — Local-only MCP server for real-time news monitoring and academic research.\n\n[![npm](https://img.shields.io/npm/v/igs-mcp-server)](https://www.npmjs.com/package/igs-mcp-server)\n[![License](https://img.shields.io/github/license/ishan-parihar/igs-mcp)](https://github.com/ishan-parihar/igs-mcp/blob/main/LICENSE)\n\n---\n\n## The Problem\nFragmented intelligence is an operational bottleneck for AI agents. High-authority news and research are scattered across hundreds of disparate RSS feeds, HTML sites, and niche APIs (arXiv, Semantic Scholar, Reddit). Manually aggregating this into a coherent \"intelligence picture\" is nearly impossible for an agent in a single turn, as it requires complex filtering, deduplication, and the ability to cross-reference entities across completely different data formats and domains.\n\n## Engineering Highlights\n\n### Multi-Tiered Intelligence Aggregation\nI implemented a system that monitors 223+ curated sources across 45 countries and 23 domains. By utilizing a pool-based organization system, the agent can pivot from \"Global Breaking News\" to \"India Regional\" in a single tool call. A specialized parser layer normalizes varied formats (RSS, Atom, JSON, HTML) into a consistent schema, allowing the agent to treat the entire global news web as a single, queryable database.\n\n### Local-First NLP Enrichment\nTo avoid the latency and cost of external LLM calls for basic data cleaning, I built a local NLP enrichment layer. This system extracts entities, topics, and sentiment directly on the host machine using optimized pattern matching and keyword extraction. This allows the agent to perform high-speed pre-filtering and \"triage\" of intelligence sources before requesting a deep-dive analysis.\n\n### Hybrid Research Pipeline\nThe server integrates disparate research modalities—combining real-time news with academic archives (arXiv/Semantic Scholar) and community discourse (Reddit). This enables a \"triangulation\" workflow: the agent can identify a breaking trend in the news, verify the underlying theory via a Semantic Scholar paper, and gauge real-world sentiment via Reddit—all within a single reasoning loop.\n\n---\n\n## 🚀 Quick Start\n\n### 1. Install \u0026 Configure (Recommended)\n\n```bash\ngit clone https://github.com/ishan-parihar/igs-mcp.git\ncd igs-mcp\n./setup.sh\n```\n\nThe setup script checks prerequisites (Node.js v20+), installs dependencies, builds, bootstraps config files, and prints your Claude Desktop config at the end — ready to copy-paste.\n\n### Manual Install\n\n```bash\ngit clone https://github.com/ishan-parihar/igs-mcp.git\ncd igs-mcp\nnpm install\nnpm run build\n```\n\n### 2. Configure Claude Desktop\n\nEdit `claude_desktop_config.json`:\n\n**macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`  \n**Windows:** `%APPDATA%\\Claude\\claude_desktop_config.json`  \n**Linux:** `~/.config/Claude/claude_desktop_config.json`\n\n```json\n{\n  \"mcpServers\": {\n    \"igs\": {\n      \"command\": \"node\",\n      \"args\": [\"/absolute/path/to/igs-mcp/dist/server.js\"]\n    }\n  }\n}\n```\n\n### 3. Restart Claude Desktop\n\nCompletely quit and relaunch Claude Desktop.\n\n---\n\n## 📚 Documentation\n\n| Document | Description |\n|----------|-------------|\n| [INSTALL.md](INSTALL.md) | Complete installation guide |\n| [AUDIT_REPORT.md](AUDIT_REPORT.md) | Installation audit \u0026 verification |\n| `docs/plans/` | Architecture \u0026 design documents |\n\n---\n\n## 🛠️ Available Tools\n\n### News Monitoring\n\n| Tool | Description | Example |\n|------|-------------|---------|\n| `news.fetch` | Fetch from curated sources | `news.fetch(pools=['GLOBAL_BREAKING'], keywords=['AI'], limit=10)` |\n| `news.testSource` | Test single source | `news.testSource(id='reuters')` |\n\n### Research Tools\n\n| Tool | Description | Example |\n|------|-------------|---------|\n| `reddit.search` | Search Reddit | `reddit.search(query='technology', subreddits=['news'], limit=10)` |\n| `research.search` | Search papers | `research.search(query='transformer', sources=['arxiv'], limit=10)` |\n| `research.paper` | Paper details + PDF | `research.paper(paperId='arxiv:2401.12345', extractPDF=true)` |\n\n### Source Management\n\n| Tool | Description |\n|------|-------------|\n| `sources.list` | List all sources |\n| `sources.upsert` | Add/update source |\n| `sources.delete` | Remove source |\n| `sources.countries` | Countries with source counts |\n| `sources.cities` | Cities with source counts |\n| `sources.domains` | Topic domains |\n\n### Pool Management\n\n| Tool | Description |\n|------|-------------|\n| `pools.list` | List all pools |\n| `pools.upsert` | Create/update pool |\n| `pools.delete` | Delete pool |\n\n---\n\n## 📊 Pre-configured Content\n\n### News Pools (14)\n\n| Pool | Description | Sources |\n|------|-------------|---------|\n| `GLOBAL_BREAKING` | Fast breaking news | Reuters, BBC, FT, Al Jazeera |\n| `GLOBAL_GEOECON` | Geopolitics \u0026 geoeconomics | Foreign Affairs, RAND, Brookings |\n| `GLOBAL_LAW_REG` | Law \u0026 regulation | OFAC, Lawfare, SCOTUSblog |\n| `GLOBAL_TECH_CYBER` | Tech \u0026 cyber | MIT Tech Review, Ars Technica, CISA |\n| `GLOBAL_ENV_HEALTH` | Environment \u0026 health | WHO, Mongabay, Nature |\n| `GLOBAL_CULT_SOC` | Culture \u0026 society | Aeon, UnHerd, Noema |\n| `INDIA_NATIONAL_BASE` | India national | The Hindu, Indian Express, Economic Times |\n| `INDIA_WATCHDOG` | India investigative | Scroll.in, The Wire, Article 14 |\n| `INDIA_FACTCHECK_DATA` | India fact-check | Alt News, BOOM Live, IndiaSpend |\n| `INDIA_BUSINESS_REG` | India business | Moneycontrol, NSE, RBI, SEBI |\n| `INDIA_REGION` | India regional | IE South/East/Northeast |\n| `INDIA_CITIES` | India cities | Delhi, Mumbai, Bengaluru feeds |\n| `GLOBAL_COUNTRIES` | Country feeds | Guardian/France24 country feeds |\n| `GLOBAL_CITIES` | Global cities | London, NYC, Singapore |\n\n### Source Inventory\n\n- **Total Sources:** 223\n- **Countries Covered:** 45\n- **Cities Covered:** 23\n- **Topic Domains:** 23 (geopolitics, finance, tech, science, etc.)\n\n---\n\n## 🔧 Configuration\n\n### User Config Directory\n\nIGS stores configuration in:\n\n- **Linux/macOS:** `~/.config/igs-mcp/`\n- **Windows:** `%APPDATA%\\igs-mcp\\`\n- **Custom:** Set `IGS_CONFIG_DIR` environment variable\n\n### Files\n\n```\n~/.config/igs-mcp/\n├── pools.yml      # Pool definitions\n├── sources.yml    # News sources (223 configured)\n└── settings.yml   # HTTP/cache settings\n```\n\n### Add Custom Source\n\nEdit `~/.config/igs-mcp/sources.yml`:\n\n```yaml\n- id: my_tech_blog\n  name: My Tech Blog\n  type: rss\n  url: https://example.com/feed.xml\n  parser: rss\n  pools: [GLOBAL_TECH_CYBER]\n  countries: [US]\n  domains: [tech]\n  is_active: true\n```\n\nNo restart required — changes load automatically.\n\n---\n\n## 🏗️ Architecture\n\n```\n┌─────────────────────────────────────────────┐\n│           MCP Client (Claude)               │\n└─────────────────┬───────────────────────────┘\n                  │ stdio\n┌─────────────────▼───────────────────────────┐\n│         IGS MCP Server (Node.js)            │\n├─────────────────────────────────────────────┤\n│  Tools: 11 MCP tools                        │\n│  • news.fetch      • research.search        │\n│  • reddit.search   • research.paper         │\n│  • sources.*       • pools.*                │\n├─────────────────────────────────────────────┤\n│  Parsers: 9 parsers                         │\n│  • RSS/Atom        • Generic HTML           │\n│  • Semantic Scholar• PDF extraction         │\n│  • OFAC, WHO, USSF (custom HTML)            │\n├─────────────────────────────────────────────┤\n│  APIs:                                      │\n│  • Reddit Search   • arXiv OAI-PMH          │\n│  • Semantic Scholar• Google News proxy      │\n└─────────────────────────────────────────────┘\n```\n\n---\n\n## ⚡ Performance\n\n| Metric | Value |\n|--------|-------|\n| Cold start | ~2 seconds |\n| news.fetch (9 sources) | ~3 seconds |\n| news.fetch (223 sources) | ~25 seconds |\n| Cache hit ratio | ~90% (warm) |\n| Memory usage | 50-150 MB |\n\n---\n\n## 🔒 Security\n\n- ✅ **Local-only** — Runs on your machine\n- ✅ **No telemetry** — No data collection\n- ✅ **HTTPS only** — All external requests encrypted\n- ✅ **No API keys required** — For basic functionality\n- ✅ **User-controlled config** — You own your data\n\n---\n\n## 📦 Dependencies\n\n### Runtime (15)\n\n- @modelcontextprotocol/sdk — MCP protocol\n- cheerio — HTML parsing\n- pdf-parse — PDF text extraction\n- xml2js — arXiv XML parsing\n- p-queue — Concurrency control\n- undici — HTTP client\n- zod — Schema validation\n- js-yaml — YAML parsing\n- rss-parser — RSS/Atom parsing\n- pino — Logging\n- compromise — NLP (enrichment)\n- sentiment — Sentiment analysis\n- keyword-extractor — Keyword extraction\n- date-fns + date-fns-tz — Date handling\n\n### Development (5)\n\n- typescript\n- @types/node\n- @types/js-yaml\n- @types/xml2js\n- ts-node-dev\n\n---\n\n## 🧪 Development\n\n```bash\n# Install\nnpm install\n\n# Development mode (auto-reload)\nnpm run dev\n\n# Build\nnpm run build\n\n# Type check\nnpm run typecheck\n\n# Start production\nnpm start\n```\n\n---\n\n## 🐛 Troubleshooting\n\n### Server won't start\n\n```bash\n# Check Node version (must be v20+)\nnode --version\n\n# Rebuild\nrm -rf dist/\nnpm run build\n```\n\n### Config not loading\n\n```bash\n# Check user config directory\nls -la ~/.config/igs-mcp/\n\n# Force bootstrap\nrm -rf ~/.config/igs-mcp/\nnpm start\n```\n\n### Cache issues\n\n```bash\n# Clear all caches\nrm -rf ~/.config/igs-mcp/cache/\nrm -rf ./cache/\n```\n\n### Rate limiting\n\n- **Semantic Scholar:** 100 requests/day (free tier)\n- **Reddit:** ~60 requests/minute\n- **arXiv:** 3000 requests/day\n\nTool includes automatic retry with backoff.\n\n---\n\n## 📝 Changelog\n\n### v0.2.0 (April 2026)\n\n**Major Upgrade: Research Tools**\n\n- ✨ NEW: `reddit.search` — Dynamic Reddit search\n- ✨ NEW: `research.search` — arXiv + Semantic Scholar\n- ✨ NEW: `research.paper` — PDF extraction with citations\n- 🔧 Parallel source fetching (6 concurrent)\n- 🔧 Historical date queries via Google News URL rewriting\n- 🗑️ REMOVED: RSS-based Reddit/arXiv/Semantic Scholar sources\n- 📚 Added INSTALL.md and AUDIT_REPORT.md\n- 🛠️ Added `setup.sh` one-line installer with auto-config\n\n### v0.1.0 (Initial Release)\n\n- Core news fetching with pools/sources\n- RSS/HTTP parsers\n- YAML configuration\n- Local caching with ETag support\n\n---\n\n## 📄 License\n\nMIT License — See [LICENSE](LICENSE) file.\n\n---\n\n## 🙏 Acknowledgments\n\n- MCP SDK: [Model Context Protocol](https://modelcontextprotocol.io/)\n- arXiv API: [arXiv.org](https://arxiv.org/)\n- Semantic Scholar: [Allen Institute for AI](https://www.semanticscholar.org/)\n- Reddit API: [Reddit](https://www.reddit.com/dev/api/)\n\n---\n\n## 📬 Support\n\n- **Issues:** https://github.com/ishan-parihar/igs-mcp/issues\n- **Discussions:** https://github.com/ishan-parihar/igs-mcp/discussions\n\n---\n\nDeveloped by [Ishan Parihar](https://github.com/ishanparihar) — If you find this useful, [consider supporting](https://rzp.io/rzp/ishan-parihar)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fishan-parihar%2Figs-mcp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fishan-parihar%2Figs-mcp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fishan-parihar%2Figs-mcp/lists"}