{"id":51115177,"url":"https://github.com/faisalaffan/ragi-instant","last_synced_at":"2026-06-24T21:01:18.487Z","repository":{"id":360510875,"uuid":"1240731705","full_name":"faisalaffan/ragi-instant","owner":"faisalaffan","description":"RAG System that implement simple and fast. like ragi to make indonesian bread","archived":false,"fork":false,"pushed_at":"2026-05-26T19:11:45.000Z","size":2836,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":0,"default_branch":"dev","last_synced_at":"2026-05-26T19:17:28.361Z","etag":null,"topics":["document-processing","embeddings","hybrid-search","indonesian","llm","pgvector","production-ready","rag","rag-evaluation","regulatory-compliance","reranking","retrieval-augmented-generation","semantic-search","vector-sesarch"],"latest_commit_sha":null,"homepage":"https://showcase.faisalaffan.com/ragi-instant","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/faisalaffan.png","metadata":{"files":{"readme":"README.id.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":"SUPPORT.md","governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["faisalaffan"],"custom":["https://saweria.co/faisalaffan"]}},"created_at":"2026-05-16T13:54:21.000Z","updated_at":"2026-05-26T19:11:52.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/faisalaffan/ragi-instant","commit_stats":null,"previous_names":["faisalaffan/ragi-instant"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/faisalaffan/ragi-instant","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faisalaffan%2Fragi-instant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faisalaffan%2Fragi-instant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faisalaffan%2Fragi-instant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faisalaffan%2Fragi-instant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/faisalaffan","download_url":"https://codeload.github.com/faisalaffan/ragi-instant/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faisalaffan%2Fragi-instant/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34749211,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-24T02:00:07.484Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["document-processing","embeddings","hybrid-search","indonesian","llm","pgvector","production-ready","rag","rag-evaluation","regulatory-compliance","reranking","retrieval-augmented-generation","semantic-search","vector-sesarch"],"created_at":"2026-06-24T21:01:16.205Z","updated_at":"2026-06-24T21:01:18.435Z","avatar_url":"https://github.com/faisalaffan.png","language":"TypeScript","funding_links":["https://github.com/sponsors/faisalaffan","https://saweria.co/faisalaffan"],"categories":["Daftar Proyek"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"README.md\"\u003e🇬🇧 English\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets/01_BANNER_DARK.png\"\u003e\n    \u003cimg src=\"assets/02_BANNER_LIGHT.png\" alt=\"Ragi-Instant Banner\" width=\"100%\"\u003e\n  \u003c/picture\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets/03_ICON_DARK.png\"\u003e\n    \u003cimg src=\"assets/04_ICON_LIGHT.png\" alt=\"Ragi-Instant Logo\" width=\"120\"\u003e\n  \u003c/picture\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eRAG production-grade untuk kepatuhan regulasi. Hybrid search, reranking, routing, deteksi halusinasi — dalam satu pipeline.\u003c/strong\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/badge/status-active--development-green.svg\" alt=\"Status\"\u003e\u003c/a\u003e\n  \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.12+-blue.svg\" alt=\"Python\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n**Ragi-Instant** adalah sistem RAG (Retrieval-Augmented Generation) yang dibangun untuk regulatory \u0026 compliance intelligence. Melampaui siklus `embed → search → generate` ala tutorial dengan pipeline production-grade: query rewriting, intent routing, hybrid retrieval, cross-encoder reranking, context compression, structured output dengan kutipan, dan pengecekan halusinasi — semuanya ditracing dan dievaluasi otomatis.\n\n## Pipeline\n\n```\nPertanyaan User\n    │\n    ▼\nQuery Rewriting (LLM)     \"POJK terbaru pinjol\" → \"Peraturan OJK peer-to-peer lending 2024 2025\"\n    │\n    ▼\nIntent Router (LLM)       regulation_lookup | definition | comparison | obligation_check\n    │\n    ▼\nHybrid Search             Dense (pgvector cosine) + Sparse (PostgreSQL FTS BM25) → RRF fusion\n    │\n    ▼\nCohere Rerank v3          Top-40 → top-5 final\n    │\n    ▼\n[Context Compression]     Ringkasan LLM mempertahankan fakta legal (opsional)\n    │\n    ▼\nLLM Generation            GPT-4o mini / Claude Haiku + Instructor structured output\n    │\n    ▼\nHallucination Check       Verifikasi setiap klaim terhadap konteks sumber\n    │\n    ▼\nJawaban + Kutipan + Confidence + Regulasi Terkait\n    │\n    ▼\nLangFuse Trace + RAGAS Eval (async)\n```\n\n## Kemampuan Utama\n\n**Query Rewriting**\nEkspansi kueri berbasis LLM. \"POJK terbaru pinjol\" menjadi \"Peraturan OJK terbaru mengenai peer-to-peer lending 2024 2025\" sebelum retrieval. +15-20% kualitas retrieval untuk kueri ambigu.\n\n**Intent Routing**\nMengklasifikasikan kueri ke regulation_lookup, definition, comparison, atau obligation_check. Setiap intent dipetakan ke strategi pencarian yang berbeda (dense-only, sparse-only, atau hybrid).\n\n**Hybrid Search**\nPencarian vektor dense (pgvector HNSW) dikombinasikan dengan pencarian kata kunci sparse (PostgreSQL FTS BM25 dengan GIN index), difusikan melalui Reciprocal Rank Fusion. Menangkap kecocokan semantik dan istilah hukum eksak.\n\n**Reranking**\nCohere Rerank v3 cross-encoder pada kandidat hasil retrieval. Meningkatkan recall@5 secara signifikan dibanding raw vector similarity.\n\n**Context Compression**\nRingkasan berbasis LLM yang mempertahankan angka, persentase, nomor pasal, dan definisi hukum sambil menghapus transisi dan pengulangan. Pengurangan token 40-60%.\n\n**Structured Output \u0026 Kutipan**\nSetiap jawaban disertai kutipan sumber yang menunjuk ke chunk, dokumen, halaman, dan pasal yang tepat. Bukan jawaban black-box — setiap klaim bisa diverifikasi.\n\n**Deteksi Halusinasi**\nLangkah verifikasi pasca-generasi. Setiap klaim dalam jawaban diperiksa terhadap konteks sumber. Menghasilkan skor halusinasi dan menandai klaim yang tidak didukung.\n\n**Deteksi Perubahan Regulasi**\nUnggah dua versi regulasi dan dapatkan diff terstruktur — pasal mana yang berubah, apa yang ditambahkan, apa yang dihapus — dikelompokkan berdasarkan dampak (HIGH/MEDIUM/LOW).\n\n**Evaluasi Otomatis**\nPipeline evaluasi RAGAS bawaan dengan 30 pasangan Q\u0026A yang dikurasi. Mengukur faithfulness, answer relevancy, dan context precision. Anda rilis dengan angka, bukan asumsi.\n\n**Observabilitas**\nLangFuse tracing end-to-end setiap langkah: query rewriting, routing, hybrid search, reranking, compression, generation, dan hallucination check. Latency dan confidence per langkah.\n\n## Demo Aplikasi\n\nBerikut adalah pratinjau visual dari antarmuka dashboard utama:\n\n| Startup Splash Screen | Dashboard Sistem Utama |\n|---|---|\n| \u003cimg src=\"assets/DEMO_APPS/00_SPLASH_SCREEN.png\" width=\"100%\"\u003e | \u003cimg src=\"assets/DEMO_APPS/01_DASHBOARD.png\" width=\"100%\"\u003e |\n\n| AI Query Workspace \u0026 Kutipan | Benchmark Evaluasi RAGAS |\n|---|---|\n| \u003cimg src=\"assets/DEMO_APPS/11_RESPONSE_OK_WITH_80_PERCENT_CONFIDENT.png\" width=\"100%\"\u003e | \u003cimg src=\"assets/DEMO_APPS/14_EVALUATE_RAGAS.png\" width=\"100%\"\u003e |\n\n👉 **Jelajahi galeri lengkap berisi 16 tangkapan layar dengan penjelasan detail setiap langkah pipeline di [Galeri Demo Ragi Instant](./docs/demo.md).**\n\n## Instalasi\n\n```bash\ncd backend\npip install -e .\n```\n\n## Mulai Cepat\n\n```bash\n# 1. Salin template env dan isi API keys\ncp .env.example .env\n# Wajib: OPENAI_API_KEY, COHERE_API_KEY\n\n# 2. Deploy ke VPS\nmake deploy\n\n# 3. Unggah dokumen regulasi\ncurl -F \"file=@POJK_10_2022.pdf\" \\\n     -F \"title=POJK No. 10 Tahun 2022\" \\\n     http://vps:8000/api/ingest/documents\n\n# 4. Tanya regulasi\ncurl -X POST http://vps:8000/api/query \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"question\": \"Apa batas maksimum bunga pinjaman online menurut OJK?\"}'\n\n# Response:\n# {\n#   \"answer\": \"Berdasarkan POJK No. 10/PT. LKM/2022, batas maksimum...\",\n#   \"citations\": [{\"document_title\": \"POJK No. 10/2022\", \"page\": 12, \"quote\": \"...\"}],\n#   \"confidence\": 0.89,\n#   \"related_regulations\": [\"POJK No. 22/2023\"]\n# }\n```\n\n## API Reference\n\n| Endpoint | Method | Deskripsi |\n|---|---|---|\n| `/health` | GET | Health check |\n| `/api/ingest/documents` | POST | Unggah PDF/DOCX untuk indexing |\n| `/api/ingest/documents` | GET | Daftar semua dokumen |\n| `/api/ingest/documents/{id}` | GET | Detail dokumen |\n| `/api/ingest/documents/{id}/chunks` | GET | Lihat chunk hasil indeks |\n| `/api/query` | POST | Pipeline RAG lengkap |\n| `/api/analysis/compare` | POST | Bandingkan dua versi dokumen |\n\n## Benchmark\n\nDiuji pada 30 pasangan Q\u0026A dari dokumen regulasi keuangan Indonesia (POJK, PBI).\n\n| Metrik | Skor | Target |\n|---|---|---|\n| Faithfulness | **0.89** | \u003e 0.85 |\n| Answer Relevancy | **0.83** | \u003e 0.80 |\n| Context Precision | **0.76** | \u003e 0.75 |\n| Avg Latency | **1.27d** | \u003c 2.0d |\n| Avg Cost/Query | **\u003c$0.01** | \u003c $0.01 |\n\n*Skor dievaluasi menggunakan GPT-4o sebagai evaluator pada dataset patokan POJK \u0026 PBI.*\n\n## 🔍 Observabilitas \u0026 Kontrol Biaya (Production-Grade)\n\nDalam kepatuhan regulasi dan hukum, meluncurkan pipeline RAG secara membabi buta dan mengandalkan \"keberuntungan halusinasi\" adalah risiko besar. **Ragi Instant** menggunakan pendekatan berorientasi rekayasa (*engineering-first*): **kami melacak dan mengukur setiap token, latensi, biaya, dan langkah perantara.**\n\nMenggunakan **Langfuse**, seluruh alur hybrid RAG ditelusuri secara transparan:\n\n| Penelusuran End-to-End (Spans) | Kompresi Konteks \u0026 Biaya |\n|---|---|\n| \u003cimg src=\"assets/LANGFUSE_OBSERVABILITY/01_ALL_SPAN.png\" width=\"100%\"\u003e | \u003cimg src=\"assets/LANGFUSE_OBSERVABILITY/06_CONTEXT_COMPRESSION.png\" width=\"100%\"\u003e |\n\nKami melakukan penelusuran dan audit pada setiap langkah modular:\n*   **Query Rewriting**: Memantau bagaimana kueri ambigu diperluas.\n*   **Query Intent Routing**: Memeriksa klasifikasi niat (*intent classification*) oleh LLM dan target routing.\n*   **Hybrid Search**: Melacak log pencarian dense pgvector \u0026 sparse PostgreSQL FTS BM25.\n*   **Cohere Reranking**: Mengevaluasi skor cross-encoder dan perankingan kandidat.\n*   **Context Compression**: Mengaudit penghematan token dan kompresi ringkasan LLM sebelum proses generasi.\n*   **Hallucination Check**: Memverifikasi keselarasan klaim terhadap fakta konteks sumber untuk menangkap klaim tanpa dasar.\n\n👉 **Jelajahi trace lengkap untuk setiap langkah di [Galeri Observabilitas Langfuse](./docs/demo.md#5-production-grade-observability-langfuse).**\n\n## Stack\n\n| Layer | Teknologi |\n|---|---|\n| Document Parsing | Docling (layout-aware PDF + ekstraksi tabel) |\n| Chunking | LlamaIndex SemanticSplitterNodeParser |\n| Embedding | OpenAI text-embedding-3-small |\n| Vector Store | pgvector (HNSW index) |\n| Keyword Search | PostgreSQL FTS (BM25, GIN index) |\n| Backend API | FastAPI (async) |\n| Frontend | Next.js 15 + shadcn/ui + Tailwind CSS |\n| Tracing | LangFuse |\n| Evaluasi | RAGAS |\n\n## Lisensi\n\nMIT © [Muhammad Faisal Affan](https://github.com/faisalaffan)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaisalaffan%2Fragi-instant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffaisalaffan%2Fragi-instant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaisalaffan%2Fragi-instant/lists"}