{"id":22796022,"url":"https://github.com/eja/wikilite","last_synced_at":"2026-01-18T14:18:06.987Z","repository":{"id":267269923,"uuid":"900724067","full_name":"eja/wikilite","owner":"eja","description":"Offline Lexical and Semantic Wikipedia Search","archived":false,"fork":false,"pushed_at":"2025-11-28T13:16:33.000Z","size":17577,"stargazers_count":10,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-30T03:31:12.144Z","etag":null,"topics":["sqlite3","wikipedia"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eja.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-09T11:01:38.000Z","updated_at":"2025-11-28T13:15:57.000Z","dependencies_parsed_at":"2024-12-09T12:20:09.599Z","dependency_job_id":"c920f51f-4bd2-4e43-8476-13b9181b1cb1","html_url":"https://github.com/eja/wikilite","commit_stats":null,"previous_names":["eja/wikilite"],"tags_count":55,"template":false,"template_full_name":null,"purl":"pkg:github/eja/wikilite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eja%2Fwikilite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eja%2Fwikilite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eja%2Fwikilite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eja%2Fwikilite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eja","download_url":"https://codeload.github.com/eja/wikilite/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eja%2Fwikilite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28537498,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T13:04:05.990Z","status":"ssl_error","status_checked_at":"2026-01-18T13:01:44.092Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["sqlite3","wikipedia"],"created_at":"2024-12-12T05:09:42.550Z","updated_at":"2026-01-18T14:18:06.969Z","avatar_url":"https://github.com/eja.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# Wikilite\n\nWikilite is a self-contained tool for creating a local SQLite database of Wikipedia articles, indexed with FTS5 for efficient lexical searching with optional semantic search capabilities through embedded embeddings. Built with Go, Wikilite provides command-line tools, a web interface, and a native Android application for offline access, browsing, and searching of Wikipedia content.\n\n## Features\n\n* **Lexical Search**: Utilizes FTS5 for efficient keyword-based searching within the SQLite database, ideal for exact word and phrase matching.\n* **Optional Semantic Search**: Implements ANN quantization and MRL (Matryoshka Representation Learning) with text embeddings to find semantically similar content, effectively handling misspellings, morphological variations, and synonymy.\n* **Complete llama.cpp Integration**: Full AI engine embedded directly into Wikilite with GGUF models contained within database files.\n* **Cross-Platform \u0026 Android**: Available for Linux, macOS, Windows, Termux, and as a native Android application.\n* **Minimal Deployment**: Requires only the Wikilite executable and the database file on POSIX platforms.\n* **Offline Operation**: Complete functionality without internet connectivity.\n* **Dual Interfaces**: Command-line interface for terminal usage and web interface for browser-based access.\n* **Interactive Wizard**: When started without command-line options, Wikilite enters an interactive mode that guides users through database setup and search operations.\n\n## Installation\n\n### Source Compilation\n* Clone the repository: `git clone --recursive https://github.com/eja/wikilite.git`\n* Build the binary: `make`\n* Check the available options: `./build/bin/wikilite --help`\n\n### Pre-built Binaries \u0026 Android App\nPre-compiled binaries for Linux, macOS, Windows, and Termux are available in the [latest release](https://github.com/eja/wikilite/releases/latest).\n\nA native [Android application](https://github.com/eja/wikilite/releases/latest/download/wikilite-android.apk) is also available in the releases.\n*   **External Storage Support**: If a `wikilite.db` file is already present in the external SD card, the Android app will detect and use it directly.\n*   **In-App Download**: If no database is found on launch, the app provides an option to download a pre-built database.\n\n## Usage\n\nWikilite can be used in several ways:\n\n**Interactive Mode** (recommended for new users):\n```bash\n./wikilite\n```\nThis launches a wizard that guides you through database installation, CLI search, and web interface setup.\n\n**Direct Command Line**:\n```bash\n./wikilite --cli --db \u003cfile.db\u003e\n```\n\n**Web Interface Only**:\n```bash\n./wikilite --web --db \u003cfile.db\u003e\n```\nAccess the interface at `http://localhost:35248`\n\n## API Documentation\n\nWikilite provides a comprehensive RESTful API supporting both GET and POST methods. Key endpoints include:\n\n* `/api/search`: Combined search across titles, content, and vectors\n* `/api/search/title`: Title-specific search\n* `/api/search/lexical`: Full-text search of titles and content\n* `/api/search/semantic`: Vector-based semantic search\n* `/api/search/distance`: Vocabulary distance search\n* `/api/article`: Article retrieval by ID\n\nAll search endpoints support pagination via the `limit` parameter and return consistent JSON formatting. Complete API documentation is available in the [API specification](API.md).\n\n## Semantic Search Implementation\n\nThe semantic search functionality employs text embeddings with GGUF models embedded directly in the database. This approach identifies content with similar semantic meaning rather than relying solely on lexical matching, providing enhanced search capabilities for:\n\n* Query misspellings and typographical errors\n* Conceptual similarity despite different terminology\n* Synonym and related term matching\n* Morphological variations (plurals, verb tenses)\n\nSemantic search complements the FTS5 lexical search to deliver more comprehensive results.\n\n## Pre-built Databases\n\nPre-configured databases for multiple languages are available on [Hugging Face](https://huggingface.co/datasets/eja/wikilite/tree/main). These can be installed directly through the setup command, the interactive wizard, or downloaded and extracted manually.\n\nDatabases in the \"lexical\" directory support full-text search only, while others include both lexical and semantic search capabilities.\n\n## Acknowledgments\n\n* **Wikipedia**: For providing the valuable data that powers Wikilite.\n* **SQLite**: For providing the robust database engine that enables fast and efficient local data storage.\n* **LLaMA.cpp**: For enabling the internal generation of embeddings, enhancing the standalone semantic search capabilities of Wikilite.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feja%2Fwikilite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feja%2Fwikilite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feja%2Fwikilite/lists"}