{"id":50949613,"url":"https://github.com/immortalbob/kiwix-search","last_synced_at":"2026-06-18T00:02:38.353Z","repository":{"id":364180775,"uuid":"1266762882","full_name":"immortalbob/kiwix-search","owner":"immortalbob","description":"Command-line search tool for self-hosted Kiwix servers. Supports single-book and multi-library queries across offline ZIM collections.","archived":false,"fork":false,"pushed_at":"2026-06-12T01:04:44.000Z","size":38,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-12T02:11:06.624Z","etag":null,"topics":["agent-tools","beautifulsoup4","cli","homelab","ifixit","kiwix","local-ai","local-llm","offline","offline-first","open-source","privacy","python","search","self-hosted","stack-exchange","wikipedia","zim"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/immortalbob.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-11T23:44:29.000Z","updated_at":"2026-06-12T01:04:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/immortalbob/kiwix-search","commit_stats":null,"previous_names":["immortalbob/kiwix-search"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/immortalbob/kiwix-search","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/immortalbob%2Fkiwix-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/immortalbob%2Fkiwix-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/immortalbob%2Fkiwix-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/immortalbob%2Fkiwix-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/immortalbob","download_url":"https://codeload.github.com/immortalbob/kiwix-search/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/immortalbob%2Fkiwix-search/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34470324,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-tools","beautifulsoup4","cli","homelab","ifixit","kiwix","local-ai","local-llm","offline","offline-first","open-source","privacy","python","search","self-hosted","stack-exchange","wikipedia","zim"],"created_at":"2026-06-18T00:02:38.130Z","updated_at":"2026-06-18T00:02:38.338Z","avatar_url":"https://github.com/immortalbob.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kiwix-search\n\nA command-line search tool and importable Python module for querying a self-hosted [Kiwix](https://kiwix.org) server across one or multiple ZIM libraries. Built as a homelab utility and designed as connective tissue for local AI agent tool integration.\n\n## Requirements\n\n- Python 3.x\n- A running Kiwix server (see [kiwix-serve](https://github.com/kiwix/kiwix-tools))\n- Dependencies: `requests`, `beautifulsoup4`, `PyYAML`\n\n```bash\npip install -r requirements.txt\n```\n\n## Configuration\n\nCopy `config.example.yaml` to `config.yaml` and update to match your setup:\n\n```bash\ncp config.example.yaml config.yaml\n```\n\nKey config options:\n\n```yaml\nkiwix_server_url: \"http://192.168.3.5:8081\"   # your Kiwix server\ndefault_book: \"wikipedia_en_all_maxi_2026-02\"  # default book for unscoped searches\nresults_per_book: 3       # results fetched per book in --all mode\nmax_total_results: 10     # total results returned in --all mode\nsingle_book_limit: 5      # results returned for single book searches\nbooks:                    # list of ZIM files on your server\n  - \"wikipedia_en_all_maxi_2026-02\"\n  - ...\nkeyword_mapping:          # maps query keywords to relevant books for relevance weighting\n  raspberry pi:\n    - \"raspberrypi.stackexchange.com_en_all_2026-02\"\n  ...\n```\n\nIf `config.yaml` is not found the script falls back to built-in defaults with a warning. Book names correspond to ZIM filenames without the `.zim` extension.\n\n## CLI Usage\n\n### Search default book (Wikipedia)\n```bash\npython kiwix_search.py artificial intelligence\n```\n\n### Search a specific book\n```bash\npython kiwix_search.py --book unix.stackexchange.com_en_all_2026-02 bash scripting\npython kiwix_search.py -b raspberrypi.stackexchange.com_en_all_2026-02 gpio pwm\n```\n\n### Search all configured books (relevance-weighted)\n```bash\npython kiwix_search.py --all docker networking\npython kiwix_search.py -a raspberry pi gpio\n```\n\n### Fetch full article content\n```bash\npython kiwix_search.py --fetch /content/wikipedia_en_all_maxi_2026-02/Python_(programming_language)\npython kiwix_search.py -f /content/raspberrypi.stackexchange.com_en_all_2026-02/questions/117591/controlling-raspberry-pi-gpio-with-c-and-python\n```\n\n### Print OpenAI function calling schema\n```bash\npython kiwix_search.py --tool-definition\n```\n\n## Module Usage\n\n`kiwix_search` can be imported directly for use in scripts or agent tool integrations:\n\n```python\nfrom kiwix_search import search, fetch_article, get_tool_definition\n\n# Search default book (Wikipedia)\nresults = search(\"artificial intelligence\")\n\n# Search a specific book\nresults = search(\"gpio pwm\", book=\"raspberrypi.stackexchange.com_en_all_2026-02\")\n\n# Search all configured books with relevance weighting\nresults = search(\"raspberry pi gpio\", all_books=True)\n\n# Fetch full article text from a result URL path\narticle = fetch_article(results[0]['url_path'])\n\n# Get OpenAI function calling schema for agent integration\nschema = get_tool_definition()\n\n# Results are a list of dicts\nfor r in results:\n    print(r['title'])\n    print(r['book'])\n    print(r['excerpt'])\n    print(r['url_path'])\n```\n\n## How It Works\n\n- Single book searches return up to `single_book_limit` results (default 5)\n- `--all` / `all_books=True` uses keyword-based relevance weighting to prioritize the most relevant books for the query, fetches `results_per_book` results per book (default 3), deduplicates by URL path, and returns up to `max_total_results` (default 10)\n- `fetch_article()` retrieves the full plain text of any article by URL path — enables a complete search → read loop for agent use\n- `get_tool_definition()` returns the OpenAI function calling schema for drop-in agent integration\n- The Kiwix `/search` endpoint requires scoping to a specific book — searches without `books.name` will fail if your server hosts ZIMs in multiple languages\n\n## Relevance Weighting\n\nWhen using `--all`, the query is matched against `keyword_mapping` in `config.yaml` to prioritize relevant books. Matched books are searched first in order of relevance score; unmatched books follow. The mapping is fully user-configurable — add new keywords and books as your ZIM library grows.\n\n## ZIM Library\n\nThe default books list includes the following ZIM files. Update `config.yaml` to match your server:\n\n- `wikipedia_en_all_maxi_2026-02` — English Wikipedia (full)\n- `wiktionary_en_all_nopic_2025-09` — English Wiktionary\n- `ifixit_en_all_2025-12` — iFixit repair guides\n- `freecodecamp_en_all_2026-05` — freeCodeCamp\n- `freecodecamp_en_coding-interview-prep_2026-05` — freeCodeCamp interview prep\n- `freecodecamp_en_rosetta-code_2026-05` — Rosetta Code\n- `devdocs_en_nginx_2026-04` — nginx documentation\n- `devdocs_en_python_2026-05` — Python documentation\n- Stack Exchange communities: `ai`, `arduino`, `cs`, `datascience`, `dba`, `devops`, `electronics`, `iot`, `math`, `mathematica`, `movies`, `raspberrypi`, `retrocomputing`, `reverseengineering`, `robotics`, `scicomp`, `scifi`, `space`, `unix`\n\n## Tools\n\n### Open WebUI\n\n`tools/kiwix_openwebui_tool.py` is a native Open WebUI tool that exposes Kiwix search directly to any model running in Open WebUI.\n\n**Install:** Workspace → Tools → Create Tool → paste file contents → save.\n\nThe tool uses the same relevance-weighted book prioritization as the core library. Tag index pages are filtered from results automatically. Update `kiwix_url` and `books` in the `__init__` method to match your server.\n\n## Roadmap\n\n- [ ] Expand keyword mapping (open for contributions)\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimmortalbob%2Fkiwix-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimmortalbob%2Fkiwix-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimmortalbob%2Fkiwix-search/lists"}