{"id":16483494,"url":"https://github.com/dbohdan/tinyfts","last_synced_at":"2025-03-21T07:30:53.464Z","repository":{"id":52184092,"uuid":"230608477","full_name":"dbohdan/tinyfts","owner":"dbohdan","description":"Very small standalone full-text search HTTP/SCGI server","archived":false,"fork":false,"pushed_at":"2024-11-18T09:21:57.000Z","size":381,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-01T03:51:20.238Z","etag":null,"topics":["fts","fts5","full-text-search","scgi","search","search-engine","search-server","server","sqlite","tcl","wapp"],"latest_commit_sha":null,"homepage":"https://wiki.tcl-lang.org/page/tinyfts","language":"Tcl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dbohdan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-28T12:40:38.000Z","updated_at":"2024-11-18T09:22:00.000Z","dependencies_parsed_at":"2024-05-09T16:31:17.022Z","dependency_job_id":"001d7feb-cb11-4ce8-87c3-03c6d6c5fb22","html_url":"https://github.com/dbohdan/tinyfts","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbohdan%2Ftinyfts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbohdan%2Ftinyfts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbohdan%2Ftinyfts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dbohdan%2Ftinyfts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dbohdan","download_url":"https://codeload.github.com/dbohdan/tinyfts/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244119566,"owners_count":20401014,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fts","fts5","full-text-search","scgi","search","search-engine","search-server","server","sqlite","tcl","wapp"],"created_at":"2024-10-11T13:14:13.032Z","updated_at":"2025-03-21T07:30:53.458Z","avatar_url":"https://github.com/dbohdan.png","language":"Tcl","readme":"# tinyfts\n\n![CI badge](https://github.com/dbohdan/tinyfts/workflows/CI/badge.svg)\n\nA very small standalone full-text search HTTP/SCGI server.\n\n![A screenshot of what the unofficial tinyfts search service for the Tcler's Wiki looked like](screenshot.png)\n\n\n## Contents\n\n- [Dependencies](#dependencies)\n- [Usage](#usage)\n- [Query syntax](#query-syntax)\n- [Setup](#setup)\n- [Operating notes](#operating-notes)\n- [License](#license)\n\n\n## Dependencies\n\n### Server\n\n- Tcl 8.6\n- tclsqlite3 with [FTS5](https://sqlite.org/fts5.html)\n\n### Building, tools, and tests\n\nThe above and\n- Tcllib\n- kill(1), make(1), sqlite3(1)\n- tDOM and file(1) to run `tools/dir2jsonl`\n\nOn recent Debian and Ubuntu install the dependencies with\n\n```sh\nsudo apt install libsqlite3-tcl make sqlite3 tcl tcllib tdom\n```\n\nOn FreeBSD with sudo install the dependencies with\n\n```sh\nsudo pkg install sqlite3 tcl-sqlite3 tcl86 tcllib tdom\ncd /usr/local/bin\nsudo ln -s tclsh8.6 tclsh\n```\n\n\n## Usage\n\n```none\nUsage:\n    tinyfts --db-file path [option ...] [wapp-arg ...]\nOptions:\n    --css-file ''\n    --credits \u003cHTML\u003e\n    --header \u003cHTML\u003e\n    --footer \u003cHTML\u003e\n    --title tinyfts\n    --subtitle \u003cHTML\u003e\n    --table tinyfts\n    --rate-limit 60\n    --result-limit 100\n    --log 'access bad-request error rate'\n    --behind-reverse-proxy false\n    --snippet-size 20\n    --title-weight 1000.0\n    --query-min-length 2\n    --query-syntax web\n```\n\nThe basic usage is\n\n```sh\ntools/import jsonl example.jsonl example.sqlite3\n# Local server\n./tinyfts --db-file example.sqlite3 --local 8080\n# Server available over the network\n./tinyfts --db-file example.sqlite3 --server 8080\n```\n\n## Query syntax\n\n### Default or \"web\"\n\nThe default full-text search query syntax in tinyfts resembles that of a Web search engine.\nIt can handle the following types of expressions.\n\n- `foo` — search for the word `foo`.\n- `\"foo bar\"` — search for the phrase `foo bar`.\n- `foo AND bar`, `foo OR bar`, `NOT foo` — search for both `foo` and `bar`,\n  at least one of `foo` and `bar`,\n  and documents without `foo` respectively.\n  `foo AND bar` is identical to `foo bar`.\n  The operators `AND`, `OR`, and `NOT` must be in all caps.\n- `-foo`, `-\"foo bar\"` — the same as `NOT foo`, `NOT \"foo bar\"`.\n\n### FTS5\n\nYou can allow your users to write full\n[FTS5 queries](https://www.sqlite.org/fts5.html#full_text_query_syntax)\nwith the command line option `--query-syntax fts5`.\nFTS5 queries are more powerful but expose the technical details of the underlying database.\n(For example, the column names.)\nUsers who are unfamiliar with the FTS5 syntax will find it surprising and run into errors because they did not quote a word that has a special meaning.\n\n\n## Setup\n\nTinyfts searches the contents of an SQLite database table with a particular schema.\nThe bundled import tool `tools/import` can import serialized data\n(text files with one [JSON object](https://jsonlines.org/) or Tcl dictionary per line)\nand wiki pages from a [Wikit](https://wiki.tcl-lang.org/page/Wikit)/Nikit database to a tinyfts database.\n\n### Example\n\nThis example shows how to set up search for a backup copy of the\n[Tcler's Wiki](https://wiki.tcl-lang.org/page/About+the+WIki).\nThe instructions should work on most Linux distributions and FreeBSD with the dependencies and Git installed.\n\n1\\. Go to \u003chttps://sourceforge.net/project/showfiles.php?group_id=211498\u003e.\n    Download and extract the last Wikit  database snapshot of the Tcler's Wiki.\nCurrently that is `wikit-20141112.zip`.\n    Let's assume you have extracted the database file to `~/Downloads/wikit.tkd`.\n\n2\\. Download, build, and test tinyfts.\n    In this example we use Git to get the latest development version.\n\n```sh\ngit clone https://github.com/dbohdan/tinyfts\ncd tinyfts\nmake\n```\n\n3\\. Create a tinyfts search database from the Tcler's Wiki database.\n    The repository includes an import tool that supports Wikit databases.\n    Depending on your hardware, this may take up to several minutes with an input database size in the hundreds of megabytes.\n\n```sh\n./tools/import wikit ~/Downloads/wikit.tkd /tmp/fts.sqlite3\n```\n\n4\\. Start tinyfts on \u003chttp://localhost:8080\u003e.\n    The server URL should open automatically in your browser.\n    Try searching.\n\n```sh\n./tinyfts --db-file /tmp/fts.sqlite3 --title 'tinyfts demo' --local 8080\n```\n\n\n## Operating notes\n\n- If you put tinyfts behind a reverse proxy, remember to start it with the command line option `--behind-reverse-proxy true`.\n  It is necessary for\ncorrect client IP address detection, which rate limiting depends on.\n  Do **not** enable `--behind-reverse-proxy` if tinyfts is not behind a reverse proxy.\n  It will let clients spoof their IP with the header `X-Real-IP` or `X-Forwarded-For` and evade rate limiting themselves and rate limit others.\n\n\n## License\n\nMIT.\n[Wapp](https://wapp.tcl.tk/) is copyright (c) 2017-2022 D. Richard Hipp and is distributed under the Simplified BSD License.\n[Tacit](https://github.com/yegor256/tacit) is copyright (c) 2015-2020 Yegor Bugayenko and is distributed under the MIT license.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbohdan%2Ftinyfts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdbohdan%2Ftinyfts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdbohdan%2Ftinyfts/lists"}