{"id":39497287,"url":"https://github.com/czcorpus/mquery-sru","last_synced_at":"2026-02-07T04:06:25.001Z","repository":{"id":215420537,"uuid":"738899811","full_name":"czcorpus/mquery-sru","owner":"czcorpus","description":"CLARIN FCS 2.0 Endpoint for Manatee-open corpus search engine","archived":false,"fork":false,"pushed_at":"2025-02-03T10:13:35.000Z","size":527,"stargazers_count":2,"open_issues_count":5,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-06-23T17:09:24.511Z","etag":null,"topics":["clarin","corpora","fcs","fcs-endpoint","linguistics"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/czcorpus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-04T09:59:40.000Z","updated_at":"2025-02-03T10:13:39.000Z","dependencies_parsed_at":"2024-06-21T12:53:46.274Z","dependency_job_id":"3acbc956-6f16-460f-8bba-7aff693838b1","html_url":"https://github.com/czcorpus/mquery-sru","commit_stats":null,"previous_names":["czcorpus/mquery-fcs","czcorpus/mquery-sru"],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/czcorpus/mquery-sru","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czcorpus%2Fmquery-sru","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czcorpus%2Fmquery-sru/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czcorpus%2Fmquery-sru/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czcorpus%2Fmquery-sru/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/czcorpus","download_url":"https://codeload.github.com/czcorpus/mquery-sru/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czcorpus%2Fmquery-sru/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29186091,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T03:35:06.566Z","status":"ssl_error","status_checked_at":"2026-02-07T03:34:57.604Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clarin","corpora","fcs","fcs-endpoint","linguistics"],"created_at":"2026-01-18T05:43:28.324Z","updated_at":"2026-02-07T04:06:24.987Z","avatar_url":"https://github.com/czcorpus.png","language":"Go","funding_links":[],"categories":["Endpoint Implementations"],"sub_categories":[],"readme":"# MQuery-SRU\n\nMQuery-SRU is an easy to set up endpoint for [Clarin FCS](https://www.clarin.eu/content/federated-content-search-clarin-fcs-technical-details) 2.0 (Federated Content Search) based on the [Manatee-open](https://nlp.fi.muni.cz/trac/noske) corpus search engine and developed and maintained by the [Institute of the Czech National Corpus](https://ucnk.ff.cuni.cz/en/).\n\n## Features\n\n* Full support for the [FCS-QL](https://clarin-eric.github.io/fcs-misc/fcs-core-2.0-specs/fcs-core-2.0.html#_fcs_ql_ebnf) query language\n    * definable mapping between FCS-QL layers and Manatee-open positional attributes\n* Level 1 support for basic search via CQL (Context Query\nLanguage)\n* simultaneous search in multiple defined corpora\n* (optional) backlinks to respective concordances in KonText\n\n\n## Requirements\n\n* a working Linux server with installed [Manatee-open](https://nlp.fi.muni.cz/trac/noske) library\n* [Redis](https://redis.io/) database\n* [Go](https://go.dev/)  language compiler and tools\n* (optional) an HTTP proxy server (Nginx, Apache, ...)\n\n\n## How to install\n\n1. Install `Go` language environment, either via a package manager or manually from Go [download page](https://go.dev/dl/)\n   1. make sure `/usr/local/go/bin` and `~/go/bin` are in your `$PATH` so you can run any installed Go tools without specifying a full path\n2. Install Manatee-open from the [download page](https://nlp.fi.muni.cz/trac/noske). No specific language bindings are required.\n   1. `configure --with-pcre --disable-python \u0026\u0026 make \u0026\u0026 sudo make install \u0026\u0026 sudo ldconfig`\n3. Get MQuery-SRU sources (`git clone --depth 1 https://github.com/czcorpus/mquery-sru.git`)\n4. Run `./configure`\n5. Run `make`\n6. Run `make install`\n      * the application will be installed in `/opt/mquery-sru`\n      * for data and registry, `/var/opt/corpora/data` and `/var/opt/corpora/registry` directories will be created\n      * systemd services `mquery-sru-server.service` and `mquery-sru-worker-all.target` will be created\n8. Copy at least one corpus and its configuration (registry) into respective directories (`/var/opt/corpora/data`, `/var/opt/corpora/registry`)\n9. Update corpora entries in `/opt/mquery-sru/conf.json` file to match your installed corpora\n10. start the service:\n      * `systemctl start mquery-sru-server`\n      * `systemctl start mquery-sru-worker-all.target`\n\n## HTTP access\n\nIn most cases, it is not recommended to expose the server directly to the Internet. It is therefore advisable to put the service behind an HTTP proxy.\nE.g. in Nginx, the configuration may look like this:\n\n```\nlocation /mquery-fcs/ {\n    proxy_pass http://127.0.0.1:8080/;\n    proxy_set_header Host $http_host;\n    proxy_redirect off;\n    proxy_read_timeout 30;\n    proxy_set_header X-Forwarded-For $remote_addr;\n    proxy_set_header X-Forwarded-Proto $scheme;    \n}\n```\n\n## Worker considerations\n\nIt's important to understand that endpoints experiencing low traffic can still benefit from having multiple workers. Specifically, if an endpoint is configured to search across multiple corpora, MQuery-SRU can leverage these workers to execute searches in parallel. This approach can significantly reduce the response time by querying all configured corpora simultaneously, thereby improving efficiency even under conditions of minimal load.\n\n## Configuration\n\nTo run the endpoint, you need at least\n\n1. to configure listening address and port\n2. defined path to your Manatee corpora registry (= configuration) files\n2. defined corpora along with:\n    * positional attributes to be exposed and also layer names they belong to\n    * mapping of FCS-QL's `within` structures (`s`, `sentence`, `p` etc.) to your specific corpora structures\n3. address of your Redis service plus a number of database to be used for passing queries and results around\n\nSee [configuration reference](https://github.com/czcorpus/mquery-sru/blob/main/config-reference.md) and/or [conf.sample.json](https://github.com/czcorpus/mquery-sru/blob/main/conf.sample.json) for detailed info.\n\n## OS integration (systemd)\n\nThis applies in case `make install` is not used.\n\n(Here we assume the service will run with user `www-data`)\n\nCreate a directory for logging (e.g. `/var/log/mquery-sru`) and set proper permissions for `www-data` to be able to write there.\n\nYou can use predefined systemd files from [/scripts/systemd/*](https://github.com/czcorpus/mquery-sru/tree/main/scripts/systemd). Copy (or link) them to `/etc/systemd/system` and then run:\n\n```\nsystemctl enable mquery-sru-server.service\nsystemctl enable mquery-sru-worker-all.target\n```\n\nNow you can try to run the service:\n\n```\nsystemctl start mquery-sru-server\nsystemctl start mquery-sru-worker-all.target\n```\n\n## See MQuery-SRU in action\n\nA CNC instance of MQuery-SRU is running as one of the endpoints for Clarin [Content Search](https://contentsearch.clarin.eu/) page.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fczcorpus%2Fmquery-sru","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fczcorpus%2Fmquery-sru","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fczcorpus%2Fmquery-sru/lists"}