{"id":15520840,"url":"https://github.com/proycon/colloquery","last_synced_at":"2025-06-30T08:34:17.991Z","repository":{"id":66967649,"uuid":"80527589","full_name":"proycon/colloquery","owner":"proycon","description":"Web application for searching for phrases/collocations/synonyms in phrase translation tables","archived":false,"fork":false,"pushed_at":"2019-10-17T14:49:40.000Z","size":329,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-24T04:16:14.998Z","etag":null,"topics":["computational-linguistics","machine-translation","mt","natural-language-processing","nlp"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/proycon.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":"codemeta.json","zenodo":null}},"created_at":"2017-01-31T14:28:50.000Z","updated_at":"2023-06-28T17:13:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"726df04e-8ce5-4541-9ce8-c2a53930a4c7","html_url":"https://github.com/proycon/colloquery","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/proycon/colloquery","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proycon%2Fcolloquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proycon%2Fcolloquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proycon%2Fcolloquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proycon%2Fcolloquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/proycon","download_url":"https://codeload.github.com/proycon/colloquery/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/proycon%2Fcolloquery/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262739384,"owners_count":23356714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computational-linguistics","machine-translation","mt","natural-language-processing","nlp"],"created_at":"2024-10-02T10:29:40.174Z","updated_at":"2025-06-30T08:34:17.868Z","avatar_url":"https://github.com/proycon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":".. image:: http://applejack.science.ru.nl/lamabadge.php/colloquery\n   :target: http://applejack.science.ru.nl/languagemachines/\n\n.. image:: https://www.repostatus.org/badges/latest/inactive.svg\n   :alt: Project Status: Inactive – The project has reached a stable, usable state but is no longer being actively developed; support/maintenance will be provided as time allows.\n   :target: https://www.repostatus.org/#inactive\n\nColloquery\n============\n\nColloquery is a web application to search for phrase translations, or\ncollocations, as well as synonyms,in bilingual phrase translation tables.\n\nIt is developed for `Van Dale \u003chttp://vandale.nl\u003e`_ by the `Centre for Language\nand Speech Technology \u003chttp://www.ru.nl/clst\u003e`_, Radboud University Nijmegen, and is licensed under the\nAffero GNU Public License.\n\n.. image:: https://raw.github.com/proycon/colloquery/master/screenshot.jpg\n    :alt: Colloquery screenshot\n    :align: center\n\nInstallation\n--------------\n\nFirst, clone this repository and edit ``settings.py``.\n\nColloquery is not trivial to set-up and train, as it relies on numerous\nexternal dependencies:\n\n* Python 3\n* `MongoDB \u003chttps://mongodb.com\u003e`_\n* `mongoengine \u003chttp://mongoengine.org\u003e`_\n* `Django \u003chttps://djangoproject.com\u003e`_\n\nOn Debian/Ubuntu systems, these can be installed using ``sudo apt-get install\npython3 mongodb python3-mongoengine python3-django``.\n\nFor the data generation step, the following additional dependencies are required:\n\n* `colibri-core \u003chttps://proycon.github.io/colibri-core\u003e`_ (shipped as part of\n  `LaMachine \u003chttps://proycon.github.io/LaMachine\u003e`_)\n* `colibri-mt \u003chttps://github.com/proycon/colibri-mt\u003e`_\n\nTo create phrase translation-tables in the first place, use the Moses training\npipeline, which in turn invokes GIZA++:\n\n* `Moses \u003chttp://statmt.org/moses/\u003e`_\n* `GIZA++ \u003chttps://github.com/moses-smt/giza-pp\u003e`_\n\nData Generation\n--------------------\n\n* Prepare your parallel corpus files. A parallel corpus consists of two plain-text UTF8 encoded\n  files, one for the source language (``corpus.fr`` in our example) and one for the target\n  language (``corpus.en``).  Make sure they are tokenised, lower-cased and\n  contain one sentence per line (you can use `ucto\n  \u003chttps://languagemachines.github.io/ucto\u003e`_ for this), sentences on the same line in the other file\n  are considering translations.\n* Train a phrase translation table using Moses::\n\n  $ /path/to/moses/scripts/training/train-model.perl -external-bin-dir /path/to/moses/bin -root-dir .  --parallel --corpus corpus --f fr --e en  --first-step 1 --last-step 8\n\n* Invoke the data generation pipeline of Colloquery, adjust the thresholds as\n  needed (see ``./manage.py generatedata --help``). This assumes a running\n  and properly configured MongoDB::\n\n  ./manage.py generatedata --title \"YourCorpus\" --phrasetable corpus.fr-en.phrasetable --sourcelang fr --targetlang en --targetcorpus corpus.fr --sourcecorpus corpus.en --pst 0.2 --pts 0.2 --divergencethreshold 0.1 --freqthreshold 4\n\nThe Moses and data generation pipeline may take considerable time and system\nresources (most notably memory). Set sane thresholds to prevent the data from\nbecoming unmanageably large.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproycon%2Fcolloquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fproycon%2Fcolloquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproycon%2Fcolloquery/lists"}