{"id":21467965,"url":"https://github.com/danr/korpusimportkedjan","last_synced_at":"2025-08-13T17:15:52.312Z","repository":{"id":5296984,"uuid":"6477861","full_name":"danr/korpusimportkedjan","owner":"danr","description":"Korpannoteringslabbet (Corpus Annotation Laboratory)","archived":false,"fork":false,"pushed_at":"2013-01-14T11:18:35.000Z","size":1396,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-17T06:13:47.717Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://spraakbanken.gu.se/korp/annoteringslabbet","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-10-31T17:26:38.000Z","updated_at":"2016-06-03T11:57:55.000Z","dependencies_parsed_at":"2022-09-23T21:52:41.137Z","dependency_job_id":null,"html_url":"https://github.com/danr/korpusimportkedjan","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/danr/korpusimportkedjan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danr%2Fkorpusimportkedjan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danr%2Fkorpusimportkedjan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danr%2Fkorpusimportkedjan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danr%2Fkorpusimportkedjan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danr","download_url":"https://codeload.github.com/danr/korpusimportkedjan/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danr%2Fkorpusimportkedjan/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270278577,"owners_count":24557182,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T08:20:54.631Z","updated_at":"2025-08-13T17:15:52.213Z","avatar_url":"https://github.com/danr.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Korpannoteringslabbet (Corpus Annotation Laboratory)\n\nThis project is split into three parts, the wsgi backend, the web frontend, and\nthe catapult which runs a python instance that shares the loaded lexicon, keeps\nmalt processes running and lowers the python interpreter startup time.\n\nThis project depends on the Corpus Pipeline's python scripts and Makefiles,\navailable from\n[Språkbanken](http://spraakbanken.gu.se/swe/forskning/infrastruktur/korp/distribution/corpuspipeline).\n\n## Backend\n\nThe backend is hosted at `demo` in `/export/htdocs_sb/annoteringslabb`, and its\nsubdirectory `pipeline` contains hosts running and completed builds. It is\navailable at [http://spraakbanken.gu.se/ws/korp/annotate](http://spraakbanken.gu.se/ws/korp/annotate).\n\nIf the backend is run with the python interpreter directly, it will try to use\nthe eventlet wsgi implementation if it exists. Otherwise it falls back on the\nreference implementation. Eventlet is preferred because it handles concurrent\nrequests.\n\n### Configuration\n\nThe `index.wsgi` has two variables at the top that needs to be configured:\n\n * `paths`: where the python paths to the `sb` python directory and the\n   directory of the backend is, and\n\n * `log_file_location`: location of the log. it can be omitted to log to\n   stdout.\n\nThe rest of the configuration is in `config.py`, the most important setting is\nthe pipeline's directory, and location of the settings schema.\n\n## Catapult\n\nThe catapult is running on `demo`, in the\n`/home/dan/annotate/webservice/catapult/` directory.\n\nScripts are run on the catapult with the tiny c program `catalaunch`, which\nis built by issuing `make` in the `catapult` directory.\n\n### Configuration\n\nDirectories are hard-coded into the keep alive script `keep-alive.sh`, and\nthe start server script `start-server.sh` and the saldo update script\n`update-saldo.sh`. Go nuts!\n\n## Frontend\n\nThe frontend is hosted at `k2` and available at\n[http://spraakbanken.gu.se/korp/annoteringslabbet](http://spraakbanken.gu.se/korp/annoteringslabbet).\n\n### Configuration\n\nThe `config.js` file contains the configuration of the backend's address, and\nalso the address to [Karp](http://spraakbanken.gu.se/karp/).\n\n## Settings JSON Schema\n\nThe backend creates the makefile from a JSON object that must satisfy the JSON\nschema stored in the backend. The frontend builds its form based on this\nschema. New entries can be added and hopefully the frontend will render them\nsomewhat ok. The file that creates the makefile from is\n`backend/make_makefile.py`.\n\n## Cron jobs\n\nThe scripts mentioned here in the `catapult` directory have some absolute\naddresses that needs to be configured.\n\nThe catapult is kept alive with `catapult/keep-alive.sh` and restarts it with\n`catapult/start-server.sh` if it does not respond to ping.\n\nThe optional script `catapult/update-saldo.sh` updates saldo. This takes some\ntime, and is therefore run during the night. The catapult is restarted\nafterwards by the `keep-alive.sh` script.\n\nBuilds that have not been accessed for 24 hours are removed every midnight by\nissuing [http://spraakbanken.gu.se/ws/korp/annotate/cleanup](http://spraakbanken.gu.se/ws/korp/annotate/cleanup).\n\nThe cronjobs are in `catapult/cronjobs`, and looks like this:\n\n    MAILTO=/dev/null\n    1 0 * * * curl http://spraakbanken.gu.se/ws/korp/annotate/cleanup\n    */5 * * * * /home/dan/annotate/webservice/catapult/keep-alive.sh\n    1 3 * * * /home/dan/annotate/webservice/catapult/update-saldo.sh\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanr%2Fkorpusimportkedjan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanr%2Fkorpusimportkedjan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanr%2Fkorpusimportkedjan/lists"}