{"id":37982319,"url":"https://github.com/korpling/annatto","last_synced_at":"2026-04-10T11:01:19.568Z","repository":{"id":52192780,"uuid":"302373291","full_name":"korpling/annatto","owner":"korpling","description":"Converts linguistic data formats based on the graphANNIS data model as intermediate representation and can apply consistency tests.","archived":false,"fork":false,"pushed_at":"2026-03-30T15:24:16.000Z","size":6859,"stargazers_count":3,"open_issues_count":27,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-03-30T17:21:37.688Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/korpling.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-10-08T14:48:03.000Z","updated_at":"2026-03-30T15:24:45.000Z","dependencies_parsed_at":"2023-09-25T13:09:14.050Z","dependency_job_id":"f7991388-7060-40d2-8fc3-0dcf7bc1464e","html_url":"https://github.com/korpling/annatto","commit_stats":null,"previous_names":[],"tags_count":71,"template":false,"template_full_name":null,"purl":"pkg:github/korpling/annatto","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/korpling%2Fannatto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/korpling%2Fannatto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/korpling%2Fannatto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/korpling%2Fannatto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/korpling","download_url":"https://codeload.github.com/korpling/annatto/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/korpling%2Fannatto/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31639524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T07:40:12.752Z","status":"ssl_error","status_checked_at":"2026-04-10T07:40:11.664Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-16T18:35:46.954Z","updated_at":"2026-04-10T11:01:19.563Z","avatar_url":"https://github.com/korpling.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"![docs.rs](https://img.shields.io/docsrs/annatto)\n[![codecov](https://codecov.io/gh/korpling/annatto/graph/badge.svg?token=51VXZ3IJPA)](https://codecov.io/gh/korpling/annatto)\n\n# Annatto\n\nThis software aims to test and convert data within the [RUEG](https://hu.berlin/rueg)\nresearch group at Humboldt-Universität zu Berlin. Tests aim at \ncontinuously evaluating the state of the [RUEG corpus data](https://zenodo.org/record/3236068)\nto early identify issues regarding compatibility, consistency, and \nintegrity to facilitate data handling with regard to annotation, releases\nand integration. \n\nFor efficiency annatto relies on the [graphANNIS representation](https://korpling.github.io/graphANNIS/docs/v2.2/data-model.html)\nand already provides a basic set of data handling modules. We recommend to get acquianted with the [ANNIS Query language](http://korpling.github.io/ANNIS/4/user-guide/aql/index.html)\nto better understand the more advanced features of Annatto.\n\n## Installing and running annatto\n\nAnnatto is a command line program, which is available pre-compiled for Linux, Windows and macOS.\nDownload and extract the [latest release file](https://github.com/korpling/annatto/releases/latest) for your platform. \n\nAfter extracting the binary to a directory of your choice, you can run the binary by opening a terminal and execute\n```bash\n\u003cpath-to-directory\u003e/annatto\n```\non Linux and macOS and \n```bash\n\u003cpath-to-directory\u003e\\annatto.exe\n```\non Windows.\nIf the annatto binary is located in the current working directory, you can also just execute `./annatto` on Linux and macOS and `annatto.exe` on Windows.\nIn the following examples, the prefix to the path is omitted.\n\nThe main usage of annatto is through the command line interface. Run\n```bash\nannatto --help\n```\nto get more help on the sub-commands.\nThe most important command is `annatto run \u003cworkflow-file\u003e`, which runs all the modules as defined in the given [workflow] file.\n\n## Modules\n\nAnnatto comes with a number of modules, which have different types:\n\n**Importer** modules allow importing files from different formats.\nMore than one importer can be used in a workflow, but then the corpus data needs\nto be merged using one of the merger manipulators.\nWhen running a workflow, the importers are executed first and in parallel.\n  \n\n**Graph operation** modules change the imported corpus data.\nThey are executed one after another (non-parallel) and in the order they have been defined in the workflow.\n\n**Exporter** modules export the data into different formats.\nMore than one exporter can be used in a workflow.\nWhen running a workflow, the exporters are executed last and in parallel.\n\nTo list all available formats (importer, exporter) and graph operations run\n```bash\nannatto list\n```\n\nTo show information about modules for the given format or graph operation use\n```bash\nannatto info \u003cname\u003e\n```\n\nThe documentation for the modules are also included [here](https://github.com/korpling/annatto/blob/v0.51.0/docs/README.md).\n\n## Creating a workflow file\n\nAnnatto workflow files list which importers, graph operations and exporters to execute.\nWe use an [TOML file](https://toml.io/) with the ending `.toml` to configure the workflow.\nTOML files can be as simple as key-value pairs, like `config-key = \"config-value\"`.\nBut they allow representing more complex structures, such as lists.\nThe [TOML website](https://toml.io/) has a great \"Quick Tour\" section which explains the basics concepts of TOML with examples.\n\n### Import\n\nAn import step starts with the header `[[import]]`, and a\nconfiguration value for the key `path` where to read the corpus from and the key `format` which declares in which format the corpus is encoded.\nThe file path is relative to the workflow file.\nImporters also have an additional configuration header, that follows the `[[import]]` section and is marked with the `[import.config]` header.\n\n\n```toml\n[[import]]\npath = \"textgrid/exampleCorpus/\"\nformat = \"textgrid\"\n\n[import.config]\ntier_groups = { tok = [ \"pos\", \"lemma\", \"Inf-Struct\" ] }\nskip_timeline_generation = true\nskip_audio = true\nskip_time_annotations = true\naudio_extension = \"wav\"\n```\n\nYou can have more than one importer, and you can simply list all the different importers at the beginning of the workflow file.\nAn importer always needs to have a configuration header, even if it does not set any specific configuration option.\n\n```toml\n[[import]]\npath = \"a/mycorpus/\"\nformat = \"format-a\"\n\n[import.config]\n\n[[import]]\npath = \"b/mycorpus/\"\nformat = \"format-b\"\n\n[import.config]\n\n[[import]]\npath = \"c/mycorpus/\"\nformat = \"format-c\"\n\n[import.config]\n\n# ...\n```\n\n### Graph operations\n\nGraph operations use the header `[[graph_op]]` and the key `action` to describe which action to execute.\nSince there are no files to import/export, they don't have a `path` configuration.\n\n```toml\n[[graph_op]]\naction = \"check\"\n\n[graph_op.config]\n# Empty list of tests\ntests = []\n```\n\n### Export\n\nExporters work similar to importers, but use the keyword `[[export]]` instead.\n\n```toml\n[[export]]\npath = \"output/exampleCorpus\"\nformat = \"graphml\"\n\n[export.config]\nadd_vis = \"# no vis\"\nguess_vis = true\n```\n\n### Full example\n\nYou cannot mix import, graph operations and export headers. You have to first list all the import steps, then the graph operations and then the export steps.\n\n```toml\n[[import]]\npath = \"conll/ExampleCorpus\"\nformat = \"conllu\"\n\n[import.config]\n\n[[graph_op]]\naction = \"check\"\n\n[graph_op.config]\nreport = \"list\"\n\n[[graph_op.config.tests]]\nquery = \"tok\"\nexpected = [ 1, inf ]\ndescription = \"There is at least one token.\"\n\n[[graph_op.config.tests]]\nquery = \"node -\u003edep node\"\nexpected = [ 1, inf ]\ndescription = \"There is at least one dependency relation.\"\n\n[[export]]\npath = \"grapml/\"\nformat = \"graphml\"\n\n[export.config]\nadd_vis = \"# no vis\"\nguess_vis = true\n\n```\n\n\n## Developing annatto\n\nYou need to install Rust to compile the project.\nWe recommend installing the following Cargo subcommands for developing annis-web:\n\n- [cargo-release](https://crates.io/crates/cargo-release) for creating releases\n- [cargo-about](https://crates.io/crates/cargo-about) for re-generating the\n  third party license file\n- [cargo-llvm-cov](https://crates.io/crates/cargo-llvm-cov) for determining the code coverage\n- [cargo-insta](https://crates.io/crates/cargo-insta) allows reviewing the test snapshot files\n- [cargo-dist](https://crates.io/crates/cargo-dist) for configuring the GitHub actions that create the release binaries.\n\n### Execute tests\n\nYou can run the tests with the default `cargo test` command.\nTo calculate the code coverage, you can use `cargo-llvm-cov`:\n\n```bash\ncargo llvm-cov --open --all-features --ignore-filename-regex 'tests?\\.rs'\n```\n\n\n### Performing a release\n\nYou need to have [`cargo-release`](https://crates.io/crates/cargo-release)\ninstalled to perform a release. Execute the follwing `cargo` command once to\ninstall it.\n\n```bash\ncargo install cargo-release cargo-about\n```\n\nTo perform a release, switch to the main branch and execute:\n\n```bash\ncargo release [LEVEL] --execute\n```\n\nThe [level](https://github.com/crate-ci/cargo-release/blob/HEAD/docs/reference.md#bump-level) should be `patch`, `minor` or `major` depending on the changes made in the release.\nRunning the release command will also trigger a CI workflow to create release binaries on GitHub.\n\n\n## Funding\n\nDie Forschungsergebnisse dieser Veröffentlichung wurden gefördert durch die Deutsche Forschungsgemeinschaft (DFG) – SFB 1412, 416591334 sowie FOR 2537, 313607803, GZ LU 856/16-1.\n\nThis research was funded by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) – SFB 1412, 416591334 and FOR 2537, 313607803, GZ LU 856/16-1.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkorpling%2Fannatto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkorpling%2Fannatto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkorpling%2Fannatto/lists"}