{"id":18524375,"url":"https://github.com/binpash/annotations","last_synced_at":"2025-08-22T07:14:18.491Z","repository":{"id":41429926,"uuid":"459771209","full_name":"binpash/annotations","owner":"binpash","description":"Repo containing annotations for the PaSh project","archived":false,"fork":false,"pushed_at":"2025-05-27T21:51:56.000Z","size":475,"stargazers_count":2,"open_issues_count":9,"forks_count":7,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-01T06:30:40.232Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/binpash.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-15T22:30:26.000Z","updated_at":"2025-04-04T20:46:20.000Z","dependencies_parsed_at":"2025-04-04T21:39:43.575Z","dependency_job_id":null,"html_url":"https://github.com/binpash/annotations","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/binpash/annotations","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fannotations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fannotations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fannotations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fannotations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/binpash","download_url":"https://codeload.github.com/binpash/annotations/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fannotations/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265688321,"owners_count":23811416,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T17:41:09.139Z","updated_at":"2025-07-18T01:34:14.993Z","avatar_url":"https://github.com/binpash.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Annotations\n\nThis repository contains a framework for generating annotations for command invocations.\nIt comprises a parser which turns a string into a command invocation data structure.\nFor the time being, there are two sets of annotation generators:\n- input-output information which specifies how a command invocation interacts with the files, pipes, stdin, stdout, etc.\n- parallelizability information which describes how a command invocation can be parallelized - containing information about how to split inputs, mappers and aggregators, etc.\n\n## Command-line tool\n`main.py` contains a command line tool which, provided a command invocation returns:\n- the parsed command invocation data structure\n- the input-output information generated\n- the parallelizability information generated\n\n## Adding an annotation\n\n- Add the command in the dictionary in (https://github.com/binpash/annotations/blob/main/pash_annotations/annotation_generation/AnnotationGeneration.py#L13)\n- Add a json file with the command flags in (https://github.com/binpash/annotations/tree/main/pash_annotations/parser/command_flag_option_info/data). This could be used to generate a first version of it: (https://github.com/binpash/annotations/blob/main/pash_annotations/parser/command_flag_option_info/manpage-to-json.sh).\n- Add an `InputOutputInfoGeneratorXXX.py` in (https://github.com/binpash/annotations/tree/main/pash_annotations/annotation_generation/annotation_generators)\n- (Optionally) add a `ParallelizabilityInfoGeneratorXXX.py` in (https://github.com/binpash/annotations/tree/main/pash_annotations/annotation_generation/annotation_generators)\n\n## Parser\nUse command_flag_option_info JSON files to parse xbd-type terminal commands.\nWill split on spaces (`\" \"`) and equal signs (`\"=\"`).\n\n## Flag and Option Information\nThe folder command_flag_option_info contains [command_name].json files with list of flags and options for each command. \nFor arguments that have two options (e.g. `-a` and `--all`), store them as a pair in the format [short version, long version].\nIn addition, we store here in which way an argument is accessed if applicable, e.g., if it is a file.\n\nWe also have a regex-based script that can be used to generate initial JSON files with parsed arguments.\nSince there is no standard for man-pages, the quality of results varies but it usually provides a good skeleton and saves quite some time.\n\n## Annotation Generation\nCurrently, annotation generators for input-output information and parallelizability information has been implemented.\nEach annotation generator implements a specific generator interface (e.g., `InputOutputInfoGenerator_Interface.py`) which specializes a more general generator interface (`Generator_Interface.py`).\nThe general generator interface contains functions that help to check conditions on the command invocation while \nthe more specific generator interface provides functionality to change the respective information (object) generated.\n\n## Terms\n- flag = takes no arguments, e.g. `--verbose`\n- option = takes arguments, e.g. `-n 10`\n- operand = argument with no flag, e.g. `input.txt`\n\n## Coding\n\n## typing \nWe strive to use types and typecheck with `pyright` (v1.1.232).\nThis does not only help to catch bugs but shall also help future developers to understand the code more easily.\n\n## tests\nUse `pytest` to run tests. \nIt will run all tests found (recursively) in the current directory.\n\n## imports\nFor clean imports, we add empty `__init__.py` modules in all non-root directories.\nThus, `pytest` will add the root directory to sys.path and \nwe can import modules by prefixing the path from there.\nFor instance, to import `Parallelizer.py`, we use \n```\nfrom annotation_generation.parallelizers.Parallelizer import Parallelizer\n```\n\n# How to Add a New Command to PaSh's Annotation System\n\n\nHere is a detailed walkthrough on [how to add a new command to PaSh's Annotation System](docs/adding-annotations.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinpash%2Fannotations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbinpash%2Fannotations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinpash%2Fannotations/lists"}