{"id":13741419,"url":"https://github.com/RichardLitt/ldc-word-aligner","last_synced_at":"2025-05-08T21:34:07.630Z","repository":{"id":141499895,"uuid":"130917525","full_name":"RichardLitt/ldc-word-aligner","owner":"RichardLitt","description":"The LDC Word Aligner is a Python-based tool used for annotating manual word alignments (or gold standard alignments). Sentence-segmented parallel texts are required as input.","archived":false,"fork":false,"pushed_at":"2018-04-24T21:49:24.000Z","size":14920,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-22T09:49:05.900Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.ldc.upenn.edu/language-resources/tools/ldc-word-aligner","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RichardLitt.png","metadata":{"files":{"readme":"README.txt","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-04-24T21:49:00.000Z","updated_at":"2022-03-22T03:36:56.000Z","dependencies_parsed_at":"2024-01-07T18:10:50.711Z","dependency_job_id":null,"html_url":"https://github.com/RichardLitt/ldc-word-aligner","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardLitt%2Fldc-word-aligner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardLitt%2Fldc-word-aligner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardLitt%2Fldc-word-aligner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RichardLitt%2Fldc-word-aligner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RichardLitt","download_url":"https://codeload.github.com/RichardLitt/ldc-word-aligner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224774812,"owners_count":17367795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T04:00:59.024Z","updated_at":"2024-11-15T11:31:18.470Z","avatar_url":"https://github.com/RichardLitt.png","language":"Python","funding_links":[],"categories":["Software"],"sub_categories":["Utilities"],"readme":"LDC Word Aligner\nversion 1.20\n\nStephen Grimes\nsgrimes@ldc.upenn.edu\nLinguistic Data Consortium\nNovember 2011\n\nTable of Contents\n------------------------------------\n 1. Description\n 2. System Requirements\n 3. Package Contents\n 4. Getting Started\n  4.1 Starting the program\n  4.2 Startup Wizard\n  4.3 Using the program\n 5. Features / Usage\n  5.1 Aligment deletion or removing a token from an alignment\n  5.2 Adding a token to an alignment\n  5.3 Combining two links\n  5.4 Automatic file saving\n  5.5 Sentence navigation\n  5.6 Alignment table\n  5.7 Rejecting sentences\n  5.8 Undo\n  5.9 Colors\n 6. Known Issues\n 7. Copyright Info\n 8. Acknowledgments\n------------------------------------\n\n1. Description\n\nThe LDC Word Aligner is a Python-based tool used for annotating manual word\nalignments (or gold standard alignments). Sentence-segmented parallel texts\nare required as input.\n\n2. System Requirements\n\nThe tool was developed on a Debian Linux system using Python and PyQt.\nBefore using the tool, install python2.6 and PyQt4 and ensure that python2.6\nis in your path. On a Linux system using apt-get, this can be done as\nfollows:\n\n sudo apt-get install python2.6 python-qt4\n\nTest your system setup by typing \"python2.6\" to start python followed by\n\"import PyQt4\". If this is successful, you have met the required dependencies.\n\nA Windows executable was created using py2exe and is included. It has not been\nextensively tested.\n\n3. Package contents\n\ndocs/:\n\n README.txt -- this file\n gpl-3.0.txt -- GNU General Public License\n\n guidelines/\n   Annotator guidelines used by LDC under the GALE project.\n   LDC_GALE_Arabic_alignment_guidelines_v6.0.pdf\n   LDC_GALE_Chinese_WA_tagging_guidelines_v1.0.pdf\n   LDC_GALE_Chinese_alignment_guidelines_v4.0.pdf\n\n papers/\n   Conference proceedings related to LDC Word Aligner.\n\nsamples/:\n\n .wa files\n parallel texts for creating .wa files\n\nscripts/:\n\n makeWA.pl - create an empty .wa file from two parallel .txt files\n reverseWA.pl - switch source and translation lanaguages in a WA file\n compareWA.py - calculate precision, recall, f-score between two WA files\n WordAlignment.py - module used for a variety of manipulations of WA files\n\ntool/:\n\n ldc_word_aligner - wrapper script to launching tool\n\n code/\n    This directory contains all .py modules files required and the main\n    program, aligner.py. Additionally, tags.txt allows for adding or removing\n    possible word tags. See header of tags.txt for details.\n\n windows/\n    LDCWordAligner.v1.20.msi - MS Windows installer\n\n4. Getting Started\n\n 4.1 Starting the program\n\nThe user changes to the appropriate directory of the executable, or the user\ntypes the entire path to the exectuable. This is optionally followed by the\nfilename of the alignment file to open.\n\nIf no file is specified, a wizard is used to select an existing alignment\nfile or create one from a source and tokenized pair of files.\n\n  Sample usage:\n\n  a. Full paths, existing file:\n  /path/to/tool/aligner.py -f /path/to/file/filename.wa\n\n  b. Relative paths, create blank parallel file from source and tokenized:\n  aligner.py -s chinese.tkn -t english.tkn -f parallel.wa\n\n  c. Opening the file to a specific sentence, say the third:\n  /path/to/tool/aligner.py -i 3 -f /path/to/file/filename.wa\n\n  If the user does not use -i to specify a sentence to start at, the tool\n  will resume to the first sentence for which annotation has not been\n  completed.\n\n 4.2 File Wizard\n\nUsing the wizard a new .wa file can be created from two language files with\nan equal number of sentences. Each sentence must appear on one line. See\nexamples in the \"samples\" directory. The wizard can also be used to open an\nexisting .wa file.\n\nNote: Do not use the option to add \"raw\" files; it has not been fully\nimplemented. The wizard offers the option to include raw source files\n(untokenized), which can be used to display an alternate version of the\nsource and translation texts in the sentence navigation area. This current\nversion of the tool does not fully support raw text data (it will not be\nstored properly).\n\nAvoid using the LDC Arabic Config and LDC Chinese Config options. These are\nprimarily intedended for LDC-internal annotation.\n\n 4.3 Using the program\n\nThe user begins by highlighting words to be aligned together. When all\nnecessary tokens are selected, a link is created by selecting the \"Correct\"\nor \"Incorrect\" buttons (or choosing the corresponding keyboard shortcuts,\nspace and \"i\").\n\nTo assign tokens as \"Not Translated - Correct\" or \"Not Translated -\nIncorrect\" click the appropriate button or use the corresponding shortcut.\nFor convenience, multiple not translated tokens from the same language can\nbe highlighted at once and assigned \"Not Translated - Correct\" by pressing\nthe space bar.\n\nTags are selected by right clicking in the box containing a token and\nchoosing from the context menu. Default tags are Meta Word (MET),\nTokenization Error (TOK), and Typo (TYP), but these can be altered in the\ntags.txt configuration file according to the needs of a project.\n\n 4.4 Saving and exiting\n\nThe user may save the file at any time by selecting File-\u003eSave from the\nmenu.  The program automatically saves when advancing to the next sentence.\n\nThe program does not have Exit option in Menu. Instead, to quit, select the\nX at the top corner of your screen (or on some systems Alt+F4. You will be\ngiven the option to save; if you do not save, all annotated sentences but\nthe last are likely to have been saved.\n\n5. Program features\n\n 5.1 Alignment deletion or removing a token from an alignment\n\nAlignments may be deleted by first selecting the link to delete from the\nlower right link table, then clicking the Delete button in the bottom right\ncorner of the screen. Links selected by clicking them in the annotation\narea on the left of the screen can be deleted by clicking the Delete button\nor by pressing the Del key on the keyboard. Additionally, individual words\nmay be removed from a larger link by double clicking on the box of the\nword.  If the word is the last remaining word from one language in a link,\nthe entire link is removed. Deleting \"Not Translated\" links works the same\nway.\n\n 5.2 Adding a token to an alignment\n\nTokens can be added to existing links by selecting the existing link (either\nclick on a word that is part of the link or select the link in the table).\nOnce the existing link is selected, select the word to be added and press\nspace bar (or \"i\" for an incorrect link).\n\n 5.3 Combining two links\n\nMultiple alignments can be combined by selecting them while holding down\nCTRL on the keyboard and then clicking the appropriate link type button (or\npressing space bar or i) as usal. Once two or more existing links are\nselected with the CTRL key, it is possible to add previously unlinked words\nas described above in 5.2.\n\n 5.4 Automatic file saving\n\nThe user may save the file at any time by selecting File-\u003eSave from the\nmenu.  The program automatically saves when advancing to the next sentence.\n\n 5.5 Sentence navigation (upper right of display)\n\nThe Next and Previous buttons may be used for sentence navigation.\nAlternatively, the user may change to any sentence by double clicking on\nthat sentence in the source or translation boxes on the upper right side.\nThe Refresh button reloads the current sentence and can be used to revert\nannotations to the saved version.\n\n 5.6 Alignment table (lower right of display)\n\nThe alignment table displays all alignments for the current sentence. It can\nbe used to quickly review in table format the links for the sentence. The\ntable is sorted by a given column heading by clicking on that heading.\nSelecting a link brings it to focus in the annotation area to the left.\nLinks can be deleted from this table by clicking the Delete button at the\nbottom right corner of the screen.\n\n 5.7 Rejecting sentences\n\nAnnotators may explicity reject sentences. This is handled in the edit menu.\nThis makes clear that the sentence is invalid for annotation for some reason\nsuch as incorrect translation, offensive material, or plain gibberish. The\n.wa file for rejected sentences contains status=\"rejected\" instead of\nstatus=\"\".\n\n 5.8 Undo\n\nThe Undo button removes the last link in the link table.  This is usually\nthe last link added, although the order of links in the link table may\nchange if the sentence is reloaded.\n\n 5.9 Colors\n\n Annotation area (left side):\n\n Yellow      Unannotated tokens\n Lavender    Selected tokens\n Green       Composite links\n Light blue  Not translated\n Red         Incorrect, Not Translated Incorrect\n\n Sentence navigation area (upper right):\n\n White      Blank sentence (no alignments)\n Red        Partially-annotated sentence\n Green      Completely alignmed sentence\n\n6. Known issues\n\nIn order to tag a word, the user may have difficulty if right-clicking on\nthe word itself. Instead click in an area of the box where there are no\nwords or tags.\n\nCurrently one word is not permitted to be a member of multiple separate\nlinks. Also, if a word is a member of an alignment, it must be linked to all\nwords in the oppoistive language participating in the alignment. These\nbehaivors are by design.\n\nThe file format for alignments has grown organically but would be useful to\nbe modified eventually. The current format is designed to be human-readable.\n\n7. Acknowledgments\n\nThanks to John Mayer, Mishal Awadah, Pranshu Sharma, and Katherine Peterson\nfor their help in developing, enhancing, and testing features of the tool.\n\n8. Copyright Info\n\nLDC Word Aligner version 1.20 developed by the Linguistic Data Consortium.\nPortions (c) 2011 Trustees of the University of Pennsylvania\n\nPortions (c) 2001, 2002, 2003, 2004, 2005, 2006 Python Software\nFoundation, All Rights Reserved.\n\nParts of this program are written in Python, and this package\ncontains Python runtime.\n\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License version 3\nalong with this program. If not, see \u003chttp://www.gnu.org/copyleft/gpl.htm\u003e.\n\n----------------------\nREADME Created October 27, 2011 Stephen Grimes\nREADME Modified November 10, 2011 Stephen Grimes\nCode copied and uploaded April 24, 2018 Richard Littauer\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRichardLitt%2Fldc-word-aligner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRichardLitt%2Fldc-word-aligner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRichardLitt%2Fldc-word-aligner/lists"}