https://github.com/zamgi/lingvo--postagger-en

Part-of-Speech Tagging / Normalization of words in English texts
https://github.com/zamgi/lingvo--postagger-en

linguistics lingvo morphological-analysis morphologies morphology natural-language-processing nlp nlp-machine-learning part-of-speech-tagging pos-tagger pos-tagging

Last synced: 10 days ago
JSON representation

Part-of-Speech Tagging / Normalization of words in English texts

• Host: GitHub
• URL: https://github.com/zamgi/lingvo--postagger-en
• Owner: zamgi
• License: mit
• Created: 2017-02-20T11:34:43.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2023-11-25T16:37:10.000Z (12 months ago)
• Last Synced: 2023-11-25T17:38:23.822Z (12 months ago)
• Topics: linguistics, lingvo, morphological-analysis, morphologies, morphology, natural-language-processing, nlp, nlp-machine-learning, part-of-speech-tagging, pos-tagger, pos-tagging
• Language: C#
• Homepage:
• Size: 20.5 MB
• Stars: 3
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# lingvo--PosTagger-ru

[ live demo ]



    


        


        Normalization of text is called to bring all the words of the text to the dictionary form: in the nominative case, a single number (if any) or the infinitive for verbs.

        


	    Normalization is necessary, for example, to quickly find words in dictionaries, syntactic and semantic analysis of the text.

        This procedure is especially important for language grammatical groups, such as russian or finnish, whose rich morphology (strong inflection in the investigation of the grammatical variation).

	    



	    Normalization is a disambiguation with and without it.

	    (Homonymy - Match words, the semantic values which does not involve).

	    Disambiguation means that a particular algorithm selected one word from the set of proposed morfoslovarem.

	    


	    Disambiguation - it time-consuming and costly in time and resources.

        Therefore, it is not often used in text processing systems, and cost "Stemming" - bringing the words to a normal form by the end of it.

        In this case, selecting from a plurality of normal form is random.

	    


	    Methods disambiguation different. The basis of most of them is the morphological and chastirechny analysis of each word, followed by the imposition of the rules of coordination and management.

	    



	    A special feature of this module is that it involves the entire sequence of linguistic processing of text:

	    



		    
 - the text is divided into proposals;

		    

 - defined parts of speech of words of text (called PoS-tagging - Part-of-Speech tagging);

		    

 - morphological characteristics are all words;

		    

 - removed homonymy;

	    



	    The final quality is determined by all processing steps and is defined as the morphological dictionaries, and statistical models.

        This makes it possible to achieve good results on the normalization of the relationship speed / quality.

	    

	    


	    The processing speed of this module is about 350 KB/sec.