Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with lexical-analysis

A curated list of projects in awesome lists tagged with lexical-analysis .

https://github.com/doctorwkt/acwj

A Compiler Writing Journey

c compiler lexical-analysis parsing

Last synced: 13 May 2025

https://github.com/DoctorWkt/acwj

A Compiler Writing Journey

c compiler lexical-analysis parsing

Last synced: 16 Mar 2025

https://github.com/baidu/lac

百度NLP：分词，词性标注，命名实体识别，词重要性

chinese-nlp chinese-word-segmentation java lexical-analysis named-entity-recognition part-of-speech-tagger python word-segmentation

Last synced: 14 May 2025

https://github.com/seanlee97/xmnlp

xmnlp：提供中文分词, 词性标注, 命名体识别，情感分析，文本纠错，文本转拼音，文本摘要，偏旁部首，句子表征及文本相似度计算等功能

lexical-analysis ner nlp pinyin postagging radical segmentation sentence-embeddings sentence-similarity sentiment-analysis spell-checker

Last synced: 15 May 2025

https://github.com/SeanLee97/xmnlp

xmnlp：提供中文分词, 词性标注, 命名体识别，情感分析，文本纠错，文本转拼音，文本摘要，偏旁部首，句子表征及文本相似度计算等功能

lexical-analysis ner nlp pinyin postagging radical segmentation sentence-embeddings sentence-similarity sentiment-analysis spell-checker

Last synced: 27 Mar 2025

https://github.com/dmitryryumin/interspeech-2023-24-papers

INTERSPEECH 2023-2024 Papers: A complete collection of influential and exciting research papers from the INTERSPEECH 2023-24 conference. Explore the latest advances in speech and language processing. Code included. Star the repository to support the advancement of speech technology!

acoustic adaptation asr audio-signals interspeech interspeech2023 interspeech2024 language-modeling lexical-analysis linguistic-analysis machine-translation prosody self-supervised-learning signal-processing speech-analysis speech-production speech-recognition speech-synthesis speech-technology transmission

Last synced: 24 Jan 2026

https://github.com/wgrape/lexer

A lexical analyzer based on DFA that is built using JS and supports multi-language extensions / 一个基于DFA的支持多语言扩展的JS版开源词法分析器

dfa javascript lexer lexical-analysis lexical-analyzer

Last synced: 05 Apr 2025

https://github.com/ghaiklor/llvm-kaleidoscope

LLVM Tutorial: Kaleidoscope (Implementing a Language with LLVM)

kaleidoscope language lexer lexical-analysis llvm llvm-ir llvm-tutorial

Last synced: 12 Apr 2025

https://github.com/lsys/lexicalrichness

:smile_cat: :speech_balloon: A module to compute textual lexical richness (aka lexical diversity).

data-mining data-science information-retrieval lexical-analysis lexical-analyzer linguistic-analysis natural-language natural-language-processing nlp python

Last synced: 09 Apr 2025

https://github.com/techcentaur/PyLex

Perform lexical analysis on words, one word at a time.

cli lexical-analysis nlp poets python3 scraping words

Last synced: 15 Mar 2025

https://github.com/quil-lang/alexa

A Lexical Analyzer Generator

common-lisp lex lexer lexical-analysis

Last synced: 16 Dec 2025

https://github.com/gbroques/compiler

A simple compiler written from scratch in C++ for an undergraduate course in program translation.

assembly-language compiler compiler-design compiler-frontend compiler-optimization lexical-analysis parse-trees parser scanner

Last synced: 11 Jun 2025

https://github.com/yaccconstructor/yaccconstructor

Platform for parser generators and other grammarware research and development. GLL, RNGLR, graph parsing algorithms, and many others are included.

ebnf fsharp gll-parser glr-parser grammarware lexical-analysis lexing parser-generator parsing

Last synced: 10 Jul 2025

https://github.com/soul-soft/sqlbatis

A high performance Micro-ORM supporting SQL Server, MySQL, Sqlite etc..

dapper expression lexical-analysis linq mybatis mybatis-net orm sqlbatis

Last synced: 25 Apr 2025

https://github.com/yassram/SwiLex

A universal lexer library in Swift.

lexer lexical-analysis swift-package-manager syntax-analysis tokenizer

Last synced: 22 Jul 2025

https://github.com/felipetomazec/lexical-analyzer

A basic Lexical-Analyzer written in Java.

compilers java javafx javafx-application lexical-analysis

Last synced: 10 Apr 2025

https://github.com/ftchirou/blink-lexer

Starting code of the first challenge in the Let's Build a Programming Language series.

compiler-design interpreter lbpl lexical-analysis programming-language-concepts

Last synced: 11 Aug 2025

https://github.com/acquitelol/elle

A procedural programming language built in Rust and QBE

c-style-lang compiler educational elle experimental language lexer lexical-analysis procedural qbe rust

Last synced: 15 Apr 2025

https://github.com/Yuziquan/CppCompiler

A C# application that simulates the important workflow of a C++ compiler, continuous updating ...

compiler compiler-principles dll flex gui lexical-analysis ll1-grammar syntax-analysis tiny winforms

Last synced: 12 Mar 2025

https://github.com/christianparpart/klex

C++17 (CLI/embeddable) Lexical Scanner Generator

generator lexer-generator lexical-analysis regular-expressions scanner

Last synced: 06 May 2025

https://github.com/yuziquan/cppcompiler

A C# application that simulates the important workflow of a C++ compiler, continuous updating ...

compiler compiler-principles dll flex gui lexical-analysis ll1-grammar syntax-analysis tiny winforms

Last synced: 06 May 2025

https://github.com/harmim/vut-ifj-project

Formální jazyky a překladače - Projekt - Implementace překladače imperativního jazyka IFJ17

c compiler ifj lexical-analysis semantic-analysis syntactic-analysis vut

Last synced: 25 Aug 2025

https://github.com/khaouitiabdelhakim/pascal-compiler

This repository contains a minimalistic Pascal compiler implemented in C, covering lexical and syntax analysis. It serves as a learning resource for understanding the fundamental aspects of compiler design. The code is intentionally kept concise to provide a clear insight into the key stages of lexical and syntax analysis in a compiler.

c-programming compiler lexical-analysis pascal pascal-programming syntax syntax-analysis

Last synced: 11 Sep 2025

https://github.com/adelrizq/a3e

A3E is a simple programming language compiler similar to C++ built using Flex and Bison and a simple GUI for testing.

a3e bison compiler compiler-design flex lex lexical-analysis semantic-analysis yacc

Last synced: 10 Feb 2026

https://github.com/sahilbansal17/ac_circuit_solver

Given the netlist, this program draws an AC circuit and finds the current/voltage values.

ac-circuits circuit-simulator circuit-solver cpp lexical-analysis yacc

Last synced: 18 Sep 2025

https://github.com/kariaro/ampleprogramminglanguage

Creating a compiler for my own programming language

compiler lexical-analysis programming-language

Last synced: 12 Apr 2025

https://github.com/takamin/lex-bnf

General-purpose recursive descent parser and evaluator

analyzer bnf bnf-parser javascript lex-bnf lexical-analysis npm parser-generator

Last synced: 21 Mar 2025

https://github.com/laputanmachines/simple-script

An interpreted, BASIC-like programming language. The language is built with Python. Project was built as a proof-of-concept language but has since been used for data processing at home.

basic-programming interpreted-programming-language interpreter lexer-parser lexical-analysis programming-language

Last synced: 12 Apr 2025

https://github.com/merzak-x/lexicalanalyser

Lexical Analyser is a Lexer / Tokenizer that determines whether a source code is accepted by a given Deterministic Finite State Automaton or not.

deterministic-finite-automata dfa dfsa lexical-analyser lexical-analysis lexical-analyzer

Last synced: 12 Apr 2025

https://github.com/mateuskoppe/lex-analyzer

Generate Deterministic Finite Automata and Non-Deterministic Finite Automata tables based on a input grammar

lexical-analysis

Last synced: 27 Mar 2026

https://github.com/seucs/compiler

SEU CS Compiler Course Design: Python Implementation of LEX & YACC for Cminus

compiler dfa lex lexical-analysis nfa python syntax-analysis visualization yacc

Last synced: 18 Aug 2025

https://github.com/discus-lang/inchworm

Simple parser combinators for lexical analysis.

haskell lexical-analysis parser-combinators parsing

Last synced: 29 Apr 2025

https://github.com/m0saan/minishell

The project's main idea is to implement a simple shell. As beautiful as a real shell.

1337school 42born2code bash c cat echo lexical-analysis minishell parsing pipes shell signals

Last synced: 26 Feb 2026

https://github.com/okeylanders/prose-minion-vscode

VS Code extension for creative writers combining AI-powered assistance with professional prose analysis. Track metrics, analyze word frequency, find patterns across manuscripts, and get context-aware suggestions—all in your sidebar. Offline tools for metrics and search, plus OpenRouter integration for AI features and fiction-focused dictionary.

ai-assistant ai-dictionary ai-tools anthropic chatgpt claude deepseek fiction gemini gpt lexical-analysis llm manuscript-metrics novelist-tools openai openrouter text-analysis text-metrics writing-assistant

Last synced: 30 Jun 2026

https://github.com/melchisedech333/lex-yacc-experiments

:wrench: My studies involving context-free grammar analysis. The analyzers were built using familiar tools such as YACC, Lex and Bison. Topics covered include token filtering, simple variable manipulation, and arrays.

bison bison-grammar bison-parser bison-yacc compiler-development language-development lex lex-example lex-yacc-parser lexical-analysis lexical-analyzer parser-generator parsing semantic-analysis syntactic-analysis tokenization tokenizer-parser yacc yacc-example yacc-lex

Last synced: 07 Feb 2026

https://github.com/ramonmeza/compiler

Trying to make a little compiler, just for fun and learning.

code code-generation compiler emitter lex lexer lexical-analysis lisp parser string-ify syntactic-analysis token tokenizer

Last synced: 10 Apr 2025

https://github.com/queckezz/json-tokenize

Splits a JSON string into an annotated list of tokens

json lexer lexical-analysis tokenize tokenizer tokens

Last synced: 15 May 2025

https://github.com/NFestoso/Compiler

Front-end Platypus compiler implemented in C. Includes Buffer, Scanner, and Parser. Platypus is a language comprised of parts from a variety of other languages.

buffer c compiler demo example finite-automata grammar interpreter lexical-analysis parser scanner

Last synced: 11 May 2025

https://github.com/drafakiller/tokenparser-dart

An intuitive Token Parser that includes grammar definition, tokenization, parsing, syntax error and debugging. Implementation based on Lexical Analysis for Dart.

dart debugging grammar lexer lexical-analysis package parser tokenizer

Last synced: 02 Aug 2025

https://github.com/grahms/slang-rust-lexical-analyzer

The SamoraLang Rust Lexical Analyzer is a lexer implemented in Rust for the SamoraLang programming language. This lexer is designed to tokenize SamoraLang source code, providing a foundation for further stages in the language processing pipeline.

lexer lexical-analysis lexical-analyzer rust-lang samora-lang

Last synced: 26 Feb 2026

https://github.com/youssefali11997/compiler-generator

Built a compiler generator tool which can be supplied with lexical and grammar rules of any programming language and used to generate the lexical and syntax analyzer needed to generate intermediate code.

compiler-design compilers dfa grammars lexical-analysis nfa regex syntax-analysis

Last synced: 28 Jul 2025

https://github.com/thiagoh/lex

From the classic "Lex - A Lexical Analyzer Generator" from http://dinosaur.compilertools.net/lex/index.html

c compilers cpp flex lex lexer lexical-analysis lexical-parser

Last synced: 24 Apr 2026

https://github.com/stoufa/mini-pascal-compiler

[EN] Simplified version of a Pascal language compiler | [FR] Version simplifié d'un compilateur du langage Pascal

bison c c-plus-plus compiler flex gui lexical-analysis pascal qt syntax-analysis

Last synced: 08 May 2026

https://github.com/katahiromz/macroparser

plain C++ parser example

lexer lexical-analysis parser

Last synced: 05 Jun 2026

https://github.com/graphpql/graphpinator-tokenizer

:zap::globe_with_meridians::zap: Tokenizer subproject for GraPHPinator: GraphQL server implementation for PHP.

graphql graphql-php lexical-analysis php tokenizer

Last synced: 06 May 2025

https://github.com/tarekmineroyal/minipascal-compiler

End-to-end compiler for MiniPascal, a simplified Pascal language. Built with C++, Flex, and Bison, it parses, semantically analyzes, and generates assembly for a custom stack-based VM.

abstract-syntax-tree bison c-plus-plus c-plus-plus-17 code-generation compiler compiler-construction compiler-design flex lexical-analysis minipascal parsing pascal programming-language semantic-analysis stack-based-virtual-machine syntax-analysis visitor-pattern

Last synced: 15 Jun 2025

https://github.com/ibrahimsharaf/pascal-compiler

A Compiler for Pascal programming language built using Gold parser connected with a C# application.

compilers lexical-analysis semantic-parser

Last synced: 03 Apr 2025

https://github.com/hediske/compiler

A Compiler made using Maven that offers to the user the possibility to provide the language of the compiler .works only on SLR grammars and generates the SLR table according to the grammar given , An SLR parser generator and type checking.

compilation compiler lexical-analysis semantic-analysis slr slr-parser syntax-analysis

Last synced: 23 Feb 2026

https://github.com/njmarko/spp_mips_translator

Program that translates MAVN(Higher level MIPS assembler) assembly language to MIPS 32bit assembly language. It creates zero-byte files.

compiler cplusplus graph-coloring lexical-analysis mavn mips-assembly register-allocation semantic-analysis syntax-analysis visitor-design-pattern zero-byte-files

Last synced: 09 Jun 2026

https://github.com/sanjinkurelic/mvcgenerator

Most of today's simpler software solutions closely link their work to basic database operations. The basic operations are retrieving, entering, modifying, and deleting individual data, so-called CRUD operations. To simplify the implementation of such software solutions, it is possible to build a code generator that builds a series of program sections based on the definition of the database structure. The database structure is defined using a structured query language (SQL).

analysis chunk code-generator compiler crud hibernate intermediate-code-generation java-11 junit lexical-analysis mvc parsing reflections rest-api semantic-analysis spring sql synthesis target-code-generation three-tier-architecture

Last synced: 19 May 2026

https://github.com/missipsag/lex-ical-analyzer

This project implements a lexical analyzer using Lex to tokenize and analyze the structure of a given input file. It efficiently identifies tokens such as keywords, identifiers, operators, and literals, serving as the foundational component of a compiler or interpreter.

compiler flex lex lexical-analysis lexical-analyzer lexical-parser

Last synced: 04 Feb 2026

https://github.com/upupming/compiler

Compiler implementation in node.js, just for teaching and learning purpose

compiler compilers grammar-analysis lexical-analysis semantic-analysis

Last synced: 09 Jul 2026

https://github.com/oleksiygarnik/programming-language-translator

This project implements a compiler for a custom programming language featuring lexical and syntax analysis, intermediate representation in Polish Notation, and code generation.

compiler compiler-design grammar intermediate-representation language-design lexical-analysis polish-notation syntax-analysis

Last synced: 06 Feb 2026

https://github.com/mgjules/jalex

JaLex is an extensible lexer built for educational purposes.

go golang lexer lexical-analysis

Last synced: 03 Apr 2025

https://github.com/vonderklaas/tiny-lexer

A program written in pure C language, that can perform lexical tokenization of an arbitrary programming language, 'tinylang' in this particular case.

c lexer lexer-parser lexical-analysis

Last synced: 10 Feb 2026

https://github.com/xstupi00/IFJ2017

Implementation compiler of imperative language IFJ17.

c compiler ifj17 lexical-analysis semantic-analysis syntactic-analysis vut vut-fit

Last synced: 11 Mar 2025

https://github.com/saiteja-talluri/cs-316-assignments

Assignments for the Compilers course in Spring 2019, IIT Bombay

compiler-construction compilers-design lexical-analysis parsing-engine

Last synced: 23 Jun 2025

https://github.com/yuqingc/lxa

Lexical analysis/Regular expression engine written in TypeScript

compiler lexical-analysis regular-expression regular-expression-engine

Last synced: 17 Mar 2025

https://github.com/pungrumpy/lexical-analyzer

🫀 Break down regular expression

java jflex lexical-analysis lexical-analyzer

Last synced: 13 Feb 2026

https://github.com/missipsag/lexic-analyser

This project is a Lexical Analyzer (Scanner) implemented in C, designed to tokenize a simple programming language. It processes an input string (source code) and identifies valid tokens such as keywords, identifiers, numbers, operators, and punctuation marks. The analyzer is based on a Deterministic Finite Automaton (DFA).

automaton c c-programming compilation dfa lexical-analysis lexical-analyzer programming-languages theory-of-programming-languages

Last synced: 30 Jan 2026

https://github.com/raihan4520/compiler-design

C and C++ projects from the Compiler Design course at AIUB, covering lexical analysis, syntax analysis, and code generation phases.

c compiler-design cpp intermediate-code-generation lexical-analysis syntax-analysis

Last synced: 15 May 2026

https://github.com/domfarolino/compiler

A compiler for Wilsey's compiler "theory" class

compiler lexer lexical-analysis ll1-grammar llvm parser

Last synced: 20 Jan 2026

https://github.com/kevyonan/text-input-math-calculator-workshop

educational workshop & material for teaching Computer Science & Computer Engineering students in how to create a math calculator using lexical analysis and parsing theory.

education educational educational-project lexer lexical-analysis lexical-analyzer math mathematics parsing

Last synced: 16 Jun 2025

https://github.com/kevyonan/Text-Input-Math-Calculator-Workshop

educational workshop & material for teaching Computer Science & Computer Engineering students in how to create a math calculator using lexical analysis and parsing theory.

education educational educational-project lexer lexical-analysis lexical-analyzer math mathematics parsing

Last synced: 28 Jun 2026

https://github.com/michel-leonard/turtle

Turtle Language Interpreter using C, Flex (lexical analysis) and Bison (parsing).

bison-flex grammar-parser lexical-analysis

Last synced: 28 Mar 2025

https://github.com/rangoo94/universal-lexer

Parse any text input to tokens, according to provided regular expressions.

lexer lexical-analysis parser parsing regular-expression scanner tokenizer

Last synced: 09 Sep 2025