https://github.com/chobits/tinylex
A tiny lexical analyser generator
https://github.com/chobits/tinylex
lexer-generator
Last synced: 3 months ago
JSON representation
A tiny lexical analyser generator
- Host: GitHub
- URL: https://github.com/chobits/tinylex
- Owner: chobits
- Created: 2011-06-07T13:05:32.000Z (over 14 years ago)
- Default Branch: master
- Last Pushed: 2011-11-29T04:43:15.000Z (about 14 years ago)
- Last Synced: 2023-03-22T19:12:14.425Z (almost 3 years ago)
- Topics: lexer-generator
- Language: C
- Homepage:
- Size: 156 KB
- Stars: 14
- Watchers: 4
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README
Awesome Lists containing this project
README
+ What is tinylex?
A tiny lexical analyser generator
+ How to implement?
I refer to Allen Holub's book 'Compiler design in C'
and write it by myself.
+ Format of the input file
(see test/colour.l for example, which is a C syntax higlighter):
%{
< code block for definition or macro >
%}
< Macro Definition: macroname macrotext >
< e.g: digit [0-9] >
< recursive macro definition is not allowed.>
%%
< Rules: regularexp action string >
< e.g: {digit}+ printf("%d", yytext); >
< only one-line action string is allowed. >
%%
< code block for main function >
+ Usage:
compile:
# make
[generate tinylex and egrep_like ]
# ./tinylex -p lex.yy.part.c {source script}.l -o {generated ouput}.c
test:
# make test
[see the output information]
debug:
# make debug
[generate many debug programes, such as nfa]
# ./nfa test/common_test.l
[output NFAS information according to
common_test.l script (regular expresion for testing) ]
+ TODO:
1. bugfix:
using `grep -C 2 -n FIXME *.c` to get bug information
2. make tinylex compatible with flex/lex.
(Tinylex is only subset of flex/lex now.
And many auxiliary functions or flags aren't supported,
such as yywrap/yymore/yyrestart)
3. support multi-line accept action string
(Maybe I need a internal C parser.)
4. support recursive macro definition
+ Author:
Xiaochen Wang