https://github.com/basemax/simple-lexer-c
A lightweight and easy-to-use lexer library written in C. This library—named simple-lexer—is designed to simplify the creation of lexers by providing a set of utility functions and a flexible API for iterating over tokens.
https://github.com/basemax/simple-lexer-c
c c-language c-lexer clang lexer lexer-analyzer lexer-c lexer-example lexer-generator lexer-library lexer-parser lexers simple-lexer
Last synced: 7 months ago
JSON representation
A lightweight and easy-to-use lexer library written in C. This library—named simple-lexer—is designed to simplify the creation of lexers by providing a set of utility functions and a flexible API for iterating over tokens.
- Host: GitHub
- URL: https://github.com/basemax/simple-lexer-c
- Owner: BaseMax
- License: mit
- Created: 2025-03-30T16:06:54.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-03-30T16:53:11.000Z (10 months ago)
- Last Synced: 2025-06-09T22:47:08.828Z (7 months ago)
- Topics: c, c-language, c-lexer, clang, lexer, lexer-analyzer, lexer-c, lexer-example, lexer-generator, lexer-library, lexer-parser, lexers, simple-lexer
- Language: C
- Homepage:
- Size: 16.6 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# simple-lexer-c
A lightweight and easy-to-use lexer library written in C. This library—named **simple-lexer**—is designed to simplify the creation of lexers by providing a set of utility functions and a flexible API for iterating over tokens.
## Features
- **Simple API:** Easily define token types (e.g. punctuations, keywords) and iterate over them.
- **Customizable:** Supports custom token rules, single-line and multi-line comments.
- **Portable:** Written in standard C with minimal dependencies.
- **Extensible:** Designed to be adapted and extended for different projects.
## Files
- **simple-lexer.h**: The main header file containing the public API.
- **simple-lexer.c**: The implementation file (enable by defining `SIMPLELEXER_IMPLEMENTATION`).
- **example.c**: An example usage of the library demonstrating how to build and use the lexer.
## Getting Started
### Prerequisites
A C compiler (e.g. `gcc`, `clang`) and a standard POSIX environment (or any environment supporting standard C).
### Building the Example
1. Clone the repository:
```bash
git clone https://github.com/BaseMax/simple-lexer-c.git
cd simple-lexer-c
```
2. Compile the example using gcc:
```bash
gcc -o simple_lexer_example example.c simple-lexer.c
```
3. Run the example:
```bash
./simple_lexer_example
```
4. Integrating Into Your Project
Simply include the header in your source files:
```c
#include "simple-lexer.h"
```
Then compile the implementation file (simple-lexer.c) along with your project. Customize token types, comment markers, and other settings as needed.
## Example
Below is a short excerpt of how to initialize and use the lexer:
```c
#include
#include "simple-lexer.h"
int main(void) {
const char *file_path = "example.input";
const char *content =
"#include \n"
"if (a == 180*2 + 50) { // single line comment\n"
" /* multi-line\n"
" comment */\n"
" return b;\n"
"}\n";
SimpleLexer lexer = simplelexer_create(file_path, content, strlen(content));
SimpleLexer_Token token = {0};
while (simplelexer_get_token(&lexer, &token)) {
lexer.diagf(token.loc, "TOKEN", "%s: %.*s",
simplelexer_kind_name(SIMPLELEXER_KIND(token.id)),
(int)(token.end - token.begin), token.begin);
}
return 0;
}
```
## License
This project is licensed under the MIT License. See the LICENSE file for details.
## Contributing
Contributions are welcome! Feel free to fork this repository and submit pull requests. For major changes, please open an issue first to discuss what you would like to change.
## Acknowledgements
Special thanks to [this YouTube video](https://www.youtube.com/watch?v=pdsq1YeHLuQ) for helping and inspiring me to reproduce it.
Copyright 2025, Max Base