https://github.com/lestrrat-go/lex
Simple lexer for Go
https://github.com/lestrrat-go/lex
Last synced: about 2 months ago
JSON representation
Simple lexer for Go
- Host: GitHub
- URL: https://github.com/lestrrat-go/lex
- Owner: lestrrat-go
- License: mit
- Created: 2014-03-28T03:17:26.000Z (about 11 years ago)
- Default Branch: master
- Last Pushed: 2018-02-21T00:11:56.000Z (over 7 years ago)
- Last Synced: 2024-06-20T01:59:02.630Z (about 1 year ago)
- Language: Go
- Homepage:
- Size: 30.3 KB
- Stars: 8
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
lex
======
This is a simple lexer, based loosely on `text/template`'s lexer.
[](https://travis-ci.org/lestrrat-go/lex)
[](http://godoc.org/github.com/lestrrat-go/lex)
## HOW TO USE
The lexing is done by chaining `lex.LexFn` functions. Create a `StringLexer` or a `ReaderLexer`, and pass it an entry point to start lexing. The result will be passed through a channel as a series of `lex.Item`s:
```go
l := NewStringLexer(buf, lexStart)
go l.Run()
for item := range l.Items() {
// Do whatever
}
```
In your lexing functions, you should do whatever processing necessary, and return the next lexing function. If you are done and want the lexing to stop, return a `nil` for `lex.LexFn`
```go
func lexStart(l lex.Lexer) lex.LexFn {
if !l.AcceptString("Hello") {
l.EmitErrorf("expected 'Hello'")
return nil
}
if !l.AcceptRun(" ") {
l.EmitErrorf("expected space")
return nil
}
return lexWorld
}
func lexWorld(l lex.Lexer) lex.LexFn {
if !l.AcceptString("World") {
l.EmitErrorf("expected 'World'")
return nil
}
// In reality we should check for EOF, but for now, we just end processing
return nil
}
```