{"id":18335979,"url":"https://github.com/lucasb-eyer/flex-bison-indentation","last_synced_at":"2025-04-06T04:34:45.648Z","repository":{"id":7712433,"uuid":"9077598","full_name":"lucasb-eyer/flex-bison-indentation","owner":"lucasb-eyer","description":"An example of how to correctly parse python-like indentation-scoped files using flex (and bison).","archived":false,"fork":false,"pushed_at":"2019-03-23T15:14:54.000Z","size":12,"stargazers_count":39,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-15T02:44:25.285Z","etag":null,"topics":["bison","flex","indentation","parse","python","scanner"],"latest_commit_sha":null,"homepage":null,"language":"Lex","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucasb-eyer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-03-28T13:29:30.000Z","updated_at":"2024-02-28T19:55:50.000Z","dependencies_parsed_at":"2022-09-09T04:51:28.791Z","dependency_job_id":null,"html_url":"https://github.com/lucasb-eyer/flex-bison-indentation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fflex-bison-indentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fflex-bison-indentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fflex-bison-indentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fflex-bison-indentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucasb-eyer","download_url":"https://codeload.github.com/lucasb-eyer/flex-bison-indentation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223238124,"owners_count":17111359,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bison","flex","indentation","parse","python","scanner"],"created_at":"2024-11-05T20:05:28.514Z","updated_at":"2024-11-05T20:05:29.222Z","avatar_url":"https://github.com/lucasb-eyer.png","language":"Lex","funding_links":[],"categories":[],"sub_categories":[],"readme":"flex-bison-indentation\n======================\n\nAn example of how to correctly parse python-like indentation-scoped files using flex (and bison).\n\nBesides that, this project also serves as a template CMake-based project for a flex\u0026bison parser\nand includes rules to track the current line and column of the scanner.\n\nQuick overview\n==============\n\nAll the magic happens in the scanner, which emits `TOK_INDENT` and `TOK_OUTDENT` tokens whenever\nthe level of indentation increases or decreases. The parser in this project just echoes the tokens.\n\nThe scanner includes the `\u003cnormal\u003e` mode which it starts in. That's where you\nput your regular rules. Whenever a newline is encountered in that mode, the\nparser enters the `\u003cindent\u003e` mode, in which it keeps counting the spaces and\ntabs (and ignoring blank lines) until it sees anything else, in which case it\noutputs either a `TOK_INDENT`, one or more `TOK_OUTDENT` as necessary or none\nof these tokens and goes back to `\u003cnormal\u003e` mode.\n\nThe scanner also does its best to keep track of the column where the current\nmatch starts, which can be accessed (and changed) through `yycolumn`. The line\nnumber is kept track of by flex internally.\n\nAll of this means that you can write the parser as usual, make use of the\n`TOK_INDENT` and `TOK_OUTDENT` tokens in order to handle indentation and access\nthe current line of tokens through `@1.first_line` (and `@1.last_line` if the\ntoken spans multiple lines, which I don't recommend.) and the column range of it\nthrough `@1.first_column` and `@1.last_column`.\n\nOne caveat is that if one of your rules includes a newline character and is\nmatches text longer than one symbol, you will need to reset `yycolumn` by hand.\n\nAnother one is that, for technical reasons, the column-range of the\n`TOK_INDENT` and `TOK_OUTDENT` tokens is the first character of the line or,\nfor outdents happening through reaching the end of the file, `0-0`.\n\nUntil I write a full tutorial, I recommend you look at the code, it is short and fully commented.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucasb-eyer%2Fflex-bison-indentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucasb-eyer%2Fflex-bison-indentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucasb-eyer%2Fflex-bison-indentation/lists"}