{"id":15566970,"url":"https://github.com/alixinne/glsl-lang","last_synced_at":"2025-04-06T12:10:43.995Z","repository":{"id":37795336,"uuid":"337169306","full_name":"alixinne/glsl-lang","owner":"alixinne","description":"LALR parser for GLSL","archived":false,"fork":false,"pushed_at":"2025-03-23T05:26:53.000Z","size":45668,"stargazers_count":24,"open_issues_count":5,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-30T11:08:51.777Z","etag":null,"topics":["glsl","parser","rust"],"latest_commit_sha":null,"homepage":"https://alixinne.github.io/glsl-lang/glsl_lang/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alixinne.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-08T18:22:59.000Z","updated_at":"2024-11-22T12:04:22.000Z","dependencies_parsed_at":"2023-10-24T21:35:23.504Z","dependency_job_id":"a3bbbb1d-5008-4397-9e25-74360caad9ab","html_url":"https://github.com/alixinne/glsl-lang","commit_stats":{"total_commits":439,"total_committers":11,"mean_commits":39.90909090909091,"dds":"0.15717539863325736","last_synced_commit":"8759d968397084dae98ed9239a7a6864ff7e1b1d"},"previous_names":["alixinne/glsl-lang","vtavernier/glsl-lang"],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alixinne%2Fglsl-lang","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alixinne%2Fglsl-lang/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alixinne%2Fglsl-lang/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alixinne%2Fglsl-lang/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alixinne","download_url":"https://codeload.github.com/alixinne/glsl-lang/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247478324,"owners_count":20945266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["glsl","parser","rust"],"created_at":"2024-10-02T17:09:13.523Z","updated_at":"2025-04-06T12:10:43.973Z","avatar_url":"https://github.com/alixinne.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [glsl-lang](https://github.com/alixinne/glsl-lang)\n\n[![Build](https://github.com/alixinne/glsl-lang/workflows/build/badge.svg?branch=master)](https://github.com/alixinne/glsl-lang/actions)\n[![Crates.io](https://img.shields.io/crates/v/glsl-lang)](https://crates.io/crates/glsl-lang)\n[![docs.rs](https://img.shields.io/docsrs/glsl-lang)](https://docs.rs/glsl-lang/)\n[![License](https://img.shields.io/github/license/alixinne/glsl-lang)](LICENSE)\n\n`glsl-lang` is a crate implementing a LALR parser for the GLSL language, with\nfull support for preprocessor directives. Its AST and features are modeled\nafter [Dimitri Sabadie's `glsl` crate](https://github.com/phaazon/glsl).\n\n## Table of contents\n\n\u003c!-- vim-markdown-toc GFM --\u003e\n\n* [Repository structure](#repository-structure)\n* [`glsl-lang` vs. `glsl` crates](#glsl-lang-vs-glsl-crates)\n  * [Why pick this crate?](#why-pick-this-crate)\n    * [It's fast](#its-fast)\n    * [Syntax nodes have location information](#syntax-nodes-have-location-information)\n    * [Re-written GLSL transpiler](#re-written-glsl-transpiler)\n    * [`glsl-lang-quote` quoting support](#glsl-lang-quote-quoting-support)\n    * [Full preprocessing support](#full-preprocessing-support)\n    * [Tested on the glslangValidator test suite](#tested-on-the-glslangvalidator-test-suite)\n  * [Why not pick this crate?](#why-not-pick-this-crate)\n    * [Stateful lexer](#stateful-lexer)\n    * [Parser generation and compile times](#parser-generation-and-compile-times)\n    * [`glsl-lang-quote` state](#glsl-lang-quote-state)\n    * [AST differences](#ast-differences)\n    * [Documentation](#documentation)\n* [Limitations](#limitations)\n* [License](#license)\n\n\u003c!-- vim-markdown-toc --\u003e\n\n## Repository structure\n\n| crates.io                                                                                                   | Path                                   | Description                                                       |\n| ---                                                                                                         | ---                                    | ---                                                               |\n| [![Crates.io](https://img.shields.io/crates/v/glsl-lang)](https://crates.io/crates/glsl-lang)               | [`lang`](lang)                         | Parser, visitor, transpiler for GLSL language                     |\n| [![Crates.io](https://img.shields.io/crates/v/glsl-lang-pp)](https://crates.io/crates/glsl-lang-pp)         | [`lang-pp`](lang-pp)                   | standalone preprocessor for the GLSL language                     |\n| [![Crates.io](https://img.shields.io/crates/v/glsl-lang-lexer)](https://crates.io/crates/glsl-lang-lexer)   | [`lang-lexer`](lang-lexer)             | Lexers for the GLSL language                                      |\n| [![Crates.io](https://img.shields.io/crates/v/glsl-lang-types)](https://crates.io/crates/glsl-lang-types)   | [`lang-types`](lang-types)             | AST and shared type definitions for the GLSL language             |\n| [![Crates.io](https://img.shields.io/crates/v/glsl-lang-quote)](https://crates.io/crates/glsl-lang-quote)   | [`lang-quote`](lang-quote)             | proc-macro crate to parse GLSL at compile-time                    |\n| [![Crates.io](https://img.shields.io/crates/v/glsl-lang-cli)](https://crates.io/crates/glsl-lang-cli)       | [`lang-cli`](lang-cli)                 | simple CLI tool to show GLSL syntax trees                         |\n| [![Crates.io](https://img.shields.io/crates/v/lang-util)](https://crates.io/crates/lang-util)               | [`lang-util`](lang-util)               | utilities for implementing syntax trees                           |\n| [![Crates.io](https://img.shields.io/crates/v/lang-util-derive)](https://crates.io/crates/lang-util-derive) | [`lang-util-derive`](lang-util-derive) | proc-macro crate to implement a syntax tree with span information |\n| [![Crates.io](https://img.shields.io/crates/v/lang-util-dev)](https://crates.io/crates/lang-util-dev)       | [`lang-util-dev`](lang-util-dev)       | development utilities for parsers implemented using lang-util     |\n|                                                                                                             | [`xtask`](xtask)                       | task runner, invoke with `cargo xtask`                            |\n\n## `glsl-lang` vs. `glsl` crates\n\n### Why pick this crate?\n\n#### It's fast\n\nDue to using a LALR parser and dedicated tokenizer, it's 5-480x (average case\n10x) faster than `glsl`:\n\n    $ cargo bench --bench glsl -- --samples 1000\n    # Install with `cargo install critcmp`:\n    $ critcmp new -g '([a-zA-Z0-9._-]*)/\\w+'\n    group                                               new//glsl                               new//glsl_lang\n    -----                                               ---------                               --------------\n    preprocessor.extensions.vert                        4.27     24.6±0.09µs     9.9 MB/sec     1.00      5.8±0.05µs    42.5 MB/sec\n    [...]\n    300operations.frag                                  10.70  1802.5±4.87µs   836.5 KB/sec     1.00    168.4±1.51µs     8.7 MB/sec\n    310runtimeArray.vert                                10.32   317.7±0.77µs   965.2 KB/sec     1.00     30.8±0.10µs     9.7 MB/sec\n    [...]\n    400.vert                                            13.41     2.8±0.01ms   589.5 KB/sec     1.00    209.1±5.26µs     7.7 MB/sec\n    [...]\n    deepRvalue.frag                                     25.90     2.5±0.01ms   351.4 KB/sec     1.00     97.3±0.31µs     8.9 MB/sec\n    nested_parens                                       483.25     3.6±0.04ms    10.1 KB/sec    1.00      7.4±0.12µs     4.8 MB/sec\n\n#### Syntax nodes have location information\n\nMost nodes in the AST are wrapped in a special `Node` type, which holds:\n\n* `source_id`: an `usize` to identify which parsing pass produced this node\n* `start`: the starting offset of the node in the corresponding input\n* `end`: the ending offset of the node in the corresponding input\n\n#### Re-written GLSL transpiler\n\nThe GLSL transpiler has been partially rewritten to generate indented code.\nIt's still a work-in-progress but generates (mostly) readable code.\n\n#### `glsl-lang-quote` quoting support\n\n`glsl-lang-quote` is the `glsl-lang` version of `glsl-quasiquote`. It parses\nGLSL at compile-time to generate an AST. However, you can also insert parts\nof runtime-generated AST using a quoting syntax. Currently, the following\ninsertion locations for the `#(rust code)` syntax are supported:\n\n* Identifier\n* Expression\n* Function name\n\n#### Full preprocessing support\n\n`glsl-lang-pp` implements a preprocessor following the GLSL 4.60 language\nspecification. While this adds a significant amount of complexity,\npreprocessing now happens in a proper stage before language parsing, thus\nsupporting a wider family of inputs.\n\nSince the preprocessing stage is also responsible for enabling/disabling\nextensions and/or pragmas, this allows us to track extra state at the token\ngranularity.\n\nThe preprocessor also supports include directives:\n* `GL_ARB_shading_language_include`: run-time includes\n* `GL_GOOGLE_include_directive`: compile-time includes\n\nThe preprocessor and lexer based on `glsl-lang-pp` can be used in `glsl-lang`\nby enabling the `glsl-lang/lexer-v2` feature. If it is used, the\n`glsl-lang/lexer-v1` feature including the old non-preprocessing lexer can be\ndisabled.\n\n#### Tested on the glslangValidator test suite\n\nThe `data` folder contains vendored test data from the\n[glslangValidator](https://github.com/KhronosGroup/glslang) project to be used\nas a reference point for validating the preprocessor and parser.\n\nThe `#[test]` definitions need to be generate before running the test suite on\nthe glslang sources. Use the `gen-tests` task for this:\n\n    cargo xtask gen-tests\n\nThen run the tests:\n\n    cargo test --test glslang\n\n`glsl-lang-pp` and `glsl-lang` are tested against this test suite. This is a\nsnapshot-based test suite which checks the following:\n\n* `glsl-lang-pp`: errors, events, preprocessed output and preprocessing AST\n* `glsl-lang`: if parsing succeeds, AST, else first parsing error\n\nSnapshots have not been thoroughly checked, i.e. the `glslang` test passing for\nboth crates does not mean we are spec-compliant yet. Please open issues if you\nencounter parsing or preprocessing errors.\n\n`glslang` tests are run during CI but are currently non-fatal. They are used to\ntrack the progress towards a spec-compliant parser.\n\n### Why not pick this crate?\n\n#### Stateful lexer\n\nC-based grammar are ambiguous by definition. The main ambiguity being the\ninability of the parser to solve conflicts between type names and identifiers\nwithout extra context. Thus, to enable LALR parsing of GLSL, we need to\nmaintain a list of identifiers that are declared as type names, so the lexer\ncan properly return `IDENT` or `TYPE_NAME` as it is reading the file.\n\nDepending on your use case, this might prove unwieldy since the parser is not\ncontext-free. Parsing one translation unit followed by another requires\nforwarding the type name/identifier disambiguation table to the second pass.\n\n#### Parser generation and compile times\n\nThe GLSL grammar is implemented in\n[`lang/src/parser.lalrpop`](lang/src/parser.lalrpop) using\n[LALRPOP](https://github.com/lalrpop/lalrpop). The default feature set only\nallows parsing translation units (the top-level rule in the GLSL grammar),\nwhich results in a 25k lines parser file. If you want to include more parsers\n(for example for expressions, statements, etc.) you will need to enable the\nrespective features (`parser-expr`, `parser-statement`, etc.) but this will\nslow down the compilation of `glsl-lang` by a significant amount.\n\nTo alleviate this issue, you can use the `Parsable` trait: by wrapping a syntax\nitem in a suitable source, and then matching the resulting AST, we can extract\nthe result of any rule in the grammar. Currently, this interface panics if the\noutput AST cannot be matched, so don't use it on unknown input. It's fine for\ntesting though.\n\n#### `glsl-lang-quote` state\n\nParsing preprocessor directives is currently not supported.\n\n#### AST differences\n\nThere are some differences in both crate's ASTs, so porting to `glsl-lang`\nwould require some changes to your code:\n* The `Statement/SimpleStatement/CompoundStatement` structure was flattened to `Statement`\n* The `subroutine` storage qualifier takes a `TypeSpecifier` array instead of a `TypeName` array\n* `FunIdentifier::Identifier` was replaced with `FunIdentifier::TypeSpecifier`:\n  this reflects the fact that a type specifier as a function identifier is a\n  constructor, and array specifiers are only allowed in this position.\n* The `NonEmpty` wrapper was removed\n* `Declaration::Global` was removed since it's parsed as an `InitDeclaratorList`\n\n#### Documentation\n\nMost items are documented (through `#[deny(missing_docs)]`) although we are\ncurrently missing some usage examples. These will come soon enough, promise!\n\n## Limitations\n\nAside from the limitations mentioned in the paragraph above:\n\n* Starting with the 0.2 release of `glsl-lang`, the `glsl-lang-pp` (also part\n  of this project) is used to preprocess the input before running the parser.\n  This means we can now parse shaders that are invalid without macro expansion,\n  but as a result we lose some preprocessing directives in the AST. Also, since\n  preprocessing directives can be inserted at any point in the token stream, we\n  may only recover those which are at the top-level, just like the `glsl` crate\n  does.\n* Currently, no semantic analysis\n\n## License\n\nThis work is licensed under the BSD 3-clause license. Lexer and LALR parser by\nAlixinne \u003calixinne@pm.me\u003e. Original AST, test suite and\nquoting code by Dimitri Sabadie \u003cdimitri.sabadie@gmail.com\u003e. glslangValidator\ntest suite from the Khronos group.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falixinne%2Fglsl-lang","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falixinne%2Fglsl-lang","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falixinne%2Fglsl-lang/lists"}