{"id":19922132,"url":"https://github.com/lhwdev/llang","last_synced_at":"2025-05-03T07:30:35.944Z","repository":{"id":119434671,"uuid":"602587943","full_name":"lhwdev/llang","owner":"lhwdev","description":"🚧 A toy project for dominating the world","archived":false,"fork":false,"pushed_at":"2025-04-17T14:18:20.000Z","size":1497,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-18T04:25:58.591Z","etag":null,"topics":["ast","kotlin","parser","programming-language"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lhwdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-16T14:28:37.000Z","updated_at":"2024-09-15T10:25:21.000Z","dependencies_parsed_at":"2024-12-15T05:22:09.656Z","dependency_job_id":"e69af637-3a50-4169-bec5-2db2cbf7c46e","html_url":"https://github.com/lhwdev/llang","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lhwdev%2Fllang","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lhwdev%2Fllang/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lhwdev%2Fllang/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lhwdev%2Fllang/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lhwdev","download_url":"https://codeload.github.com/lhwdev/llang/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252156707,"owners_count":21703329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","kotlin","parser","programming-language"],"created_at":"2024-11-12T22:09:42.207Z","updated_at":"2025-05-03T07:30:34.790Z","avatar_url":"https://github.com/lhwdev.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# llang\n\n\u003e A hard work of innocent student who haven't learned anything about parser/compiler.\n\nNote that 'llang' is alias and I really couldn't come up with cool name.\n\nAlso see [book](book.md) and [syntax reference](syntax-reference.md).\n\n**Useful Links:**\n\n- [Parser](modules/tooling/parser/src/commonMain/kotlin/com/lhwdev/llang/parser/impl/parser.kt)\n- [all tokens](modules/tooling/token/src/commonMain/kotlin/com/lhwdev/llang/token/TokenKinds.kt)\n\ncode -\u003e cst(with token) -\u003e ast -\u003e fir -\u003e ir\n\n- code\n\n  ``` kotlin\n  val a: Int = query(\"lhwdev\", 10 + 9)\n  ```\n\n- **cst**: target of code formatting. change of ast is applied here.\n  This is merely a 'more structured token list'.\n\n  Note that we merged 'tokenize' phase into 'Cst parsing phase'.\n  Parsing tokens also requires some context (although less than cst),\n  so I'm convinced that 'Why resolve same context two times? Just do\n  everything at once.' As said earlier, cst is merely 'structured token\n  list'. It's similar to the output of tokenizer, but more structured.\n\n  ``` kotlin\n  // Note that all tokens including whitespace are actually saved in CstNodes, but they are\n  // saved separately by CstParseContext.\n  CstLocalVariableDeclaration(\n      modifiers = emptyList(),\n      kind = listOf(CstLeafNode(valToken)),\n      type = listOf(CstLeafNode(colonToken)),\n      initializer = CstInitializer(\n          equals = CstLeafNode(equalsToken),\n          expression = CstCall(\n              function = CstGetValue(CstIdentifier(queryToken)),\n              valueArguments = listOf(\n                  CstTuple(\n                      open = CstLeafNode(parenOpenToken),\n                      items = listOf(CstConstant.String(\"lhwdev\"), commaToken, ...),\n                      close = CstLeafNode(parenCloseToken),\n                  )\n              )\n          )\n      ),\n  )\n  \n  // alternative declaration(including whitespace):\n  CstLocalVariableDeclaration(\n      modifiers = emptyList(),\n      kind = listOf(valToken, whitespace),\n      type = listOf(colonToken, whitespace2),\n      initializer = listOf(\n          whitespace3, equalsToken, whitespace4,\n          CstCall(\n              function = CstGetValue(CstIdentifier(queryToken)),\n              valueArguments = listOf(\n                  CstTuple(\n                      open = parenOpenToken,\n                      content = listOf(CstConstant.String(\"lhwdev\"), commaToken, ...),\n                      close = parenCloseToken,\n                  )\n              )\n          )\n      ),\n  )\n  ```\n\n- **ast**: target of code refactoring. directly linked to fir.\n\n  ``` kotlin\n  AstLocalVariableDeclaration(\n      isVar = false,\n      modifier = listOf(),\n      name = Identifier(\"a\"), // (such a thing like this is stub)\n      declaredType = AstTypeReference(AstClassifierReference(\"Int\")),\n      initializer = AstCall(\n          function = AstGetValue(AstReference(\"query\")),\n          typeArguments = emptyList(),\n          valueArguments = listOf(\n              AstConst.String(\"lhwdev\"),\n              AstBinaryOps(AstBinaryOperator.Plus, AstConst.IntegerKind(10), AstConst.IntegerKind(9)),\n          ),\n      ),\n  ```\n\n- **fir**: target of semantic analysis and diagnostics.\n  ast -\u003e fir is called 'semantic analysis'.\n\n  Fir should be fully analyzed, but ast can be partially replaced with firs, which\n  means if you need semantically partially-analyzed ast, just get ast and get fir\n  where you really need it.\n\n  ``` kotlin\n  FirLocalVariable( // ...,\n      name = FirVariableSymbol(\"a\", ...),\n      type = FirType(FirClassifier(intSymbol), typeArguments = emptyList()),\n      initializer = FirCall(\n          function = FirGetValue(querySymbol),\n          typeArguments = emptyList(),\n          valueArguments = listOf(\n              FirConst.String(\"lhwdev\"),\n              FirBinaryOps(FirCall(\n                  function = FirGetValue(intPlusSymbol),\n                  receiver = intSymbol,\n                  typeArguments = emptyList(),\n                  valueArguments = listOf(FirConst.Int(10), FirConst.Int(9)),\n              )),\n          )\n      )\n  ```\n\n- **ir**: target of backend compilation and optimization. will be converted into\n  some kind of binary code, or something cool.\n\n  ``` kotlin\n  IrLocalVariable( // ...,\n      name = IrVariableSymbol(\"a\", ...),\n      type = IrType(IrClassifier(intSymbol), typeArguments = emptyList()),\n      initializer = IrCall(\n          function = IrGetValue(querySymbol),\n          typeArguments = emptyList(),\n          valueArguments = listOf(\n              IrConst.String(\"lhwdev\"),\n              IrCall(\n                  function = IrGetValue(intPlusSymbol),\n                  receiver = intSymbol,\n                  typeArguments = emptyList(),\n                  valueArguments = listOf(IrConst.Int(10), IrConst.Int(9)),\n                  origin = IrCallOrigin.PlusOperator,\n              ),\n          ),\n      ),\n  )\n  ```\n\n## Basic Architecture\n\n### Reference\n\n- **cst --\u003e token** as cst itself is full of token.\n- **ast --\u003e cst** for reconstructing code from ast.\n- **ast --(resolver)--\u003e fir** in diagnostics or intermediate frontend compiler plugin (lazy)\n- **fir --\u003e ast** in code refactoring to apply code changes\n- **ir --\u003e fir** for knowing declaration structure from IR  \n  Note that fir is used to declare structures of declaration, like Descriptor in Kotlin IR.\n- **IrSymbol --\u003e IrDeclarationStructure**, but following IrDeclaration\n  may be a stub, which does not have bodies. You can get bodies of any valid symbols\n  in linking stage later.\n\n### Operations\n\n- **Parsing**: **code string** --(lexing)--\u003e **token** --(parsing)--\u003e **cst** --(semantic\n  parsing) --\u003e **fir**\n- **IDE/Code Highlighting**: ast -\u003e annotations\n- **IDE/Code Modification**: edited code -\u003e (incremental) token -\u003e ... (same as parsing)\n- **Code Refactoring**: modify fir then apply to fir -\u003e ast -\u003e cst -\u003e token\n- **Diagnostics**: inspect ast / fir\n- **Diagnostics Fix**: modify following ast / fir\n\n## Incremental Parsing\n\nOops, see [book.md](book.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flhwdev%2Fllang","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flhwdev%2Fllang","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flhwdev%2Fllang/lists"}