{"id":20508855,"url":"https://github.com/loloicci/nimly","last_synced_at":"2025-04-09T16:20:39.339Z","repository":{"id":43734355,"uuid":"89258678","full_name":"loloicci/nimly","owner":"loloicci","description":"Lexer Generator and Parser Generator as a Library in Nim.","archived":false,"fork":false,"pushed_at":"2022-06-10T09:42:54.000Z","size":215,"stargazers_count":147,"open_issues_count":17,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-23T18:37:30.832Z","etag":null,"topics":["bnf","compile-time","ebnf","lexer-generator","lexer-parser","macro","macros","nim","parser-generator"],"latest_commit_sha":null,"homepage":"","language":"Nim","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/loloicci.png","metadata":{"files":{"readme":"README.rst","changelog":"changelog.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-24T15:45:37.000Z","updated_at":"2025-03-13T19:11:43.000Z","dependencies_parsed_at":"2022-08-21T21:40:14.858Z","dependency_job_id":null,"html_url":"https://github.com/loloicci/nimly","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loloicci%2Fnimly","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loloicci%2Fnimly/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loloicci%2Fnimly/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loloicci%2Fnimly/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/loloicci","download_url":"https://codeload.github.com/loloicci/nimly/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248065285,"owners_count":21041872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bnf","compile-time","ebnf","lexer-generator","lexer-parser","macro","macros","nim","parser-generator"],"created_at":"2024-11-15T20:20:48.406Z","updated_at":"2025-04-09T16:20:39.304Z","avatar_url":"https://github.com/loloicci.png","language":"Nim","funding_links":[],"categories":[],"sub_categories":[],"readme":"#######\n nimly\n#######\n|github_workflow| |nimble|\n\nLexer Generator and Parser Generator as a Macro Library in Nim.\n\nWith nimly, you can make lexer/parser by writing definition\nin formats like lex/yacc.\n``nimly`` generates lexer and parser by using macro in compile-time,\nso you can use ``nimly`` not as external tool of your program but as a library.\n\nniml\n====\n``niml`` is a macro to generate a lexer.\n\nmacro niml\n----------\nmacro ``niml`` makes a lexer.\nAlmost all part of constructing a lexer is done in compile-time.\nExample is as follows.\n\n.. code-block:: nim\n\n  ## This makes a LexData object named myLexer.\n  ## This lexer returns value with type ``Token`` when a token is found.\n  niml myLexer[Token]:\n    r\"if\":\n      ## this part converted to procbody.\n      ## the arg is (token: LToken).\n      return TokenIf()\n    r\"else\":\n      return TokenElse()\n    r\"true\":\n      return TokenTrue()\n    r\"false\":\n      return TokenFalse()\n    ## you can use ``..`` instead of ``-`` in ``[]``.\n    r\"[a..zA..Z\\-_][a..zA..Z0..9\\-_]*\":\n      return TokenIdentifier(token)\n    ## you can define ``setUp`` and ``tearDown`` function.\n    ## ``setUp`` is called from ``open``, ``newWithString`` and\n    ## ``initWithString``.\n    ## ``tearDown`` is called from ``close``.\n    ## an example is ``test/lexer_global_var.nim``.\n    setUp:\n      doSomething()\n    tearDown:\n      doSomething()\n\nMeta charactors are as following:\n\n- ``\\``: escape character\n- ``.``: match with any charactor\n- ``[``: start of character class\n- ``|``: means or\n- ``(``: start of subpattern\n- ``)``: end of subpattern\n- ``?``: 0 or 1 times quantifier\n- ``*``: 0 or more times quantifire\n- ``+``: 1 or more times quantifire\n- ``{``: ``{n,m}`` is n or more and m or less times quantifire\n\nIn ``[]``, meta charactors are as following\n\n- ``\\``: escape character\n- ``^``: negate character (only in first position)\n- ``]``: end of this class\n- ``-``: specify character range (``..`` can be used instead of this)\n\nEach of followings is recognized as character set.\n\n- ``\\d``: ``[0..9]``\n- ``\\D``: ``[^0..9]``\n- ``\\s``: ``[ \\t\\n\\r\\f\\v]``\n- ``\\S``: ``[^ \\t\\n\\r\\f\\v]``\n- ``\\w``: ``[a..zA..Z0..9_]``\n- ``\\w``: ``[^a..zA..Z0..9_]``\n\nnimy\n====\n``nimy`` is a macro to generate a LALR(1) parser.\n\nmacro nimy\n----------\nmacro ``nimy`` makes a parser.\nAlmost all part of constructing a parser is done in compile-time.\nExample is as follows.\n\n.. code-block:: nim\n\n  ## This makes a LexData object named myParser.\n  ## first cloud is the top-level of the BNF.\n  ## This lexer recieve tokens with type ``Token`` and token must have a value\n  ## ``kind`` with type enum ``[TokenTypeName]Kind``.\n  ## This is naturally satisfied when you use ``patty`` to define the token.\n  nimy myParser[Token]:\n    ## the starting non-terminal\n    ## the return type of the parser is ``Expr``\n    top[Expr]:\n      ## a pattern.\n      expr:\n        ## proc body that is used when parse the pattern with single ``expr``.\n        ## $1 means first position of the pattern (expr)\n        return $1\n\n    ## non-terminal named ``expr``\n    ## with returning type ``Expr``\n    expr[Expr]:\n      ## first pattern of expr.\n      ## ``LPAR`` and ``RPAR`` is TokenKind.\n      LPAR expr RPAR:\n        return $2\n\n      ## second pattern of expr.\n      ## ``PLUS`` is TokenKind.\n      expr PLUS expr\n        return $2\n\nYou can use following EBNF functions:\n\n- ``XXX[]``: Option (0 or 1 ``XXX``).\n  The type is ``seq[xxx]`` where ``xxx`` is type of ``XXX``.\n- ``XXX{}``: Repeat (0 or more ``XXX``).\n  The type is ``seq[xxx]`` where ``xxx`` is type of ``XXX``.\n\nExample of these is in next section.\n\nExample\n=======\n``tests/test_readme_example.nim`` is an easy example.\n\n.. code-block:: nim\n\n  import unittest\n  import patty\n  import strutils\n\n  import nimly\n\n  ## variant is defined in patty\n  variant MyToken:\n    PLUS\n    MULTI\n    NUM(val: int)\n    DOT\n    LPAREN\n    RPAREN\n    IGNORE\n\n  niml testLex[MyToken]:\n    r\"\\(\":\n      return LPAREN()\n    r\"\\)\":\n      return RPAREN()\n    r\"\\+\":\n      return PLUS()\n    r\"\\*\":\n      return MULTI()\n    r\"\\d\":\n      return NUM(parseInt(token.token))\n    r\"\\.\":\n      return DOT()\n    r\"\\s\":\n      return IGNORE()\n\n  nimy testPar[MyToken]:\n    top[string]:\n      plus:\n        return $1\n\n    plus[string]:\n      mult PLUS plus:\n        return $1 \u0026 \" + \" \u0026 $3\n\n      mult:\n        return $1\n\n    mult[string]:\n      num MULTI mult:\n        return \"[\" \u0026 $1 \u0026 \" * \" \u0026 $3 \u0026 \"]\"\n\n      num:\n        return $1\n\n    num[string]:\n      LPAREN plus RPAREN:\n        return \"(\" \u0026 $2 \u0026 \")\"\n\n      ## float (integer part is 0-9) or integer\n      NUM DOT[] NUM{}:\n        result = \"\"\n        # type of `($1).val` is `int`\n        result \u0026= $(($1).val)\n        if ($2).len \u003e 0:\n          result \u0026= \".\"\n        # type of `$3` is `seq[MyToken]` and each elements are NUM\n        for tkn in $3:\n          # type of `tkn.val` is `int`\n          result \u0026= $(tkn.val)\n\n  test \"test Lexer\":\n    var testLexer = testLex.newWithString(\"1 + 42 * 101010\")\n    testLexer.ignoreIf = proc(r: MyToken): bool = r.kind == MyTokenKind.IGNORE\n\n    var\n      ret: seq[MyTokenKind] = @[]\n\n    for token in testLexer.lexIter:\n      ret.add(token.kind)\n\n    check ret == @[MyTokenKind.NUM, MyTokenKind.PLUS, MyTokenKind.NUM,\n                   MyTokenKind.NUM, MyTokenKind.MULTI,\n                   MyTokenKind.NUM, MyTokenKind.NUM, MyTokenKind.NUM,\n                   MyTokenKind.NUM, MyTokenKind.NUM, MyTokenKind.NUM]\n\n  test \"test Parser 1\":\n    var testLexer = testLex.newWithString(\"1 + 42 * 101010\")\n    testLexer.ignoreIf = proc(r: MyToken): bool = r.kind == MyTokenKind.IGNORE\n\n    var parser = testPar.newParser()\n    check parser.parse(testLexer) == \"1 + [42 * 101010]\"\n\n    testLexer.initWithString(\"1 + 42 * 1010\")\n\n    parser.init()\n    check parser.parse(testLexer) == \"1 + [42 * 1010]\"\n\n  test \"test Parser 2\":\n    var testLexer = testLex.newWithString(\"1 + 42 * 1.01010\")\n    testLexer.ignoreIf = proc(r: MyToken): bool = r.kind == MyTokenKind.IGNORE\n\n    var parser = testPar.newParser()\n    check parser.parse(testLexer) == \"1 + [42 * 1.01010]\"\n\n    testLexer.initWithString(\"1. + 4.2 * 101010\")\n\n    parser.init()\n    check parser.parse(testLexer) == \"1. + [4.2 * 101010]\"\n\n  test \"test Parser 3\":\n    var testLexer = testLex.newWithString(\"(1 + 42) * 1.01010\")\n    testLexer.ignoreIf = proc(r: MyToken): bool = r.kind == MyTokenKind.IGNORE\n\n    var parser = testPar.newParser()\n    check parser.parse(testLexer) == \"[(1 + 42) * 1.01010]\"\n\nInstall\n=======\n1. ``nimble install nimly``\n\nNow, you can use nimly with ``import nimly``.\n\nvmdef.MaxLoopIterations Problem\n-------------------------------\nDuring compiling lexer/parser, you can encounter errors with ``interpretation requires too many iterations``.\nYou can avoid this error to use the compiler option ``maxLoopIterationsVM:N``\nwhich is available since nim v1.0.6.\n\nSee https://github.com/loloicci/nimly/issues/11 to detail.\n\nContribute\n==========\n1. Fork this\n2. Create new branch\n3. Commit your change\n4. Push it to the branch\n5. Create new pull request\n\nChangelog\n=========\nSee changelog.rst_.\n\nDeveloping\n==========\nYou can use ``nimldebug`` and ``nimydebug`` as a conditional symbol\nto print debug info.\n\nexample: ``nim c -d:nimldebug -d:nimydebug -r tests/test_readme_example.nim``\n\n\n.. |github_workflow| image:: https://github.com/loloicci/nimly/workflows/test/badge.svg\n    :target: https://github.com/loloicci/nimly/actions?query=workflow%3Atest\n.. |nimble| image:: https://raw.githubusercontent.com/yglukhov/nimble-tag/master/nimble.png\n    :target: https://github.com/yglukhov/nimble-tag\n.. _changelog.rst: ./changelog.rst\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floloicci%2Fnimly","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Floloicci%2Fnimly","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floloicci%2Fnimly/lists"}