{"id":17124945,"url":"https://github.com/shellyln/takenoco","last_synced_at":"2025-04-13T06:13:15.062Z","repository":{"id":37751277,"uuid":"440500247","full_name":"shellyln/takenoco","owner":"shellyln","description":"A parser combinator library for Go.","archived":false,"fork":false,"pushed_at":"2023-04-25T10:30:38.000Z","size":119,"stargazers_count":12,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-13T06:13:04.496Z","etag":null,"topics":["go","golang","lexer","lexer-parser","parser","parser-combinator","parser-framework","parser-library","production-rules","production-rules-engine"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shellyln.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-21T11:55:13.000Z","updated_at":"2025-02-12T03:06:15.000Z","dependencies_parsed_at":"2023-02-05T04:46:04.680Z","dependency_job_id":null,"html_url":"https://github.com/shellyln/takenoco","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shellyln%2Ftakenoco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shellyln%2Ftakenoco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shellyln%2Ftakenoco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shellyln%2Ftakenoco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shellyln","download_url":"https://codeload.github.com/shellyln/takenoco/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248670436,"owners_count":21142904,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","lexer","lexer-parser","parser","parser-combinator","parser-framework","parser-library","production-rules","production-rules-engine"],"created_at":"2024-10-14T18:43:45.306Z","updated_at":"2025-04-13T06:13:15.018Z","avatar_url":"https://github.com/shellyln.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🎋 Takenoco\n\nParser Combinator Library for Go.  \nA framework for making easy use of parser combinators and production rules.\n\n[![Test](https://github.com/shellyln/takenoco/actions/workflows/test.yml/badge.svg)](https://github.com/shellyln/takenoco/actions/workflows/test.yml)\n[![release](https://img.shields.io/github/v/release/shellyln/takenoco)](https://github.com/shellyln/takenoco/releases)\n[![Go version](https://img.shields.io/github/go-mod/go-version/shellyln/takenoco)](https://github.com/shellyln/takenoco)\n\n\u003cimg src=\"https://raw.githubusercontent.com/shellyln/takenoco/master/_assets/logo-go-takenoco.svg\" alt=\"logo\" style=\"width:250px;\" width=\"250\"\u003e\n\n\n---\n\n## 🪄 Introduction\n\n### Introduction to takenoco, a parser combinator library for Go\n* [English](https://github.com/shellyln/takenoco/blob/master/_docs/introduction.md)\n* [日本語 (external link)](https://zenn.dev/shellyln/articles/a460f81fb7e1df)\n\n\n## 🧭 Examples\n\n* [CSV parser](https://github.com/shellyln/takenoco/tree/master/_examples/csv)\n* [Formula parser](https://github.com/shellyln/takenoco/tree/master/_examples/formula)\n* [Formula to RPN converter](https://github.com/shellyln/takenoco/tree/master/_examples/torpn)\n* [Loose JSON + TOML parsers](https://github.com/shellyln/go-loose-json-parser)\n    * [JSON parser](https://github.com/shellyln/go-loose-json-parser/blob/master/jsonlp/json.go)\n    * [TOML parser](https://github.com/shellyln/go-loose-json-parser/blob/master/jsonlp/toml.go)\n    * [Live demo (Loose JSON | TOML normalizer)](https://shellyln.github.io/jsonlp/)\n* [Dust - toy scripting language](https://github.com/shellyln/dust-lang)\n    * [Parsers](https://github.com/shellyln/dust-lang/tree/master/scripting/parser)\n    * [Production rules](https://github.com/shellyln/dust-lang/tree/master/scripting/rules)\n* [OpenSOQL parser](https://github.com/shellyln/go-open-soql-parser)\n    * [Parser](https://github.com/shellyln/go-open-soql-parser/blob/master/soql/parser/parser.go)\n    * [Live demo](https://shellyln.github.io/soql/)\n\n\n## 🚀 Getting started\n\n### Define the parser:\n\n```go\npackage csv\n\nimport (\n    \"errors\"\n    \"strconv\"\n\n    . \"github.com/shellyln/takenoco/base\"\n    . \"github.com/shellyln/takenoco/string\"\n)\n\nvar (\n    // Comma and line break characters\n    cellBreakCharacters []string\n    documentParser      ParserFn\n)\n\nfunc init() {\n    cellBreakCharacters = make([]string, 0, len(LineBreakCharacters)+1)\n    cellBreakCharacters = append(cellBreakCharacters, \",\")\n    cellBreakCharacters = append(cellBreakCharacters, LineBreakCharacters...)\n    documentParser = document()\n}\n\n// Remove the resulting AST.\nfunc erase(fn ParserFn) ParserFn {\n    return Trans(fn, Erase)\n}\n\n// Whitespaces\nfunc sp() ParserFn {\n    return erase(ZeroOrMoreTimes(WhitespaceNoLineBreak()))\n}\n\nfunc quotedCell() ParserFn {\n    return Trans(\n        OneOrMoreTimes(\n            FlatGroup(\n                sp(),\n                erase(Seq(\"\\\"\")),\n                ZeroOrMoreTimes(\n                    First(\n                        erase(Seq(\"\\\"\\\"\")),\n                        CharClassN(\"\\\"\"),\n                    ),\n                ),\n                First(\n                    erase(Seq(\"\\\"\")),\n                    FlatGroup(End(), Error(\"Unexpected EOF\")),\n                ),\n                sp(),\n            ),\n        ),\n        Concat,\n    )\n}\n\nfunc cell() ParserFn {\n    return Trans(\n        ZeroOrMoreTimes(CharClassN(cellBreakCharacters...)),\n        Trim,\n    )\n}\n\n// Convert AST to array data. (line)\nfunc lineTransform(_ ParserContext, asts AstSlice) (AstSlice, error) {\n    w := make([]string, len(asts))\n    length := len(asts)\n\n    for i := 0; i \u003c length; i++ {\n        w[i] = asts[i].Value.(string)\n    }\n\n    return AstSlice{{\n        ClassName: \"*Line\",\n        Type:      AstType_Any,\n        Value:     w,\n    }}, nil\n}\n\nfunc line() ParserFn {\n    return Trans(\n        FlatGroup(\n            ZeroOrMoreTimes(\n                First(quotedCell(), cell()),\n                erase(Seq(\",\")),\n            ),\n            First(quotedCell(), cell()),\n        ),\n        lineTransform,\n    )\n}\n\n// Convert AST to array data. (Entire document)\nfunc documentTransform(_ ParserContext, asts AstSlice) (AstSlice, error) {\n    length := len(asts)\n    w := make([][]string, length)\n\n    for i := 0; i \u003c length; i++ {\n        w[i] = asts[i].Value.([]string)\n    }\n    for i := length - 1; i \u003e= 0; i-- {\n        if len(w[i]) == 0 || len(w[i]) == 1 \u0026\u0026 w[i][0] == \"\" {\n            w = w[:i]\n        } else {\n            break\n        }\n    }\n\n    return AstSlice{{\n        ClassName: \"*Document\",\n        Type:      AstType_Any,\n        Value:     w,\n    }}, nil\n}\n\nfunc document() ParserFn {\n    return Trans(\n        FlatGroup(\n            ZeroOrMoreTimes(\n                line(),\n                erase(OneOrMoreTimes(LineBreak())),\n            ),\n            line(),\n            End(),\n        ),\n        documentTransform,\n    )\n}\n\nfunc Parse(s string) ([][]string, error) {\n    out, err := documentParser(*NewStringParserContext(s))\n    if err != nil {\n        return nil, err\n    } else {\n        if out.MatchStatus == MatchStatus_Matched {\n            return out.AstStack[0].Value.([][]string), nil\n        } else {\n            return nil, errors.New(\"Parse failed at \" + strconv.Itoa(out.SourcePosition.Position))\n        }\n    }\n}\n```\n\n### Use the parser:\n\n```go\npackage main\n\nimport (\n    \"fmt\"\n    \"os\"\n\n    csv \"github.com/shellyln/takenoco/_examples/csv\"\n)\n\nfunc main() {\n    x, err := csv.Parse(\"0,1,2,3,4,5,6,7,8,9\\n0,1,2,3,4,5,6,7,8,9\")\n    if err != nil {\n        fmt.Fprintln(os.Stderr, err)\n        os.Exit(-1)\n    }\n    fmt.Println(x)\n\n    y := csv.ToCsv(x)\n    fmt.Println(y)\n\n    os.Exit(0)\n}\n```\n\n[Run on go playground](https://go.dev/play/p/in5o3Ucc3b0)\n\n\n## 📦 Build the example app\n\n#### 🪟 Windows prerequirements:\n\n```bash\nchoco install make\n# or\n# scoop install make\n```\n* [https://chocolatey.org/](https://chocolatey.org/)\n* [https://scoop.sh/](https://scoop.sh/)\n\n\n### 🔹 Build to native executable\n\n```bash\nmake\n```\n\n### 🔹 Build to WebAssembly (Go)\n\n```bash\nmake wasm\n```\n\n### 🔹 Build to WebAssembly (TinyGo; experimental)\n#### 🪟 Windows prerequirements:\n\n```bash\nscoop install tinygo\nscoop install binaryen\n```\n* [https://scoop.sh/](https://scoop.sh/)\n* [https://tinygo.org/getting-started/install/windows/](https://tinygo.org/getting-started/install/windows/)\n* [https://github.com/tinygo-org/tinygo/issues/2601](https://github.com/tinygo-org/tinygo/issues/2601)\n* [https://github.com/WebAssembly/binaryen](https://github.com/WebAssembly/binaryen)\n\n#### 🐧 Linux prerequirements:\n\n* [https://tinygo.org/getting-started/install/linux/](https://tinygo.org/getting-started/install/linux/)\n\n#### 🍎 Mac prerequirements:\n\n* [https://tinygo.org/getting-started/install/macos/](https://tinygo.org/getting-started/install/macos/)\n\n#### Build:\n\n```bash\nmake tinywasm\n```\n\n\n## 🧩 Contributing\n\n#### Prerequirements:\n\n```bash\ngo install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest\ngo install honnef.co/go/tools/cmd/staticcheck@latest\n```\n\n\n## ⚖️ License\n\nMIT  \nCopyright (c) 2021 Shellyl_N and Authors.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshellyln%2Ftakenoco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshellyln%2Ftakenoco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshellyln%2Ftakenoco/lists"}