{"id":15399461,"url":"https://github.com/chai2010/calculator","last_synced_at":"2025-04-15T10:11:09.221Z","repository":{"id":57507148,"uuid":"170536667","full_name":"chai2010/calculator","owner":"chai2010","description":"基于flex\u0026goyacc实现的计算器","archived":false,"fork":false,"pushed_at":"2024-07-22T13:39:11.000Z","size":64,"stargazers_count":29,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-06T07:37:23.392Z","etag":null,"topics":["bison","cgo","flex","go","golang","goyacc"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chai2010.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-13T16:03:40.000Z","updated_at":"2025-01-11T13:41:30.000Z","dependencies_parsed_at":"2024-10-21T12:06:23.891Z","dependency_job_id":null,"html_url":"https://github.com/chai2010/calculator","commit_stats":{"total_commits":18,"total_committers":1,"mean_commits":18.0,"dds":0.0,"last_synced_commit":"efbed55c186d734dbb7cc13245b856e97b76f728"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chai2010%2Fcalculator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chai2010%2Fcalculator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chai2010%2Fcalculator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chai2010%2Fcalculator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chai2010","download_url":"https://codeload.github.com/chai2010/calculator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249048738,"owners_count":21204306,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bison","cgo","flex","go","golang","goyacc"],"created_at":"2024-10-01T15:48:58.623Z","updated_at":"2025-04-15T10:11:09.202Z","avatar_url":"https://github.com/chai2010.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"- *凹语言(凹读音“Wa”)(The Wa Programming Language): https://github.com/wa-lang/wa*\n\n----\n\n# 基于flex\u0026goyacc实现的计算器\n\n特性简介：\n\n- 支持整数四则运算\n- 支持小括弧提升优先级\n- 支持临时变量保存结果\n\n安装和使用(需要有GCC环境)：\n\n```shell\n$ go get github.com/chai2010/calculator\n$ calculator\n1+2*3\n= 7\nx=3-(2-1)\n= 2\nx*2\n= 4\n```\n\n## 词法符号\n\n先创建`tok.h`文件，包含词法符号：\n\n```c\nenum {\n\tILLEGAL = 10000,\n\tEOL = 10001,\n\n\tID = 258,\n\tNUMBER = 259,\n\n\tADD = 260, // +\n\tSUB = 261, // -\n\tMUL = 262, // *\n\tDIV = 263, // /\n\tABS = 264, // |\n\n\tLPAREN = 265, // (\n\tRPAREN = 266, // )\n\tASSIGN = 267, // =\n};\n```\n\n其中`ILLEGAL`表示不能识别的无效的符号，`EOL`表示行的结尾，其它的符号也字面含义相同。\n\n## 词法解析\n\n然后创建`calc.l`文件，定义每种词法的正则表达式：\n\n```lex\n%option noyywrap\n\n%{\n#include \"tok.h\"\n%}\n\n%%\n\n[_a-zA-Z]+ { return ID; }\n[0-9]+     { return NUMBER; }\n\n\"+\"    { return ADD; }\n\"-\"    { return SUB; }\n\"*\"    { return MUL; }\n\"/\"    { return DIV; }\n\"|\"    { return ABS; }\n\n\"(\"    { return LPAREN; }\n\")\"    { return RPAREN; }\n\"=\"    { return ASSIGN; }\n\n\\n     { return EOL; }\n[ \\t]  { /* ignore whitespace */ }\n.      { return ILLEGAL; }\n\n%%\n```\n\n最开始的`noyywrap`选项表示关闭`yywrap`特性，也就是去掉对flex库的依赖，生成可移植的词法分析器代码。然后在`%{`和`%}`中间是原生的C语言代码，通过包含`tok.h`引入了每种记号对应的枚举类型。在两组`%%`中间的部分是每种记号对应的正则表达式，先出现的优先匹配，如果匹配失败则继续尝试后面的规则。每个正则表达式后面跟着一组动作代码，也就是普通的C语言代码，这里都是返回记号的类型。\n\n然后通过flex工具生成C语言词法解析器文件：\n\n```shell\n$ flex --prefix=yy --header-file=calc.lex.h -o calc.lex.c calc.l\n```\n\n其中`--prefix`表示生成的代码中标识符都是以`yy`前缀。在一个项目有多个flex生成代码时，可通过前缀区分。`--header-file`表示生成头问题，这样方便在其它代码中引用生成的词法分析函数。`-o`指定输出源代码文件的名字。\n\n生成的词法分析器中，最重要的有以下几个：\n\n```c\nextern int yylineno;\nextern char *yytext;\n\nextern int yylex (void);\n```\n\n其中`yylineno`表示当前的行号，`yytext`表示当前记号对应的字符串。而`yylex`函数每次从标准输入读取一个记号，返回记号类型的值（在`tok.h`文件定义），如果遇到文件结尾则返回0。\n\n如果需要从字符串解析，则需使用以下的导出函数：\n\n```c\nYY_BUFFER_STATE yy_scan_bytes (yyconst char *bytes,yy_size_t len  );\n```\n\n通过`yy_scan_bytes`函数，可以设置字符串作为要解析的目标，然后每次调用`yylex`函数就会从字符串读取数据。这些函数都在`calc.lex.h`文件中声明。\n\n## 将C语言词法分析器包装为Go函数\n\n创建`lex.go`文件，内容如下：\n\n```go\npackage main\n\n//#include \"tok.h\"\n//#include \"calc.lex.h\"\nimport \"C\"\n\ntype calcLex struct {}\n\nfunc newCalcLexer(data []byte) *calcLex {\n\tp := new(calcLex)\n\tC.yy_scan_bytes((*C.char)(C.CBytes(data)), C.yy_size_t(len(data)))\n\treturn p\n}\n\nfunc (p *calcLex) Lex(yylval *calcSymType) int {\n\tvar tok = C.yylex()\n\tvar yylineno = int(C.yylineno)\n\tvar yytext = C.GoString(C.yytext)\n\n\tswitch tok {\n\tcase C.ID:\n\t\t// yylval.id = yytext\n\t\treturn ID\n\n\tcase C.NUMBER:\n\t\t//yylval.value, _ = strconv.Atoi(yytext)\n\t\treturn NUMBER\n\n\tcase C.ADD:\n\t\treturn ADD\n\t// ...\n\n\tcase C.EOL:\n\t\treturn EOL\n\t}\n\n\tif tok == C.ILLEGAL {\n\t\tlog.Printf(\"lex: ILLEGAL token, yytext = %q, yylineno = %d\", yytext, yylineno)\n\t}\n\n\treturn 0 // eof\n}\n```\n\n新建的`calcLex`类型对应Go语言版本的词法分析器，底层工作通过CGO调用flex生成的C语言函数完成。首先`newCalcLexer`创建一个词法分析器，参数是要分析的数据，通过`C.yy_scan_bytes`函数调用表示从字符串解析记号。然后`calcLex`类型的`Lex`方法表示每次需要解析一个记号（暂时忽略方法的`calcSymType`参数），内部通过调用`C.yylex()`读取一个记号，同时记录行号和记号对应的字符串。最后将C语言的记号转为Go语言的记号值返回，比如`C.ID`对应Go语言的`ID`。\n\n对应`ID`类型，`yytext`表示变量的名字。对于`NUMBER`类型，`yytext`保护数字对应的字符串，可以从字符串解析出数值。但是，Go语言的词法分析器如何返回变量的名字或者是数字的值呢？答案是通过`Lex`的`*calcSymType`类型的参数可以记录记号额外的属性值。而`calcSymType`类型是由`goyacc`工具生成的代码，在下面我们将介绍yacc的内容。\n\n## `goyacc`生成语法解析器\n\n`goyacc`是Go语言版本的yacc工具，是由Go语言官方团队维护的扩展包工具。\n\n创建`calc.y`文件：\n\n```yacc\n%{\npackage main\n\nvar idValueMap = map[string]int{}\n%}\n\n%union {\n\tvalue int\n\tid    string\n}\n\n%type  \u003cvalue\u003e exp factor term\n%token \u003cvalue\u003e NUMBER\n%token \u003cid\u003e    ID\n\n%token ADD SUB MUL DIV ABS\n%token LPAREN RPAREN ASSIGN\n%token EOL\n\n%%\ncalclist\n\t: // nothing\n\t| calclist exp EOL {\n\t\tidValueMap[\"_\"] = $2\n\t\tfmt.Printf(\"= %v\\n\", $2)\n\t}\n\t| calclist ID ASSIGN exp EOL {\n\t\tidValueMap[\"_\"] = $4\n\t\tidValueMap[$2] = $4\n\t\tfmt.Printf(\"= %v\\n\", $4)\n\t}\n\t;\n\nexp\n\t: factor         { $$ = $1 }\n\t| exp ADD factor { $$ = $1 + $3 }\n\t| exp SUB factor { $$ = $1 - $3 }\n\t;\n\nfactor\n\t: term            { $$ = $1 }\n\t| factor MUL term { $$ = $1 * $3 }\n\t| factor DIV term { $$ = $1 / $3 }\n\t;\n\nterm\n\t: NUMBER            { $$ = $1 }\n\t| ID                { $$ = idValueMap[$1] }\n\t;\n\n%%\n```\n\n和flex工具类型，首先在`%{`和`%}`中间是原生的Go语言代码。然后`%union`定义了属性值，用于记录语法解析中每个规则额外的属性值。通过`%type`定义BNF规则中非终结的名字，`%token`定义终结记号名字（和flex定义的记号类型是一致的）。而`%type`和`%token`就可以通过`\u003cvalue\u003e`或`\u003cid\u003e`的可选语法，将后面的名字绑定到属性。就是后续代码中`$$`对应的属性，比如`%token \u003cid\u003e ID`表示`ID`对应的属性为`id`，因此在后面的`ID { $$ = idValueMap[$1] }`表示数值`id`属性的值，其中`idValueMap`用于管理变量的值。\n\n然后通过goyacc工具生成代码：\n\n```shell\n$ goyacc -o calc.y.go -p \"calc\" calc.y\n```\n\n其中`-o`指定输出的文件名，`-p`指定标识符名字前缀（和flex的`--prefix`用法类似）。在生成的`calc.y.go`文件中将包含最重要的`calcParse`函数，该函数从指定的词法解析器中读取词法，然后进行语法分析。同时将包含`calcSymType`类型的定义，它是`Lex`词法函数的输出参数的类型。\n\n在绑定了属性之后，还需要继续完善`Lex`词法函数的代码：\n\n```go\nfunc (p *calcLex) Lex(yylval *calcSymType) int {\n\tvar tok = C.yylex()\n\tvar yylineno = int(C.yylineno)\n\tvar yytext = C.GoString(C.yytext)\n\n\tswitch tok {\n\tcase C.ID:\n\t\tyylval.id = yytext\n\t\treturn ID\n\n\tcase C.NUMBER:\n\t\tyylval.value, _ = strconv.Atoi(yytext)\n\t\treturn NUMBER\n\n\t...\n}\n```\n\n其中`yylval.id = yytext`表示词法将解析得到的变量名字填充到`id`属性中。而数字部分则是通过`yylval.value`属性保存。\n\n\n## 运行计算器\n\n创建main函数：\n\n```go\nfunc main() {\n\tcalcParse(newCalcLexer([]byte(\"1+2*3\")))\n}\n```\n\n`newCalcLexer`构造一个词法解析器，然后`calcParse`语法解析器将从词法解析器依次读取记号并解析语法，在解析语法的同时将进行表达式求值运算，同时更新`idValueMap`全局的变量。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchai2010%2Fcalculator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchai2010%2Fcalculator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchai2010%2Fcalculator/lists"}