{"id":20864873,"url":"https://github.com/lezgomatt/lpeglabel-gsoc-2016","last_synced_at":"2026-01-31T05:32:45.940Z","repository":{"id":144351873,"uuid":"66145389","full_name":"lezgomatt/lpeglabel-gsoc-2016","owner":"lezgomatt","description":"Summary of my GSoC 2016 project and experience","archived":false,"fork":false,"pushed_at":"2020-10-24T15:54:49.000Z","size":469,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-11T16:26:00.996Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lezgomatt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-20T12:23:32.000Z","updated_at":"2020-10-24T15:54:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"d5390b32-1687-4765-83c5-55131c09b414","html_url":"https://github.com/lezgomatt/lpeglabel-gsoc-2016","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lezgomatt/lpeglabel-gsoc-2016","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lezgomatt%2Flpeglabel-gsoc-2016","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lezgomatt%2Flpeglabel-gsoc-2016/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lezgomatt%2Flpeglabel-gsoc-2016/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lezgomatt%2Flpeglabel-gsoc-2016/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lezgomatt","download_url":"https://codeload.github.com/lezgomatt/lpeglabel-gsoc-2016/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lezgomatt%2Flpeglabel-gsoc-2016/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28930381,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-31T04:05:25.756Z","status":"ssl_error","status_checked_at":"2026-01-31T04:02:35.005Z","response_time":128,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T05:44:49.959Z","updated_at":"2026-01-31T05:32:45.921Z","avatar_url":"https://github.com/lezgomatt.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"This page talks about my\n[Google Summer of Code (GSoC) project](https://summerofcode.withgoogle.com/projects/#5054385788813312)\nand experience during the summer of 2016. I was fortunate enough\nto be accepted by [LabLua](http://www.lua.inf.puc-rio.br/index.html),\nand help them test out the idea of using labeled failures to improve\nerror messages in PEG-based parsers.\n\nYou can also read LabLua's GSoC 2016 wrap-up\n[here](http://www.lua.inf.puc-rio.br/gsoc/blog2016.html)\nor on\n[Google's Open Source Blog](https://opensource.googleblog.com/2017/02/google-summer-of-code-2016-wrap-up_8.html).\n\n\n# Introduction\n\n[Parsing Expression Grammars (PEGs)](https://en.wikipedia.org/wiki/Parsing_expression_grammar)\nare an expressive formalism for designing and implementing\ntop-down parsers with local backtracking. PEGs are an alternative to\n[Context Free Grammars (CFGs)](https://en.wikipedia.org/wiki/Context-free_grammar),\nwhich have a similar (but different) level of expressiveness.\n\nAn issue that users of PEG-based parsers face is poor reporting of\nsyntax errors on the part of PEG-based parsers.\n[Labeled failures](http://arxiv.org/abs/1405.6646)\nare an extension to PEGs that aims to\naddress this issue by annotating a PEG with labels corresponding to\nsyntax errors, improving the quality of error messages generated by\na PEG-based parser.\n\n[LPegLabel](https://github.com/sqmedeiros/lpeglabel) is an extension of\nthe [LPeg](http://www.inf.puc-rio.br/~roberto/lpeg/) that provides\nan implementation of PEGs with labeled failures. Labels can be used to\nsignal different kinds of errors and to specify which alternative in\na labeled ordered choice should handle a given label, which is useful for\nimplementing error recovery.\n\n\n# Goal\n\nThe goal of this project was to rewrite the (LPeg) parsers of some\nLua libraries, such as the module re from LPeg (actually it was\nLPegLabel's version of the module called relabel that was modified)\nand lua-parser, by using LPegLabel. By rewriting them, we are able to\nget parsers with better error messages.\n\nIn the end, three parsers were rewritten for this project:\n* [relabel](#relabel)\n* [Lapis](#lapis)\n* [lua-parser](#lua-parser)\n\n\n# relabel\nRelabel is [LPegLabel](https://github.com/sqmedeiros/lpeglabel)'s\nextension to LPeg's [re module](http://www.inf.puc-rio.br/~roberto/lpeg/re.html),\nwhich provides a regular expression (a.k.a. regex)\ninspired syntax to define PEG patterns and grammars.\n\nIn total, 30 labels were introduced to the parser. Annotating the\ngrammars took some thinking at first, but after a while it becomes\neasier and you build an intuition. Thankfully, I was given a\nshort exercise activity on it before tackling the project itself.\nI also managed to trigger a bug during the rewrite, which was\nquickly fixed by my mentor.\n\nOne of the most challenging things for this project was implementing\nerror recovery. Error recovery is something very tricky and hard,\nif not impossible, to perfect. Aside from the time taken to think up\nof good strategies to employ, I also took quite some time figuring out\nhow to properly use labeled choices (Lc) and match-time captures (Cmt)\nin implementing the different error recovery strategies.\n\nThe changes made have already been integrated in the main repository on\nGitHub as can be seen in\n[the commit log here](https://github.com/sqmedeiros/lpeglabel/commits/master?author=undecidabot).\nThe branch implementing error recovery can be found on\n[a separate branch on the same repository](https://github.com/sqmedeiros/lpeglabel/commits/relabel-recovery?author=undecidabot).\nA report detailing the changes is\n[available here](https://github.com/undecidabot/lpeglabel-gsoc-2016/raw/master/relabel.pdf).\n\nA sample of the new error messages is shown below:\n```\nL1:C17: missing closing ')'\n('p' ('q' / 'r')\n                ^\n```\n\n\n# Lapis\n[Lapis](https://github.com/leafo/lapis) is a web framework written\nin [MoonScript](http://moonscript.org/), a language which compiles\ninto Lua. Lapis was chosen since it was a fairly popular project and\nbecause the author was open to the idea of merging it, which would help\ngive us a better idea of how well it works in the practice.\n\nThe PEGs in Lapis are quite small, but essential! Lapis uses LPeg\nto parse the URL patterns and also some other minor things like a subset\nof SQL. For this rewrite, those two parsers were rewritten\nand error recovery was not implemented since the expected input is\nshort enough not to need it.\n\nIn total, 12 labels were introduced for the two parsers.\nThe changes can be viewed in [this pull request](https://github.com/leafo/lapis/pull/445)\nsent in the middle of June and also\n[this short report](https://github.com/undecidabot/lpeglabel-gsoc-2016/raw/master/lapis.pdf)\ndetailing the changes made (including examples for the different\nlabels / errors introduced). Unfortunately, the author seems to be\npreoccupied with other matters and has not had the time to review and\nmerge the pull request.\n\n\n# lua-parser\n[Lua-parser](https://github.com/andremm/lua-parser) is a complete\nLua 5.3 parser written with LPeg. Among the three projects, this one\nhad the biggest and most complex grammar. In fact, LPegLabel had to be\nupdated to be able to throw all the labels (LPegLabel originally only\nsupported up to 64 different labels; now it supports up to 255 labels).\nUnsurprisingly, this also took the most time to complete.\n\nIn total, 75 labels were introduced to the parser. Like relabel,\nerror recovery was also implemented for this project. Most of the time,\nthe parser would simply skip the line after encountering an error.\nHowever, to be honest, I'm not fully satisfied with it yet. It annoys me\nthat recovery can sometimes introduce false or misleading errors,\nbut it really is a challenging problem to solve. Even well-known\ncompilers like `gcc` and `javac` have not perfected this (try mispelling\n\"for\" as \"fro\" to see what I mean).\n\nThe pull request for the changes can be\n[found here](https://github.com/andremm/lua-parser/pull/3),\nand a branch implementing error recovery can be\n[found here](https://github.com/undecidabot/lua-parser/tree/error-recovery).\nA copy of the report detailing the changes is also\n[available here](https://github.com/undecidabot/lpeglabel-gsoc-2016/raw/master/lua-parser.pdf).\n\nA sample of the difference in error reporting is shown below.\n\nExample code with syntax error:\n```\nfor i=1,10, do end\n```\n\nOriginal error message:\n```\ntest.lua:1:13: syntax error, unexpected 'do', expecting '(', 'Name', '{', 'function', '...', 'true', 'false', 'nil', 'String', 'Number', '~', '#', '-', 'not'\n```\n\nNew error message:\n```\ntest.lua:1:13: syntax error, expected a step expression for the numeric range after ','\n```\n\nIn addition to the labels and recovery, some refactoring and clean-up\nwork was also done. The grammar now follows an arguably more readable\ntop-down order instead of the original bottom-up order. Some additions\nand bug fixing were also done in the handling of escape codes.\n\n\n# Pegastify\nWhile working on the documentation of lua-parser, I came up with a\ntool called [pegastify](https://github.com/undecidabot/pegastify)\n(name subject to change in the future) to help me understand the\ncode describing the grammar of Lua. Pegastify converts LPeg code\ninto an abstract syntax tree (AST) of the PEG, which can be pretty\nprinted following the re syntax.\n\nAlthough the tool still needs more work (especially in tests,\ndocumentation, some additional features, and clean-up), it was more than\nenough in helping document and understand the grammar. It also gave me\na taste of lua-parser since it uses lua-parser to parse the code!\nHopefully, I'll find time soon to make it usable by others.\n\n\n# My Experience\nOverall I've had a really great experience in this summer of code.\nI was very happy to be able to work for an organization focused\non programming languages, my favorite topic in computer science.\n\nFor the first time, I was able to experience what it really is like\nto work with others on an open source project. I was able to collaborate\nwith other people, whom I am unfortunately unable to meet in person.\nWorking at home also made me realize how much discipline one needs\nto get stuff done.\n\nI was also able to develop a better appreciation of documentation and\ntesting, which I never developed in my school projects since they were\nmostly small and not meant to be maintained. Thankfully, the code of\nthe projects I worked on were readable and had a comprehensive test suite.\nAfter learning a few idioms of Lua, I had no problem in editing the code\nand could do so without fear of breaking things.\n\n\n# Conclusion\nLPegLabel is still quite young, but the this project already\nproves that it has the potential to greatly improve error messages.\nWith a little more exposure and tests in the real world, we can\nfind out how else it can improve. Perhaps, someday it will even be\nmerged with LPeg.\n\nI'd like to end by thanking\n[Professor Sérgio Medeiros, PhD](https://sites.google.com/site/sqmedeiros/)\nfor being a very responsive, helpful, understanding, and invaluable mentor\nthroughout the whole summer of code. I would also like to thank\nLabLua and Google for giving me the opportunity to work on this project\n(and also the monetary benefits of course hehe).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flezgomatt%2Flpeglabel-gsoc-2016","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flezgomatt%2Flpeglabel-gsoc-2016","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flezgomatt%2Flpeglabel-gsoc-2016/lists"}