https://github.com/alex/rply

An attempt to port David Beazley's PLY to RPython, and give it a cooler API.
https://github.com/alex/rply

Last synced: 7 months ago
JSON representation

An attempt to port David Beazley's PLY to RPython, and give it a cooler API.

• Host: GitHub
• URL: https://github.com/alex/rply
• Owner: alex
• License: bsd-3-clause
• Created: 2012-08-06T00:56:54.000Z (over 13 years ago)
• Default Branch: master
• Last Pushed: 2023-01-21T19:10:13.000Z (almost 3 years ago)
• Last Synced: 2024-05-17T14:41:54.168Z (over 1 year ago)
• Language: Python
• Size: 191 KB
• Stars: 378
• Watchers: 16
• Forks: 60
• Open Issues: 34
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

• awesome-compilers - RPLY - Port of the PLY project to RPython. (Tools and Frameworks / Python)

README

          
RPLY

====

.. image:: https://secure.travis-ci.org/alex/rply.png

    :target: https://travis-ci.org/alex/rply

Welcome to RPLY! A pure Python parser generator, that also works with RPython.

It is a more-or-less direct port of David Beazley's awesome PLY, with a new

public API, and RPython support.

You can find the documentation `online`_.

Basic API:

.. code:: python

    from rply import ParserGenerator, LexerGenerator

    from rply.token import BaseBox

    lg = LexerGenerator()

    # Add takes a rule name, and a regular expression that defines the rule.

    lg.add("PLUS", r"\+")

    lg.add("MINUS", r"-")

    lg.add("NUMBER", r"\d+")

    lg.ignore(r"\s+")

    # This is a list of the token names. precedence is an optional list of

    # tuples which specifies order of operation for avoiding ambiguity.

    # precedence must be one of "left", "right", "nonassoc".

    # cache_id is an optional string which specifies an ID to use for

    # caching. It should *always* be safe to use caching,

    # RPly will automatically detect when your grammar is

    # changed and refresh the cache for you.

    pg = ParserGenerator(["NUMBER", "PLUS", "MINUS"],

            precedence=[("left", ['PLUS', 'MINUS'])], cache_id="myparser")

    @pg.production("main : expr")

    def main(p):

        # p is a list, of each of the pieces on the right hand side of the

        # grammar rule

        return p[0]

    @pg.production("expr : expr PLUS expr")

    @pg.production("expr : expr MINUS expr")

    def expr_op(p):

        lhs = p[0].getint()

        rhs = p[2].getint()

        if p[1].gettokentype() == "PLUS":

            return BoxInt(lhs + rhs)

        elif p[1].gettokentype() == "MINUS":

            return BoxInt(lhs - rhs)

        else:

            raise AssertionError("This is impossible, abort the time machine!")

    @pg.production("expr : NUMBER")

    def expr_num(p):

        return BoxInt(int(p[0].getstr()))

    lexer = lg.build()

    parser = pg.build()

    class BoxInt(BaseBox):

        def __init__(self, value):

            self.value = value

        def getint(self):

            return self.value

Then you can do:

.. code:: python

    parser.parse(lexer.lex("1 + 3 - 2+12-32"))

You can also substitute your own lexer. A lexer is an object with a ``next()``

method that returns either the next token in sequence, or ``None`` if the token

stream has been exhausted.

Why do we have the boxes?

-------------------------

In RPython, like other statically typed languages, a variable must have a

specific type, we take advantage of polymorphism to keep values in a box so

that everything is statically typed. You can write whatever boxes you need for

your project.

If you don't intend to use your parser from RPython, and just want a cool pure

Python parser you can ignore all the box stuff and just return whatever you

like from each production method.

Error handling

--------------

By default, when a parsing error is encountered, an ``rply.ParsingError`` is

raised, it has a method ``getsourcepos()``, which returns an

``rply.token.SourcePosition`` object.

You may also provide an error handler, which, at the moment, must raise an

exception. It receives the ``Token`` object that the parser errored on.

.. code:: python

    pg = ParserGenerator(...)

    @pg.error

    def error_handler(token):

        raise ValueError("Ran into a %s where it wasn't expected" % token.gettokentype())

Python compatibility

--------------------

RPly is tested and known to work under Python 2.7, 3.4+, and PyPy. It is

also valid RPython for PyPy checkouts from ``6c642ae7a0ea`` onwards.

Links

-----

* `Source code and issue tracker `_

* `PyPI releases `_

* `Talk at PyCon US 2013: So you want to write an interpreter? `_

.. _`online`: https://rply.readthedocs.io/