https://github.com/myint/untokenize

Transforms tokens into original source code (while preserving whitespace)
https://github.com/myint/untokenize

python tokenize untokennize

Last synced: about 2 months ago
JSON representation

Transforms tokens into original source code (while preserving whitespace)

• Host: GitHub
• URL: https://github.com/myint/untokenize
• Owner: myint
• License: mit
• Created: 2013-07-03T15:05:06.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2019-05-24T17:15:33.000Z (over 5 years ago)
• Last Synced: 2024-04-23T17:24:28.021Z (8 months ago)
• Topics: python, tokenize, untokennize
• Language: Python
• Homepage: https://pypi.python.org/pypi/untokenize
• Size: 16.6 KB
• Stars: 9
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

        
============

untokenize

============

*untokenize* transforms tokens into source code. Unlike the standard library's

``tokenize.untokenize()``, it preserves the original whitespace between tokens.

.. image:: https://travis-ci.org/myint/untokenize.svg?branch=master

    :target: https://travis-ci.org/myint/untokenize

    :alt: Build status

Usage

=====

.. code-block:: python

    import untokenize

    source_code = untokenize.untokenize(tokens)

Tests

=====

To run the unit tests::

    $ ./test_untokenize.py

There is also an acid test. It tokenizes Python code and confirms that the code

generated by untokenize exactly matches the original source code from before

tokenization::

    $ ./test_acid.py