{"id":16247797,"url":"https://github.com/mbdevpl/horast","last_synced_at":"2025-08-21T06:32:14.581Z","repository":{"id":57437530,"uuid":"101387128","full_name":"mbdevpl/horast","owner":"mbdevpl","description":"Human-oriented abstract syntax tree (AST) parser/unparser for Python 3 that doesn't discard comments.","archived":false,"fork":false,"pushed_at":"2019-10-29T13:29:59.000Z","size":115,"stargazers_count":16,"open_issues_count":4,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-27T21:08:10.978Z","etag":null,"topics":["abstract-syntax-tree","comments","parser","unparsing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mbdevpl.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-25T09:10:26.000Z","updated_at":"2024-04-25T20:44:05.000Z","dependencies_parsed_at":"2022-09-15T11:00:40.432Z","dependency_job_id":null,"html_url":"https://github.com/mbdevpl/horast","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbdevpl%2Fhorast","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbdevpl%2Fhorast/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbdevpl%2Fhorast/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mbdevpl%2Fhorast/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mbdevpl","download_url":"https://codeload.github.com/mbdevpl/horast/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229890136,"owners_count":18140042,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["abstract-syntax-tree","comments","parser","unparsing"],"created_at":"2024-10-10T14:38:42.436Z","updated_at":"2024-12-19T20:14:19.275Z","avatar_url":"https://github.com/mbdevpl.png","language":"Python","readme":".. role:: python(code)\n    :language: python\n\n\nhorast\n======\n\nHuman-oriented abstract syntax tree (AST) parser/unparser for Python 3 that doesn't discard comments.\n\n.. image:: https://img.shields.io/pypi/v/horast.svg\n    :target: https://pypi.org/project/horast\n    :alt: package version from PyPI\n\n.. image:: https://travis-ci.org/mbdevpl/horast.svg?branch=master\n    :target: https://travis-ci.org/mbdevpl/horast\n    :alt: build status from Travis CI\n\n.. image:: https://ci.appveyor.com/api/projects/status/github/mbdevpl/horast?svg=true\n    :target: https://ci.appveyor.com/project/mbdevpl/horast\n    :alt: build status from AppVeyor\n\n.. image:: https://api.codacy.com/project/badge/Grade/33195093bb1b448bb9a5368b3507d615\n    :target: https://www.codacy.com/app/mbdevpl/horast\n    :alt: grade from Codacy\n\n.. image:: https://codecov.io/gh/mbdevpl/horast/branch/master/graph/badge.svg\n    :target: https://codecov.io/gh/mbdevpl/horast\n    :alt: test coverage from Codecov\n\n.. image:: https://img.shields.io/github/license/mbdevpl/horast.svg\n    :target: https://github.com/mbdevpl/horast/blob/master/NOTICE\n    :alt: license\n\nThis package provides new AST node types (Comment and Directive) which inherit from nodes in typed_ast.ast3 module.\nAdditionally, it provides implementation of parser and unparser for the extended AST allowing\nstraightforward readable code generation.\n\nSimple example of how to use this package:\n\n.. code:: python\n\n    from horast import parse, unparse\n\n    tree = parse(\"\"\"a = 1  # a equals one after this\"\"\")\n    print(unparse(tree))\n    # this will print the code with original comment\n\nMore examples in `\u003cexamples.ipynb\u003e`_.\n\n\ntechnical details\n-----------------\n\nParser is based on built-in tokenize module, as well as community packages asttokens and typed_ast.\n\nUnparser is essentially an extension of Unparser class from static_typing package.\n\nNodes provided and handled by horast are listed below.\n\n\nComment\n~~~~~~~\n\nFull line as well as end-of-line comments are parsed/unparsed correctly when they are outside\nof multi-line expressions.\n\nCurrently, handling of comments within multi-line expressions is implemented only partially.\n\n\nBlockComment\n~~~~~~~~~~~~\n\n**Not implemented yet.**\n\nThis node type is intended to store consecutive full-line comments in a\nsingle AST node, therefore simplifying handling of large blocks of comments.\n\n\nDirective\n~~~~~~~~~\n\nSince Python doesn't offer a direct way to convey directives in code,\ncomments beginning with any of the following prefixes will not be classified\nas typical comments, but as directives:\n\n*   if\n*   else\n*   endif\n*   def\n*   undef\n*   ifdef\n*   ifndef\n\nHowever in Python code they still remain as usual comments.\n\nFor example, the comments in the following code will all be classified as directives.\n\n.. code:: python\n\n    #ifdef DEBUG\n    debugging = True\n    #else\n    debugging = False\n    #fi\n\nHowever, when executed as Python, ``debugging`` will always end up ``False``\nbecause directives are preserved as usual comments in Python and therefore\nthey are ignored.\n\nTherefore, the ``Directive`` node is not meant to enable preprocessing of\nPython, at least for now.\n\nNote: the prefix is checked exactly. See the following example:\n\n.. code:: python\n\n    #if something\n    # if some other thing\n\nThe comment in the first line will become ``Directive`` object, while the one\non the second like will become ``Comment`` object.\n\nCurrently, this node is meant to work towards AST compatibility between\nPython and other languages, to aide code generation from Python AST into code\nin other languages -- one such use case is\n`*transpyle* project \u003chttps://github.com/mbdevpl/transpyle\u003e`_.\n\n\nPragma\n~~~~~~\n\n``Pragma`` nodes follow the same general rules as ``Directive`` nodes, i.e. they are\nstored in Python code as comments, but a comment will be classified as a pragma\nwhen it's prefixed with a predefined prefix:\n\n*   ``\" pragma:\"``\n\nAdditionally, two subclasses of Pragma are defined in horast, each with its\nown prefix that builds upon the generic pragma prefix:\n\n*   ``OpenMpPragma`` class defines prefix ``\" pragma: omp\"`` and stores OpenMP pragmas.\n*   ``OpenAccPragma`` class defines prefix ``\" pragma: acc\"`` and stores OpenACC pragmas.\n\nA code snippet below contains all 3 pragma types.\n\n.. code:: python\n\n    # pragma: once\n    use_openmp = True\n    use_openacc = True\n    ...\n    a, b = np.ndarray(...)\n    c = np.zeros(...)\n    # pragma: acc parallel copyin(a,b) copyout(c)\n    # pragma: acc loop gang\n    for y in range(ymax): # type: np.int32\n        # pragma: acc loop worker\n        for i in range(imax): # type: np.int32\n            # pragma: acc loop vector reduction(+: c[y][x])\n            for x in range(xmax): # type: np.int32\n                c[y, x] += a[y, i] * b[i, x]\n    # pragma: acc end parallel\n    ...\n    # pragma: omp parallel do\n    for i in range(input_data.size):  # type: int\n        # here we compute spam spam spam\n        heavy_compute(input_data[i])\n    ...\n\nAnd thus, in the example above:\n\n*   all comments starting with ``\" pragma: omp\"`` become ``OpenMpPragma`` objects,\n*   all comments starting with ``\" pragma: acc\"`` become ``OpenAccPragma`` objects,\n*   all other comments starting with ``\" pragma:\"`` become ``Pragma`` objects,\n*   type comments are ignored, and\n*   all other comments become ``Comment`` objects.\n\n\nAdditionally, horast module provides an extensible infrastructure to define\ncustom ``Pragma`` subclasses, enabling user to define their own pragmas for\nexperimentation. The provided OpenMP and OpenACC pragma definitions serve\nas examples of how to use this feature.\n\n\nInclude\n~~~~~~~\n\nSimilarly to how pragmas are handled, if a comment begins with ``\" include:\"``\nprefix, it will be classified as a special kind of include directive.\n\nAgain, this will be preserved as comment in Python code, but it's useful\nfor enhancing syntactic compatibility between Python and other, especially\nstatically compiled languages.\n\n\nrequirements\n------------\n\nCPython 3.5 or later.\n\nPython libraries as specified in `\u003crequirements.txt\u003e`_.\n\nBuilding and running tests additionally requires packages listed in `\u003ctest_requirements.txt\u003e`_.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbdevpl%2Fhorast","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmbdevpl%2Fhorast","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmbdevpl%2Fhorast/lists"}