{"id":15020244,"url":"https://github.com/jupyter/papyri","last_synced_at":"2026-03-17T21:04:31.110Z","repository":{"id":38313306,"uuid":"281771398","full_name":"jupyter/papyri","owner":"jupyter","description":null,"archived":false,"fork":false,"pushed_at":"2024-09-07T11:57:27.000Z","size":8487,"stargazers_count":84,"open_issues_count":85,"forks_count":17,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-10-29T17:57:06.163Z","etag":null,"topics":["closember","hacktoberfest","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jupyter.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-22T19:58:47.000Z","updated_at":"2025-08-25T16:53:42.000Z","dependencies_parsed_at":"2023-02-14T17:46:37.357Z","dependency_job_id":"6c9088c1-1aec-477d-a32f-6215a0a36ff7","html_url":"https://github.com/jupyter/papyri","commit_stats":{"total_commits":1375,"total_committers":15,"mean_commits":91.66666666666667,"dds":0.136,"last_synced_commit":"7b6ca7430751b9db9fd3def71dc6edfc3d528972"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/jupyter/papyri","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jupyter%2Fpapyri","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jupyter%2Fpapyri/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jupyter%2Fpapyri/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jupyter%2Fpapyri/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jupyter","download_url":"https://codeload.github.com/jupyter/papyri/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jupyter%2Fpapyri/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30631434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-17T17:32:55.572Z","status":"ssl_error","status_checked_at":"2026-03-17T17:32:38.732Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["closember","hacktoberfest","python"],"created_at":"2024-09-24T19:54:47.835Z","updated_at":"2026-03-17T21:04:31.078Z","avatar_url":"https://github.com/jupyter.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Papyri\n\n**Papyri** is a set of tools to build, publish (future functionality - to be done), install and render\ndocumentation within IPython and Jupyter.\n\n---\n\n| Information | Links |\n| :---------- | :-----|\n|   Project   | [![License](https://img.shields.io/badge/License-MIT-gray.svg?colorA=2D2A56\u0026colorB=5936D9\u0026style=flat.svg)](https://opensource.org/license/mit/) [![Rendered documentation](https://img.shields.io/badge/%F0%9F%93%96%20Rendered%20docs-gray.svg?colorA=2D2A56\u0026colorB=5936D9\u0026style=flat.svg)](https://papyri.vercel.app/)  |\n|     CI      | [![Python Package](https://github.com/jupyter/papyri/actions/workflows/python-package.yml/badge.svg)](https://github.com/jupyter/papyri/actions/workflows/python-package.yml) [![Linting](https://github.com/jupyter/papyri/actions/workflows/lint.yml/badge.svg)](https://github.com/jupyter/papyri/actions/workflows/lint.yml) |\n\nPapyri allows:\n- bidirectional crosslinking across libraries, \n- navigation, \n- proper reflow of user docstrings text,\n- proper reflow of inline images (when rendered to html),\n- proper math rendering (both in terminal and html),\n- and more.\n\n## Motivation\n\nSee some of the reasons behind the project on [this blog post](https://labs.quansight.org/blog/2021/05/rethinking-jupyter-documentation/).\n\nKey motivation is building a set of tools to build better documentation for Python projects.\n  - Uses an opinionated implementation to enable better understanding about the structure of your project.\n  - Allow automatic cross-links (back and forth) between documentation across Python packages.\n  - Use a documentation IR (intermediate representation) to separate building the docs from rendering the docs in many contexts.\n\nThis approach should hopefully allow a conda-forge-like model, where projects upload their IR to a given repo, a _single\nwebsite_ that contains documentation for multiple projects (without sub domains). The documentation pages can then be built with better cross-links between\nprojects, and _efficient_ page rebuild.\n\nThis should also allow displaying user-facing documentation on _non html_ backends (think terminal), or provide documentation in an\nIDE (Spyder/Jupyterlab), without having to iframe it.\n\n## Overview Presentation\n\nAnd this [small presentation at CZI EOSS4 meeting in early november 2021](https://docs.google.com/presentation/d/1sSh44smooCiOlj0-Zrac9n5KX0K_ABBFznDmMIwUnbM/edit?usp=sharing).\n\n## Screenshots\n\n\u003cdetail\u003e\n\u003csummary\u003eClick to expand\u003c/summary\u003e\nNavigating astropy's documentation from within IPython. Note that this includes\nforward refs but also backward references (i.e. which pages link to the current page.)\n\n![](assets/astropy.gif)\n\nType inference and keyboard navigation in terminal: Directives are properly\nrendered in terminal, examples are type inferred, clicking (or pressing enter)\non highlighted tokens would open said page (backspace navigates back).\n\n![](assets/papyri-nav.gif)\n\nSince Jupyter Notebook and Lab pages can render HTML, it should be possible to have\ninline graphs and images when using Jupyter inline help (to be implemented). In\nterminals, we replace inline images with a button/link to open images in an\nexternal viewer (quicklook, evince, paint...)\n\n![](assets/inline_graph.png)\n\nPapyri has complete information about which pages link to other pages; this allows\nus to create a local graph of which pages mention each other to find related topics.\n\nBelow, you can see the local connectivity graph for `numpy.zeros` (d3js, draggable, clickable).\n`numpy.zeroes` links to (or is linked from) all dots present there. In green, we show other\nnumpy functions; in blue, we show skimage functions; in orange, we show scipy functions; in red, we show xarray functions. Arrows\nbetween dots indicate pages which link to each other (for example ndarray is linked\nfrom `xarray.cos`), and dot size represents the _popularity_ of a page.\n\n![](assets/local_graph_zeroes.png)\n\nMath expressions are properly rendered even in the terminal: here, `polyfit` is shown in IPyhton with\npapyri enabled (left) and disabled (right).\n\n![](assets/vs_math.png)\n\u003c/detail\u003e\n\n---\n\n## Table of contents\n\n- [Installation](#installation)\n- [Usage](#usage)\n- [Rendering](#rendering)\n- [Architecture](#architecture)\n\n## Installation (not fully functional):\n\nSome functionality is not yet available when installing from PyPI. For now you\nneed a [Development installation](#development-installation) to access all\nfeatures.\n\nYou'll need Python 3.8 or newer, otherwise pip will tell you it can't find any matching distribution.\n\nInstall from PyPI:\n\n```bash\n$ pip install papyri\n````\n\nInstall given package documentation:\n\n```bash\n$ papyri install package_name [package_name [package_name [...]]]\n```\n\nOnly numpy 1.20.0, scipy 1.5.0 and xarray 0.17.0 are currently installable and published.\nFor other packages you will need to build locally which is a much more involved\nprocess.\n\nRun IPython terminal with Papyri as an extension:\n\n```\n$ ipython --ext papyri.ipython\n```\n\nThis will augment the `?` operator to show better documentation (when installed with `papyri install ...`\n\n*Papyri does not completely build its own docs yet, but you might be able to view a static rendering of it\n[here](https://pydocs.github.io/). It is not yet automatically built, so might be out of date.*\n\n### Development installation\n\nYou may need to get a modified version of numpydoc depending on the stage of development. You will need [pip \u003e\n21.3](https://pip.pypa.io/en/stable/news/#v21-3-1) if you want to make editable installs.\n\n```\n# clone this repo\n# cd this repo\npip install -e .\n```\n\nSome functionality requires ``tree_sitter_rst``. To build the TreeSitter rst parser:\n\n```bash\n$ git submodule update --init\n$ papyri build-parser\n```\n\n[Look at CI file](https://github.com/jupyter/papyri/blob/main/.github/workflows/python-package.yml) if those instructions are not up to date.\n\nNote that papyri still uses a custom parser which will be removed in the future to rely mostly on TreeSitter.\n\n### Testing\n\nInstall extra development dependencies by running:\n\n```bash\n$ pip install -r requirements-dev.txt\n```\n\nRun tests using\n\n```bash\n$ pytest\n```\n\n## Usage\n\nPapyri relies on three steps:\n\n - IR generation (executed by package maintainers);\n - IR installation (executed by end users or via pip/conda);\n - IR rendering (usually executed by the IDE, CLI/webserver).\n\n### IR Generation (`papyri gen`)\n\nThis is the step you want to trigger if you are building documentation using Papyri for a library you maintain. Most\nlikely as an end user you will not have to issue this step and can install pre-published documentation bundles.\nThis step is likely to occur only once per new release of a project.\n\nThe Toml files in `examples` will give you example configurations from some existing libraries.\n\n```\n$ ls -1 examples/*.toml\nexamples/IPython.toml\nexamples/astropy.toml\nexamples/dask.toml\nexamples/matplotlib.toml\nexamples/numpy.toml\nexamples/papyri.toml\nexamples/scipy.toml\nexamples/skimage.toml\n```\n\nRight now these files lives in papyri but would likely be in relevant repositories under `docs/papyri.toml` later on.\n\n\u003e [!NOTE]\n\u003e It is _slow_ on full numpy/scipy; use `--no-infer` (see below) for a subpar but faster experience.\n\nUse `papyri gen \u003cpath to example file\u003e`\n\nfor example:\n\n```\n$ papyri gen examples/numpy.toml\n```\n\n```\n$ papyri gen examples/scipy.toml\n```\n\nThis will create intermediate docs files in in `~/.papyri/data/\u003clibrary name\u003e_\u003clibrary_version\u003e`. See [Generation](#generation-papyri-gen) for more details.\n\nYou can also generate intermediate docs files for a subset of objects using the `--only` flag. For example:\n\n```\n$ papyri gen examples/numpy.toml --only numpy:einsum\n```\n\n\u003e [!IMPORTANT]\n\u003e To avoid ambiguity, papyri uses [fully qualified names](#qualified-names) to refer to objects. This means that you need to use `numpy:einsum` instead of `einsum` or `numpy.einsum` to refer to the `einsum` function in the `numpy` module, for example.\n\n\n### Installation/ingestion\n\nThe installation/ingestion of documentation bundles is the step that makes all\nbundles \"aware\" of each other, and allows crosslinking/indexing to work.\n\nWe'll reserve the term \"install\" and \"installation\" for when you download pre-build\ndocumentation bundle from an external source and give only the package name\n– which is not completely implemented yet.\n\nYou can ingest local folders with the following command:\n\n```\n$ papyri ingest ~/.papyri/data/\u003cpath to folder generated at previous step\u003e\n```\n\nThis will crosslink the newly generated folder with the existing ones.\nIngested data can be found in  `~/.papyri/ingest/` but you are not supposed to\ninteract with this folder with tools external to papyri.\n\nThere are currently a couple of pre-built documentation bundles that can be\npre-installed, but are likely to break with each new version of papyri. We\nsuggest you use the developer installation and ingestion procedure for now.\n\n## Rendering\n\nThe last step of the papyri pipeline is to render the docs, or the subset that\nis of interest to you. This will likely be done by your favorite IDE, probably\njust in time when you explore documentation. Nonetheless, we've\nimplemented a couple of external renderers to help debug issues.\n\n\u003e [!WARNING]\n\u003e Many rendering methods currently require papyri's own docs to be built and ingested first.\n\n```\n$ papyri gen examples/papyri.toml\n$ papyri ingest ~/.papyri/data/papyri_0.0.7  # or any current version\n```\n\nOr you can try to pre-install an old papyri doc bundle:\n\n```\n$ papyri install papyri\n```\n\n### Standalone HTML rendering\n\nTo see the rendered documentation for all packages previously ingested, run\n\n```bash\n$ papyri serve\n```\n\nThis will start a live server that will render the pages on the fly.\n\nIf you need to render static versions of the pages, use either of the following\ncommands:\n\n```bash\n$ papyri render  # render all the html pages statically in ~/.papyri/html\n$ papyri serve-static # start a http.server with the proper root to serve above files.\n```\n\n### Rich terminal rendering\n\nTo render the documentation for a single object on a terminal, use\n\n```\n$ papyri rich \u003cfully qualified name\u003e\n```\n\nFor example:\n\n```\n$ papyri rich numpy:einsum  # note the colon for the fully qualified name.\n```\n\nTo use the experimental interactive Textual interface in the terminal, use\n\n```\n$ papyri textual \u003cfully qualified name\u003e\n```\n\n### IPython extension\n\nTo run `papyri` as an IPython extension, run:\n\n```\n$ ipython --ext papyri.ipython\n```\n\nThis will start an IPython session with an augmented `?` operator.\n\n### Jupyter extension\n\nIn progress.\n\n### More commands\n\nYou can run `papyri` without a command to see all currently available commands.\n\n## Papyri - Name's meaning\n\nSee the legendary [Villa of Papyri](https://en.wikipedia.org/wiki/Villa_of_the_Papyri), which get its name from its\ncollection of many papyrus scrolls.\n\n## Architecture\n\n### Generation (`papyri gen`)\n\nCollects the documentation of a project into a *DocBundle* -- a number of\n*DocBlobs* (currently json files), with a defined semantic structure, and\nsome metadata (version of the project this documentation refers to, and\npotentially some other blobs).\n\nDuring the generation a number of normalisation and inference steps can and\nshould happen. For example:\n\n  - Using type inference into the `Examples` sections of docstrings and storing\n    those as pairs (token, reference), so that you can later decide that\n    clicking on `np.array` in an example brings you to numpy array\n    documentation; whether or not we are currently in the numpy documentation;\n  - Parsing \"See Also\" into a well defined structure;\n  - Running examples to generate images for docs with images (partially\n    implemented);\n  - Resolve local references. For example, when building the NumPy docs,\n    `zeroes_like` is non-ambiguous and should be normalized to\n    `numpy.zeroes_like`. Similarly, `~.pyplot.histogram`, should be normalized\n    to `matplotlib.pyplot.histogram` as the **target** and `histogram` as the\n    text.\n\nThe Generation step is likely project specific, as there might be import\nconventions that are defined per-project and should not need to be repeated\n(`import pandas as pd`, for example.)\n\nThe generation step is likely to be the most time consuming, and for each\nproject, results in the following outputs:\n\n- A `papyri.json` file, which is a list of unique qualified names corresponding\n  to the documented objects and some metadata;\n- A `toc.json` file, ?\n- An `assets` folder, containing all the images generated during the\n  generation;\n- A `docs` folder, ?\n- An `examples` folder, ?\n- A `module` folder, containing one json file per documented object. \n\nAfter the generation step, *what should have been processed*?\n\n### Ingestion (`papyri ingest`)\n\nThe ingestion step takes a DocBundle and/or DocBlobs and adds them into a graph\nof known items; the ingestion is critical to efficiently build the collection\ngraph metadata and understand which items refers to which. This allows the\nfollowing:\n\n - Update the list of backreferences to a *DocBundle*;\n - Update forward references metadata to know whether links are valid.\n\nCurrently the ingestion loads all in memory and updates all the bundle in place\nbut this can likely be done more efficiently.\n\nA lot more can likely be done at larger scale, like detecting if documentation\nhas changed in previous versions to infer for which versions of a library this\ndocumentation is valid.\n\nThere is also likely some curating that might need to be done at that point, as\nobjects such as `numpy.array` have an extremely large number of back-references.\n\n### Qualified names\n\nTo avoid ambiguity when referring to objects, papyri uses the\n*fully qualified name* of the object for its operations. This means that instead\nof a dot (`.`), we use a colon (`:`) to separate the module part from the\nobject's name and sub attributes.\n\nTo understand why we need this, assume the following situation: a top level\n`__init__` imports a function from a submodule that has the same name as the\nsubmodule:\n\n```\n# project/__init__.py\nfrom .sub import sub\n```\n\nThis submodule defines a class (here we use lowercase for the example):\n\n```\n# project/sub.py\nclass sub:\n    attribute:str\nattribute = 'hello'\n```\n\nand a second submodule is defined:\n```\n# project/attribute.py\nNone\n```\n\nUsing qualified names only with dots (`.`) can make it difficult to find out\nwhich object we are referring to, or implement the logic to find the object.\nFor example, to get the object `project.sub.attribute`, one would do:\n\n```\nimport project\nx = getattr(project, 'sub')\ngetattr(x, 'attribute')\n```\n\nBut here, because of the `from .sub import sub`, we end up getting the class\nattribute instead of the module. This ambiguity is lifted with a `:` as we now\nexplicitly know the module part, and `package.sub.attribute` is distinct from\n`package.sub:attribute`. Note that `package:sub.attribute` is also\nnon-ambiguous, even if not the right fully qualified name for an object.\n\nMoreover, using `:` as a separator makes the implementation much easier, as\nin the case of `package.sub:attribute` it is possible to directly execute\n`importlib.import_module('package.sub')` to obtain a reference to the `sub`\nsubmodule, without try/except or recursive `getattr` checking for the type of an\nobject.\n\n### Tree sitter information\n\nSee https://tree-sitter.github.io/tree-sitter/creating-parsers\n\n\n### When things don't work !\n\n#### `SqlOperationalError`:\n\n- The DB schema likely have changed, try: `rm -rf ~/.papyri/ingest/`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjupyter%2Fpapyri","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjupyter%2Fpapyri","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjupyter%2Fpapyri/lists"}