{"id":26927587,"url":"https://github.com/lmmx/pypi-docs-url","last_synced_at":"2025-04-02T04:17:00.525Z","repository":{"id":276546618,"uuid":"929003168","full_name":"lmmx/pypi-docs-url","owner":"lmmx","description":"Proof of concept to get the URL of a Python package","archived":false,"fork":false,"pushed_at":"2025-03-24T22:04:04.000Z","size":70,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-24T23:19:36.224Z","etag":null,"topics":["package-metadata","pypi-packages"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/pypi-docs-url","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lmmx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-07T16:23:11.000Z","updated_at":"2025-03-24T22:04:07.000Z","dependencies_parsed_at":"2025-03-24T23:19:22.954Z","dependency_job_id":"02001500-1171-423a-9309-cee2b50d9469","html_url":"https://github.com/lmmx/pypi-docs-url","commit_stats":null,"previous_names":["lmmx/pypi-docs-url"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmx%2Fpypi-docs-url","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmx%2Fpypi-docs-url/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmx%2Fpypi-docs-url/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmmx%2Fpypi-docs-url/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lmmx","download_url":"https://codeload.github.com/lmmx/pypi-docs-url/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246752628,"owners_count":20827987,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["package-metadata","pypi-packages"],"created_at":"2025-04-02T04:16:59.398Z","updated_at":"2025-04-02T04:17:00.513Z","avatar_url":"https://github.com/lmmx.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pypi-docs-url\n\n**Proof of concept** for a tool that locates a project’s [Intersphinx inventory](https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html) (`objects.inv`) by combining **PyPI** metadata and (if necessary) **GitHub** actions/workflows analysis.\n\nMost packages use a direct `docs/` URL, but many run extra steps—**mkdocs**, **Sphinx**, GitHub Pages, multi-step publishing—so the final `objects.inv` location can be non-trivial. **pypi-docs-url** tries to guess it automatically.\n\n## Basic Overview\n\n1. **Check PyPI** (via [PyPI JSON](https://pypi.org/pypi/\u003cpackage\u003e/json)) for a doc link (`project_urls` or `home_page`).\n2. **If missing**, parse the GitHub repo from PyPI (if present).\n3. **Fetch** the `.github/workflows/docs-python.yml` to see how docs are deployed (e.g. “`target-folder: api/python/stable`”).\n4. **Build** a guessed final docs domain (e.g. `docs.pola.rs`) plus that subfolder + `/objects.inv`.\n5. **Check** if that file returns HTTP 200, and if so, print the final URL.\n\n## Installation\n\n```bash\npip install pypi-docs-url\n```\n\n*(Or however you prefer to install from your local copy, e.g. `pip install .`.)*\n\n## Usage\n\n### CLI\n\nThe primary entry point is a CLI tool, **`pypi-docs-url`**, which accepts a single argument: the **package name** you want to inspect. It simply prints the discovered `objects.inv` URL if successful, or reports failure:\n\n```bash\n$ pypi-docs-url polars\nhttps://docs.pola.rs/api/python/stable/objects.inv\n```\n\nIf it can’t find the `objects.inv`, it prints:\n```\nNo objects.inv discovered or parse failed.\n```\n\n**That’s it—**by default, it’s short and sweet.\n\n#### CLI Help\n\n```bash\n$ pypi-docs-url --help\nUsage: pypi-docs-url [OPTIONS] PACKAGE_NAME\n\n  CLI entry point to get the 'objects.inv' URL for PACKAGE_NAME, if any.\n\nOptions:\n  --help  Show this message and exit.\n```\n\n### Python API\n\nIf you want to **reuse** the logic in your own code (not just the CLI), simply import from the package:\n\n```python\nfrom pypi_docs_url import get_intersphinx_url\n\ndef example():\n    inv_url = get_intersphinx_url(\"polars\")\n    if inv_url:\n        print(f\"Found objects.inv =\u003e {inv_url}\")\n    else:\n        print(\"No objects.inv discovered.\")\n```\n\n*(This means you can embed the logic in your library or script, rather than invoking the CLI.)*\n\n## Walkthrough / Verbose Explanation\n\nIf you want to see the **internal** steps—like how we parse PyPI’s JSON, find `docs-python.yml`, guess the subfolder, etc.—**the package contains commented code** illustrating each step in detail. Look in [`examples/demo_verbose.py`](./examples/demo_verbose.py) for a fully “instrumented” version:\n\n```python\n# demo_verbose.py (example, not installed by default)\n#\n# This script shows a step-by-step chain, printing out partial PyPI JSON,\n# partial GitHub workflow lines, how we locate \"api/python/stable\",\n# and the final HEAD request on objects.inv.\n#\n# EXAMPLE USAGE:\n#   python demo_verbose.py polars\n#\n# (Output lines are commented out to avoid spamming the console by default.)\n\nimport re\nimport requests\nimport yaml\n\ndef main(package_name: str):\n    # 1) PyPI JSON fetch\n    # print(f\"Fetching PyPI JSON at: https://pypi.org/pypi/{package_name}/json\")\n    # ...\n    # print(\"Relevant fields: { ... }\")\n\n    # 2) Doc link in project_urls\n    # print(f\"Found doc link =\u003e https://docs.pola.rs/api/python/stable/reference/index.html\")\n\n    # 3) GitHub link =\u003e parse org/repo =\u003e fetch docs-python.yml\n    # print(\"Key lines from docs-python.yml containing 'deploy' or 'target-folder': ...\")\n\n    # 4) Identify 'api/python/stable' =\u003e build guess =\u003e HEAD request =\u003e final URL\n    # print(\"HEAD =\u003e 200 =\u003e success!\")\n\n    pass\n\nif __name__ == \"__main__\":\n    import sys\n    pkg = sys.argv[1] if len(sys.argv) \u003e 1 else \"polars\"\n    main(pkg)\n```\n\nYou can uncomment lines in that **demo** to see each step’s output. This is purely an example—**the main installed `pypi-docs-url` script** does not do this verbose printing.\n\n---\n\n## Example\n\nBelow is how Polars’ doc resolution might look if you were to **uncomment** the verbose lines:\n\n```text\n** Attempting to discover polars's objects.inv via PyPI → GH workflow logic **\n\n[1] Fetching PyPI JSON at: https://pypi.org/pypi/polars/json\n\n[1a] Relevant PyPI fields:\n  project_urls: {\n    \"Changelog\": \"...\",\n    \"Documentation\": \"https://docs.pola.rs/api/python/stable/reference/index.html\",\n    \"Repository\": \"https://github.com/pola-rs/polars\"\n  }\n\n[1b] Found doc link =\u003e https://docs.pola.rs/api/python/stable/reference/index.html\n\n[1c] GitHub =\u003e https://github.com/pola-rs/polars\n[1d] org=pola-rs, repo=polars\n\n[2] Attempting to fetch workflow at: https://raw.githubusercontent.com/pola-rs/polars/main/.github/workflows/docs-python.yml\n\n[2a] Key lines (containing 'deploy' or 'target-folder'):\n  ... \"target-folder: api/python/dev\"\n  ... \"target-folder: api/python/stable\"\n\n[2c] stable =\u003e api/python/stable\n\n[3] Final guess =\u003e https://docs.pola.rs/api/python/stable/objects.inv\nPerforming HEAD =\u003e status 200 =\u003e success!\n```\n\n*(That’s the deeper logic under the hood, while the actual CLI usage is a single line printing out the final URL.)*\n\n---\n\n## Testing\n\nWhen running tests, and adding new packages, we run `pytest -s \u003e tests/find_docs_test.log.txt` to\nsave the 'trace' (from debugging) to give clues for LLMs, as well as general debugging and\ninterpretation.\n\n## Contributing\n\nFeel free to open issues/PRs:\n\n- **Expand** to handle multiple subdomains or dev vs stable docs.\n- **Add** a fallback if the doc link is missing but the project uses Sphinx.\n- **Improve** logic for *mkdocs* vs *Sphinx* detection.\n\n## License\n\nMIT.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmmx%2Fpypi-docs-url","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flmmx%2Fpypi-docs-url","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmmx%2Fpypi-docs-url/lists"}