{"id":15107087,"url":"https://github.com/derwenai/mkrefs","last_synced_at":"2025-10-23T02:30:56.860Z","repository":{"id":49055456,"uuid":"366099154","full_name":"DerwenAI/mkrefs","owner":"DerwenAI","description":"MkDocs plugin to generate semantic reference Markdown pages from a knowledge graph","archived":false,"fork":false,"pushed_at":"2024-07-10T08:55:13.000Z","size":305,"stargazers_count":36,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-01-30T16:43:50.150Z","etag":null,"topics":["apidocs","bibliography","generence","glossary","jinja2-template","knowledge-graph","mkdocs","mkdocs-plugin","mkdocs-plugins","rdf","taxonomy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DerwenAI.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.txt","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-10T15:58:53.000Z","updated_at":"2025-01-27T02:56:32.000Z","dependencies_parsed_at":"2024-09-16T08:54:13.321Z","dependency_job_id":null,"html_url":"https://github.com/DerwenAI/mkrefs","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DerwenAI%2Fmkrefs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DerwenAI%2Fmkrefs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DerwenAI%2Fmkrefs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DerwenAI%2Fmkrefs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DerwenAI","download_url":"https://codeload.github.com/DerwenAI/mkrefs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237763855,"owners_count":19362310,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apidocs","bibliography","generence","glossary","jinja2-template","knowledge-graph","mkdocs","mkdocs-plugin","mkdocs-plugins","rdf","taxonomy"],"created_at":"2024-09-25T21:04:20.672Z","updated_at":"2025-10-23T02:30:56.454Z","avatar_url":"https://github.com/DerwenAI.png","language":"Python","funding_links":["https://github.com/sponsors/ceteri);"],"categories":[],"sub_categories":[],"readme":"# MkRefs\n\nThe **MkRefs** [plugin](http://www.mkdocs.org/user-guide/plugins/)\nfor [`MkDocs`](https://www.mkdocs.org/) \ngenerates reference Markdown pages from a knowledge graph, \nbased on the [`kglab`](https://github.com/DerwenAI/kglab) project.\n\nNo graph database is required; however, let us know if you'd like to\nuse one in particular.\n\nThere are several planned use cases for the **MkRefs** plugin,\nincluding:\n\n  * *biblio* – semantic bibliography entries, generated from RDF\n  * *glossary* – semantic glossary entries, generated from RDF\n  * *apidocs* – semantic [*apidocs*](https://pypi.org/search/?q=apidocs) supporting the [Diátaxis](https://derwen.ai/docs/kgl/learn/#a-grammar-of-learning) grammar for documentation, generated as RDF from Python source code\n  * *depend* – semantic dependency graph for Python libraries, generated as RDF from `setup.py`\n  * *index* – semantic search index, generated as RDF from MkDocs content\n\nOnly the *apidocs*, *biblio*, and *glossary* components have been\nadded to **MkRefs** so far, although the other mentioned components\nexist in separate projects and are being integrated.\n\n\n\u003cdetails\u003e\n  \u003csummary\u003eContributing Code\u003c/summary\u003e\n\nWe welcome people getting involved as contributors to this open source\nproject!\n\nFor detailed instructions please see:\n[CONTRIBUTING.md](https://github.com/DerwenAI/mkrefs/blob/main/CONTRIBUTING.md)\n\u003c/details\u003e\n\n\u003cdetails\u003e\n  \u003csummary\u003eSemantic Versioning\u003c/summary\u003e\n\nBefore \u003cstrong\u003eMkRefs\u003c/strong\u003e reaches release \u003ccode\u003ev1.0.0\u003c/code\u003e the \ntypes and classes may undergo substantial changes and the project is \nnot guaranteed to have a consistent API.\n\nEven so, we'll try to minimize breaking changes.\nWe'll also be sure to provide careful notes.\n\nSee:\n[changelog.txt](https://github.com/DerwenAI/mkrefs/blob/main/changelog.txt)\n\u003c/details\u003e\n\n\u003cimg\n alt=\"MkRefs, for semantic references\"\n src=\"https://raw.githubusercontent.com/DerwenAI/mkrefs/main/docs/assets/logo.png\"\n width=\"267\"\n/\u003e\n\n\n## Why does this matter?\n\nA key takeaway is that many software engineering aspects of open\nsource projects involve graphs, therefore a knowledge graph can\nprovide an integral part of an open source repository.\nMoreover, by using semantic representation (RDF) projects that\nintegrate with each other can share (i.e., federate) common resources,\nfor example to share definitions, analyze mutual dependencies, etc.\n\n\n## Installation\n\nTo install the plugin using `pip`:\n\n```\npython3 -m pip install mkrefs\n```\n\nThen add the plugin into the `mkdocs.yml` file:\n```yaml\nplugins:\n  - mkrefs\n```\nIn addition, the following configuration parameter is expected:\n\n  * `mkrefs_config` - YAML configuration file for **MkRefs**; e.g., `mkrefs.yml`\n\n---\n\n## API Docs\n\nAn `apidocs` parameter within the configuration file expects four\nrequired sub-parameters:\n\n * `page` – name of the generated Markdown page, e.g., `ref.md`\n * `template` – a [Jinja2 template](https://jinja.palletsprojects.com/en/3.0.x/) to generate Markdown, e.g., `ref.jinja`\n * `package` – name of the package being documented\n * `git` – base URL for source modules in Git, e.g., `https://github.com/DerwenAI/mkrefs/blob/main`\n\nThere is an optional `includes` parameter, as a list of class\ndefinitions to include.\nIf this is used, then all other classes get ignored.\n\nSee the source code in this repo for examples of how to format\nMarkdown within *docstrings*.\nSpecifically see the parameter documentation per method or function,\nwhich differs slightly from pre-exisiting frameworks.\n\nNote that the name of the generated Markdown page for the\napidocs must appear in the `nav` section of your `mkdocs.yml`\nconfiguration file.\nSee the structure used in this repo for an example.\n\n### Best Practices: RDF representation\n\nYou can use this library outside of MkDocs, i.e., calling it\nprogrammatically, to generate an RDF graph to represent your package\nAPI reference:\n\n```\npackage_name = \"mkrefs\"\ngit_url = \"https://github.com/DerwenAI/mkrefs/blob/main\"\nincludes = [ \"MkRefsPlugin\", \"PackageDoc\" ]\n\npkg_doc = PackageDoc(package_name, git_url, includes)\npkg_doc.build()\n\nkg = pkg_doc.get_rdf()\n```\n\nThe `PackageDoc.get_rdf()` method returns an RDF graph as an instance\nof an `kglab.KnowledgeGraph` object.\nFor more details, see \u003chttps://derwen.ai/docs/kgl/\u003e\n\n\n## Bibliography\n\nA `biblio` parameter within the configuration file expects four\nrequired sub-parameters:\n\n * `graph` – an RDF graph represented as a Turtle (TTL) file, e.g., `mkrefs.ttl`\n * `page` – name of the generated Markdown page, e.g., `biblio.md`\n * `template` – a [Jinja2 template](https://jinja.palletsprojects.com/en/3.0.x/) to generate Markdown, e.g., `biblio.jinja`\n * `queries` – [SPARQL queries](https://rdflib.readthedocs.io/en/stable/intro_to_sparql.html) used to extract bibliography data from the knowledge graph\n\nSee the [`mkrefs.ttl`](https://github.com/DerwenAI/mkrefs/blob/main/docs/mkrefs.ttl)\nfile for an example bibliography represented in RDF.\nThis comes from the documentation for the [`pytextrank`](https://derwen.ai/docs/ptr/biblio/)\nopen source project.\n\nIn the example RDF, the [*bibo*](http://bibliontology.com/) vocabulary represents\nbibliographic entries, and the [*FOAF*](http://xmlns.com/foaf/spec/) vocabulary\nrepresents authors.\nThis also uses two custom OWL relations from the [*derwen*](https://derwen.ai/ns/v1)\nvocabulary:\n\n  * `derw:citeKey` – citekey used to identify a bibliography entry within the documentation\n  * `derw:openAccess` – open access URL for a bibliography entry (if any)\n\nThe `queries` parameter has three required SPARQL queries:\n\n  * `entry` – to select the identifiers for all of the bibliograpy entries\n  * `entry_author` – a mapping to identify author links for each bibliography entry\n  * `entry_publisher` - the publisher link for each bibliography entry (if any)\n\nNote that the name of the generated Markdown page for the\nbibliography must appear in the `nav` section of your `mkdocs.yml`\nconfiguration file.\nSee the structure used in this repo for an example.\n\nYou may use any valid RDF representation for a bibliography.\nJust be sure to change the three SPARQL queries and the Jinja2\ntemplate accordingly.\n\nWhile this example uses an adaptation of the\n[MLA Citation Style](https://www.easybib.com/guides/citation-guides/mla-format/mla-citation/),\nfeel free to modify the Jinja2 template to generate whatever\nbibliographic style you need.\n\n\n### Best Practices: constructing bibliographies\n\nAs much as possible, bibliography entries should use the conventions at\n\u003chttps://www.bibsonomy.org/\u003e\nfor their [*citation keys*](https://bibdesk.sourceforge.io/manual/BibDeskHelp_2.html).\n\nJournal abbreviations should use\n[*ISO 4*](https://en.wikipedia.org/wiki/ISO_4) standards, \nfor example from \u003chttps://academic-accelerator.com/Journal-Abbreviation/System\u003e\n\nLinks to online versions of cited works should use\n[DOI](https://www.doi.org/)\nfor [*persistent identifiers*](https://www.crossref.org/education/metadata/persistent-identifiers/).\n\nWhen available, \n[*open access*](https://peerj.com/preprints/3119v1/)\nURLs should be listed as well.\n\n\n### What is going on here?\n\nFor example with the bibliography use case, when the plugin runs...\n\n1. It parses its configuration file to identify the target Markdown page to generate and the Jinja2 template\n2. The plugin also loads an RDF graph from the indicated TTL file\n3. Three SPARQL queries are run to identify the unique entities to extract from the graph\n4. The graph is serialized as [JSON-LD](https://derwen.ai/docs/kgl/ref/#kglab.KnowledgeGraph.save_jsonld)\n5. The `author`, `publisher`, and bibliography `entry` entities are used to *denormalize* the graph into a JSON data object\n6. The JSON is rendered using the Jinja2 template to generate the Markdown\n7. The Markdown page is parsed and rendered by MkDocs as HTML, etc.\n\n\n## Glossary\n\nA `glossary` parameter within the configuration file expects four\nrequired sub-parameters:\n\n * `graph` – an RDF graph represented as a Turtle (TTL) file, e.g., `mkrefs.ttl`\n * `page` – name of the generated Markdown page, e.g., `glossary.md`\n * `template` – a [Jinja2 template](https://jinja.palletsprojects.com/en/3.0.x/) to generate Markdown, e.g., `glossary.jinja`\n * `queries` – [SPARQL queries](https://rdflib.readthedocs.io/en/stable/intro_to_sparql.html) used to extract glossary data from the knowledge graph\n\nSee the [`mkrefs.ttl`](https://github.com/DerwenAI/mkrefs/blob/main/docs/mkrefs.ttl)\nfile for an example glossary represented in RDF.\nThis example RDF comes from documentation for the\n[`pytextrank`](https://derwen.ai/docs/ptr/glossary/)\nopen source project.\n\nIn the example RDF, the [*cito*](http://purl.org/spar/cito/)\nvocabulary represents citations to locally represented bibliographic\nentries.\nThe [*skos*](http://www.w3.org/2004/02/skos/core#) vocabulary\nprovides support for [*taxonomy*](http://accidental-taxonomist.blogspot.com/)\nfeatures, e.g., semantic relations among glossary entries.\nThis example RDF also uses a definition from the\n[*derwen*](https://derwen.ai/ns/v1) vocabulary:\n\n  * `derw:Topic` – a `skos:Concept` used to represent glossary entries\n\nThe `queries` parameter has three required SPARQL queries:\n\n  * `entry` – to select the identifiers for all of the bibliograpy entries\n  * `entry_syn` – a mapping of synonyms (if any)\n  * `entry_ref` – a mapping of external references (if any)\n  * `entry_cite` – citations to the local bibliography citekeys (if any)\n  * `entry_hyp` – a mapping of [*hypernyms*](https://en.wikipedia.org/wiki/Hyponymy_and_hypernymy) (if any)\n\nNote that the name of the generated Markdown page for the glossary\nmust appear in the `nav` section of your `mkdocs.yml` configuration\nfile.\nSee the structure used in this repo for an example.\n\nYou may use any valid RDF representation for a glossary.\nJust be sure to change the three SPARQL queries and the Jinja2\ntemplate accordingly.\n\n\n## Usage\n\nThe standard way to generate documentation with MkDocs is:\n```\nmkdocs build\n```\n\nIf you'd prefer to generate reference pages programmatically using\nPython scripts, see the code for usage of the `MkRefsPlugin` class,\nplus some utility functions:\n\n  * `load_kg()`\n  * `render_apidocs()`\n  * `render_biblio()`\n  * `render_glossary()`\n\nThere are also command line *entry points* provided, which can be\nhelpful during dev/test cycles on the semantic representation of your\ncontent:\n```\nmkrefs apidocs docs/mkrefs.yml\nmkrefs biblio docs/mkrefs.yml\nmkrefs glossary docs/mkrefs.yml\n```\n\n\n## Caveats\n\nWhile the [`MkDocs`](https://www.mkdocs.org/) utility is astoundingly useful,\nits documentation (and coding style) leave much room for improvement.\nThe [documentation for developing plugins](https://www.mkdocs.org/user-guide/plugins/#developing-plugins)\nis not even close to what happens when its code executes.\n\nConsequently, the **MkRefs** project is an attempt to reverse-engineer\nthe code from many other MkDocs plugins, while documenting its observed\nevent sequence, required parameters, limitations and workarounds, etc.\n\nTwo issues persist, where you will see warnings even though the **MkRefs**\ncode is handling configuration as recommended:\n\n```\nWARNING -  Config value: 'mkrefs_config'. Warning: Unrecognised configuration name: mkrefs_config \n```\n\nand\n\n```\nINFO    -  The following pages exist in the docs directory, but are not included in the \"nav\" configuration:\n  - biblio.md\n  - glossary.md\n  - ref.md\n```\n\nFor now, you can simply ignore both of these warnings.\nMeanwhile, we'll work on eliminating them.\n\n\n## Feature roadmap\n\nLet us know if you need features to parse and generate\n[BibTeX](http://www.bibtex.org/).\n\n\n## License and Copyright\n\nSource code for **MkRefs** plus its logo, documentation, and examples\nhave an [MIT license](https://spdx.org/licenses/MIT.html) which is\nsuccinct and simplifies use in commercial applications.\n\nAll materials herein are Copyright \u0026copy; 2021 Derwen, Inc.\n\n\n## Acknowledgements\n\nMany thanks to our open source [sponsors](https://github.com/sponsors/ceteri);\nand to our contributors:\n[@ceteri](https://github.com/ceteri)\n\nThis plugin code is based on the marvelous examples in\n\u003chttps://github.com/byrnereese/mkdocs-plugin-template\u003e\nwith kudos to [@byrnereese](https://github.com/byrnereese/),\nand also many thanks to \n[@louisguitton](https://github.com/louisguitton),\n[@dmccreary](https://github.com/dmccreary),\nand\n[@LarrySwanson](https://github.com/LarrySwanson)\nfor their inspiration and insights.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fderwenai%2Fmkrefs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fderwenai%2Fmkrefs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fderwenai%2Fmkrefs/lists"}