{"id":30190113,"url":"https://github.com/jlevy/sidematter-format","last_synced_at":"2026-03-01T00:06:54.978Z","repository":{"id":307885511,"uuid":"1030428729","full_name":"jlevy/sidematter-format","owner":"jlevy","description":"A convention for metadata and asset files alongside any document","archived":false,"fork":false,"pushed_at":"2025-09-19T21:36:56.000Z","size":127,"stargazers_count":4,"open_issues_count":1,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-19T07:02:46.386Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jlevy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-01T16:12:44.000Z","updated_at":"2025-09-17T06:31:58.000Z","dependencies_parsed_at":"2025-08-02T21:24:22.040Z","dependency_job_id":"b35adca4-8b1d-4263-be86-b0830d8f3135","html_url":"https://github.com/jlevy/sidematter-format","commit_stats":null,"previous_names":["jlevy/sidematter-format"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/jlevy/sidematter-format","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fsidematter-format","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fsidematter-format/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fsidematter-format/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fsidematter-format/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jlevy","download_url":"https://codeload.github.com/jlevy/sidematter-format/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlevy%2Fsidematter-format/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29955885,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T22:53:01.873Z","status":"ssl_error","status_checked_at":"2026-02-28T22:52:50.699Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-12T19:10:15.208Z","updated_at":"2026-03-01T00:06:49.967Z","avatar_url":"https://github.com/jlevy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sidematter Format\n\n**Sidematter format** is a simple, universal convention for keeping metadata and assets\nalongside a primary document.\nIt is a useful complement to\n[frontmatter format](https://github.com/jlevy/frontmatter-format).\n\n## Motivation\n\nMany tools and formats need structured data *associated with* a document but not\n*inside* it:\n\n* **Metadata** that can’t easily be stored in a document itself, due to size or because\n  it is updated more often (e.g. document annotations, full version history, etc.)\n\n* **Additional files** that must travel with a document, such as images or resources\n  associated with an HTML or Markdown file.\n\nSidematter format defines a **minimal set of conventions** for naming and resolving such\n“sidecar files” in a consistent way.\n\nSidecar patterns are often used in data pipelines, in exports from web browsers, and\nother applications. Unfortunately, there’s no consistent convention for naming and\norganizing such external files, leading to varied, ad-hoc approaches that don’t\ninteroperate well.\n\nThis repository is a **description of the format** and a **reference implementation**.\nThe implementation is in Python but the format is simple and can be adopted by any tool\nor language.\n\nSidematter format does not specify a way to bundle the outputs, but a file plus its\nsidematter files can easily be bundled together in a zip or tarball.\n\n\u003e [!TIP]\n\u003e \n\u003e Sidematter format complements\n\u003e [**frontmatter format**](https://github.com/jlevy/frontmatter-format), which allows\n\u003e placing metadata within any text file.\n\u003e A good practice is to use frontmatter format for small metadata attached at the front\n\u003e of text files, and sidematter format for larger metadata, on binary files, or for\n\u003e additional file assets.\n\n## Examples\n\nSidematter format is easiest to illustrate by an example.\nGiven a primary document `report.md`, some possible sidematter files would be:\n\n```\nreport.md              # Primary document\nreport.meta.json       # JSON metadata\nreport.meta.yml        # YAML metadata (can use in addition to or instead of JSON)\nreport.assets/         # Asset directory\n    figure1.png\n    diagram.svg\n    styles.css\n```\n\nThe document and metadata can reference assets with relative paths:\n\n```markdown\n# My Report\n\n![Key findings](report.assets/figure1.png)\n\nSee the [full diagram](report.assets/diagram.svg) for details.\n```\n\nExample metadata content:\n\n```yaml\n# report.meta.yml\ntitle: Q3 Financial Analysis\nauthor: Jane Doe\ncreated_at: 2024-01-15\ntags:\n  - finance\n  - quarterly\n  - analysis\nprocessing_history:\n  - step: data_extraction\n    timestamp: 2024-01-15T10:30:00Z\n    tool: custom_extractor_v2.1\n  - step: analysis\n    timestamp: 2024-01-15T11:45:00Z\n    tool: pandas_analyzer\nimage_files:\n  - report.assets/figure1.png\n  - report.assets/diagram.svg\n```\n\nMetadata must be in JSON or YAML. The choice is flexible.\nFor ease of reading, such as a frontend serving system, JSON is often better.\nFor ease of manual editing, YAML is preferable.\nThe implementation should look for both formats, so will read the metadata on either of\nthese layouts seamlessly.\nIf both are present, the convention is to prefer the JSON.\n\nIf desired, sidecar metadata can also be omitted.\nAnother good pattern is to use frontmatter format (simple YAML metadata inserted as\nfrontmatter on the file itself), and omitted from the sidematter:\n\n```\nreport.md              # Main file with frontmatter format metadata in YAML\nreport.assets/         # Asset directory with extra files\n    figure1.png\n    diagram.svg\n    styles.css\n```\n\n## Goals of this Approach\n\n* **Clean separation:** Keep metadata and assets separate from primary content, not\n  requiring that it be bundled (like a zip file).\n\n* **Predictable asset filenames and metadata syntax:** Sidematter files should be\n  auto-detectable via consistent naming convention so it is easy for tools to discover\n  metadata and asset files.\n\n* **Schema- and format-neutral:** This is a simple convention for attaching metadata and\n  assets. It aims to be flexible and unopinionated.\n  so does not specify any specifics on asset file formats or metadata schema (other than\n  the use of YAML or JSON). The convention works with any file format since sidecars\n  don’t modify the original document.\n\n## Format Definition\n\nThe sidematter format defines naming conventions for files and directories related to a\n*base document*, which can be any file, with any name.\n\n### Path Transformation Rule\n\nGiven a base document with filename `basename.extension`, the sidematter files are:\n\n- **Metadata files:** `basename.meta.json` or `basename.meta.yml`\n\n- **Asset directory:** `basename.assets/` (directory containing related files)\n\nThe sidematter names are formed by dropping the final extension from the base document\nname, then appending the sidematter suffix:\n\n* Files without extensions get sidematter suffixes directly: `README` →\n  `README.meta.yml`, `README.assets/`.\n\n* For files with multiple extensions (e.g., `data.tar.gz`), only the final extension is\n  dropped: `data.tar.gz` → `data.tar.meta.yml`.\n\n### Metadata Schema\n\n* The schema of metadata files is **free-form and tool-dependent**. Common metadata\n  conventions include standard fields like `title`, `description`, `author`,\n  `created_at`, and `tags`, but applications are free to define their own schemas.\n\n* Both JSON and YAML are allowed.\n  JSON is often preferred for machine-generated metadata due to ubiquitous parsing\n  support. YAML is often better for human-authored or human-readable metadata due to\n  readability and comment support.\n\n* If there is a schema associated with the metadata, follow the standard convention of\n  linking to it with the `$schema` key, so that tools like IDEs can validate the schema.\n\n### Metadata Precedence\n\nIn most cases, metadata should only reside in one place, typically `basename.meta.yml`.\nImplementations should observe precedence and pick metadata from the first location\nfound in this order:\n\n1. Metadata JSON: `basename.meta.json`\n\n2. Metadata YAML: `basename.meta.yml`\n\n3. Optionally, implementations can look for\n   [frontmatter](https://github.com/jlevy/frontmatter-format) on the file itself (if it\n   is a text file)\n\n### Asset Filenames\n\n- **Any asset files are allowed:** The `.assets/` directory structure is free-form and\n  tool-dependent. Files can be organized in subdirectories as needed.\n\n- **Relative path resolution:** References from the base document (such as Markdown or\n  HTML) to assets should use relative paths starting with the asset directory name.\n\n## Python API Usage\n\nThe Python implementation provides a simple reference implementation for reading and\nwriting sidematter.\n\n### Reading Sidematter Metadata and Assets\n\n```python\nfrom sidematter_format import Sidematter\n\n# Read all sidematter for a document by checking the filesystem.\n# Returns an immutable ResolvedSidematter.\npaths = Sidematter(Path(\"report.md\")).resolve() \nprint(paths.primary)  # Path('report.md')\nprint(paths.meta)  # {'title': 'Q3 Report', 'author': 'Jane Doe', ...}\nprint(paths.meta_path)  # Path('report.meta.yml') or None\nprint(paths.assets_path)  # Path('report.assets') or None\n```\n\n### Writing Sidematter Metadata and Assets\n\n```python\nfrom sidematter_format import Sidematter\n\n# Create a Sidematter object for read/write operations\nsm = Sidematter(Path(\"report.md\"))\n\n# Write metadata as YAML (default)\nmetadata = {\n    \"title\": \"Q3 Financial Analysis\",\n    \"author\": \"Jane Doe\"\n    \"tags\": [\"finance\", \"quarterly\"]\n}\nsm.write_meta(metadata)\n\n# Write metadata as JSON\nsm.write_meta(metadata, fmt=\"json\")\n\n# Write pre-formatted YAML/JSON string\nsm.write_meta(\"title: My Report\\nauthor: Jane Doe\\n\")\n\n# Remove all metadata files\nsm.write_meta(None)\n\n# Get the path for an asset (creates .assets/ directory)\nchart_path = sm.asset_path(\"chart.png\")\n# Returns: Path('report.assets/chart.png')\n\n# Copy a file into the assets directory\nsm.add_asset(\"~/Downloads/chart.png\")\n# or with a custom name:\nsm.add_asset(\"~/Downloads/fig1.png\", dest_name=\"chart.png\")\n\n# Check if assets directory exists\nif sm.resolve_assets():\n    print(f\"Assets found at: {sm.assets_dir}\")\n```\n\n## FAQ\n\n* **Hasn’t this been done before?**\n\n  Similar patterns exist in various tools (Jekyll’s `_files/` directories, Hugo’s page\n  bundles, etc.), but there’s no universal convention that works across different tools\n  and file types. This format provides a simple, consistent approach.\n\n* **When should I use sidematter vs frontmatter?**\n\n  Use **frontmatter format** for small, essential metadata, especially on text files of\n  any kind. Use **sidematter format** for larger metadata, when the original file cannot\n  be modified or metadata that should be updated separately from the original file, or\n  if additional asset files are needed.\n\n* **Does this work with version control?**\n\n  Yes, of course. Typically you would check in both metadata and assets, but if it’s easy\n  to put `*.assets/` or `*.meta.json` or `*.meta.yml` in `.gitignore` to avoid including\n  them in version control.\n\n* **Can I use both YAML and JSON metadata?**\n\n  Yes, tools should support either or even both formats simultaneously.\n  The convention is that if present, JSON will be used first, since that is often\n  auto-generated or faster to parse.\n\n* * *\n\n## Project Docs\n\nFor how to install uv and Python, see [installation.md](installation.md).\n\nFor development workflows, see [development.md](development.md).\n\nFor instructions on publishing to PyPI, see [publishing.md](publishing.md).\n\n* * *\n\n*This project was built from\n[simple-modern-uv](https://github.com/jlevy/simple-modern-uv).*\n```\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlevy%2Fsidematter-format","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjlevy%2Fsidematter-format","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlevy%2Fsidematter-format/lists"}