{"id":20885863,"url":"https://github.com/zotero/citeproc-rs","last_synced_at":"2025-04-14T13:34:18.934Z","repository":{"id":45426368,"uuid":"159442309","full_name":"zotero/citeproc-rs","owner":"zotero","description":"CSL processor in Rust.","archived":false,"fork":false,"pushed_at":"2024-08-06T01:16:32.000Z","size":3658,"stargazers_count":81,"open_issues_count":28,"forks_count":11,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-10T16:52:04.035Z","etag":null,"topics":["csl","pandoc","pandoc-filter","rust"],"latest_commit_sha":null,"homepage":"https://cormacrelf.github.io/citeproc-wasm-demo/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zotero.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-28T04:11:41.000Z","updated_at":"2025-03-17T03:23:25.000Z","dependencies_parsed_at":"2024-06-18T22:38:07.242Z","dependency_job_id":"a5b05362-5323-41c6-aad8-12cb5afaf0d3","html_url":"https://github.com/zotero/citeproc-rs","commit_stats":null,"previous_names":["cormacrelf/citeproc-rs"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zotero%2Fciteproc-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zotero%2Fciteproc-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zotero%2Fciteproc-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zotero%2Fciteproc-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zotero","download_url":"https://codeload.github.com/zotero/citeproc-rs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248889022,"owners_count":21178151,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csl","pandoc","pandoc-filter","rust"],"created_at":"2024-11-18T08:14:52.041Z","updated_at":"2025-04-14T13:34:18.901Z","avatar_url":"https://github.com/zotero.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `citeproc-rs`\n\nA work-in-progress implementation of [CSL][] and [CSL-M][] in [Rust][]. It is \ngeared at:\n\n* replacing `citeproc-js` by providing WebAssembly bindings such that it could \n  be embedded in Zotero or fulfil any other JavaScript-based use;\n* replacing much of `pandoc-citeproc`, by running as a Pandoc Filter;\n* making it easy to use citeproc from any programming language, and integrate \n  into any use case; and\n* correctness and high performance.\n\n[CSL]: https://docs.citationstyles.org/en/stable/specification.html\n[CSL-M]: https://citeproc-js.readthedocs.io/en/latest/csl-m/index.html\n[Rust]: https://rust-lang.org/\n\n## Supported Rust versions\n\nWith a stable Rust compiler, 1.43 or later, you can:\n\n* Build `citeproc` as a library\n* Build a WebAssembly package (see below for details)\n\nFor development, you need a recent nightly compiler. This is required to run \nthe test suite within the `cargo test` harness using `datatest`. Keep your \nnightly up to date, as there are frequent breaking changes in that area at the \nmoment, and this repo will track close to the edge.\n\nYou should install Rust and keep it up to date with \n[rustup](https://rustup.rs/). \n\n## Try it out\n\nThere is a [demo](https://cormacrelf.github.io/citeproc-wasm-demo/) of \n`citeproc-rs` in action. It includes a graph visualisation of cite ambiguity \ntesting. See if you can figure out how it works.\n\nTo see how well `citeproc-rs` is doing on the tests, visit\nhttps://cormacrelf.github.io/citeproc-rs-test-viewer/\n\n## WebAssembly usage\n\nThe WebAssembly shell lives in `crates/wasm`. It consists of a JavaScript API \nthat wraps the `citeproc` crate, and a mechanism for asynchronous locale \nfetching. The API works mostly by serializing to JSON and back, but this is \ninvisible. It includes TypeScript definitions, which are the main source of \ndocumentation. Open up the generated `.d.ts` file or view the doc-comments with \nyour editor or IDE. A useful reference is the `js-demo`, which exercises most \nof its functionality and demonstrates correct usage of the \n`batchedUpdates`/`UpdateSummary` API.\n\nNote especially that `Driver` cannot be garbage collected. You will need to \n`.free()` it manually, otherwise the whole engine and its cached data will be \nleaked. Like many things with WebAssembly MVP, this might improve. Same with \nthe WASM multithreading proposal -- computing cites is synchronous and \nsingle-threaded now, but it may be desirable in future to make it async and \nmultithreaded. We'll see.\n\n### Building\n\n1. Install [wasm-pack](https://rustwasm.github.io/wasm-pack/installer/)\n2. `cd crates/wasm`; ` wasm-pack build -h`\n\nRefer to the \n[docs](https://rustwasm.github.io/docs/wasm-pack/commands/build.html) for on \nhow to get the output you need, particularly `--target`. \n\n## Non-WebAssembly usage\n\nFor Rust users, this package will available at some point on crates.io. The \n`csl` crate is already available, if all you wanted to do was parse/validate \nstyles and locales. (Hint: `use std::str::FromStr; Style::from_str(xml);`.) For \nthe rest, it needs a slightly more stable API, and maybe some of the \ncrate-splitting rethought. (I don't want to claim crate names and then change \nmy mind later. They are split for compile time reasons.)\n\nIt will likely be possible to support communicating via JSON messages over \nstdio: drop me a line over at \nhttps://github.com/cormacrelf/citeproc-rs/issues/13 if this interests you.\nThere is currently no way to use `citeproc-rs` via C FFI. This is mostly \nbecause of the complex structured data involved, which requires significant \nconversion effort and may not be worth it. If this is something you really \nwant, file an issue.\n\n## Running the CSL test suite\n\n`citeproc-rs` comes with a full-featured test harness for the CSL test suite, \nbased on the Rust testing infrastructure. This includes colourful diffs, and \nsupport for a new YAML-based test case format. However, given that at the \nmoment not all of the tests pass, a more nuanced way of detecting failure and \ncomparing results to find regressions between revisions was necessary. So now \nit can store and diff test runs. Pull requests that cause regressions (`Ok =\u003e \nFailure`) compared to master will fail.\n\n```sh\n# setup once\ncargo pull-locales\ncargo pull-test-suite\n\ncargo test-suite --help\n\n# the whole suite in parallel\ncargo test-suite run\n\n# the whole suite with deterministic test execution order\n# this helps show related tests alongside one another in the terminal output\ncargo test-suite run -- --test-threads 1\n\n# for a particular test, paste the file name\ncargo test-suite run -- name_ParsedDroppingParticleWithApostrophe.txt\n\n# for a subset of tests with some commonality in the name (this runs 8 of them)\ncargo test-suite run -- name_Initials\n\n# store a test run for comparison\n# this will also save a copy in 'branch_name' and 'commit_hash' if your working \n# directory is clean\ncargo test-suite store [name] [-- filter_pattern]\ncargo test-suite store disamb -- disamb # all of the disamb tests\n\n# diff two named, commit-named or commit-hash test results\n# outputs any regressions and fixed test cases\n# exits with code 1 if any regressions\ncargo test-suite diff master # compares to ..current\n\n# only checks the intersection of the tests, especially if you're using a filter\ncargo test-suite diff master..disamb\n# test result: 0 regressions, 0 new passing tests, 0 new ignores, out of 107 intersecting tests\n\n# saves in 'current'\ncargo test-suite store\n# copies 'current' to 'blessed'\ncargo test-suite bless\n# (make some changes)\n# compares blessed..current\ncargo test-suite store \u0026\u0026 cargo test-suite diff\n\n# in a clean repo, go back in time to a commit and store the captured\n# result by its commit SHA and an optional name, then checkout HEAD again\ncargo test-suite checkout-store [name]\n```\n\n\u003c!--\n\nHidden because not currently working.\n\n## Try it out!\n\nCurrently it can:\n\n* parse a CSL style (ignoring `\u003cinfo\u003e`) with built-in validation, \n  type-checking, error reporting, and semantic versioning,\n* parse a CSL-M style (ignoring `\u003cinfo\u003e`, probably still missing many \n  undocumented `citeproc-js` extensions to the spec),\n* parse locale files and perform locale fallback and merging for \n  terms/dates/etc inside it\n* parse a CSL-JSON file into references\n* pluck out a particular reference, and execute the style against only that one\n* read and write cites for an entire Pandoc JSON document\n\n### Parse a style\n\n```sh\ngit clone https://github.com/cormacrelf/citeproc-rs\ncd citeproc-rs/crates/citeproc-cli\ncargo run -- --csl ../example.csl # runs on a predefined single ref\ncargo run -- --csl ../example.csl --library path/to/csl-json/file.json\n```\n\nTo test it across the entire styles repo:\n\n```sh\ncd citeproc-rs/crates/citeproc-cli\ncargo install --path . --force\ncd ../..\ngit clone https://github.com/citation-style-language/styles\nfor style in styles/*.csl; do citeproc-rs --csl $style | pandoc -f json -t html; done\n```\n\n* Some styles in the repo are possibly invalid (mostly for using terms that \n  don't exist).\n* Some will successfully output HTML!\n\n### Parse a locale\n\nYou can also parse a locale to check for errors. It can find a locale in a \nlocales directory assuming it is structured like [the official CSL locales \nrepo](https://github.com/citation-style-language/locales), found via\n[directories](https://docs.rs/directories) (the cache directory). **Shortcut**:\n\n```sh\n# clones the locales repo into place for you\ncargo pull-locales\n```\n\nThen:\n\n```sh\n# currently broken\ncd crates/citeproc-cli\ncargo run -- parse-locale --lang en-GB\n```\n\n### The big end-to-end Pandoc filter (currently broken)\n\n#### Step 1: export a CSL-JSON library somewhere, with Zotero for example\n\n#### Step 2: create a markdown file\n\nIt must contain inline `csl`/`bibliography` metadata. Currently, and contrary \nto its documentation, Pandoc will automatically add `-F pandoc-citeproc` \nwhenever you add command line `--metadata csl=XXX` or `--metadata \nbibliography=XXX` flags. (That is, as far as I know, only supposed to happen if \nyou use shorthand `--csl XXX` or `--bibliography XXX`.)\n\n    ---\n    csl: path-to-my-csl.csl\n    bibliography: path-to-my-csl-json-library.json\n    ---\n\n    First paragraph.[@knownCitekey]\n\n    Second paragraph.[@knownCitekey; @anotherOne]\n\n#### Step 4: Run as a filter!\n\n```sh\n# much quicker than `build --release` or `install --path .`\ncargo build\n\npandoc -F ../target/debug/citeproc-rs input.md -s -o out.html\n\nopen out.html\n```\n\n--\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzotero%2Fciteproc-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzotero%2Fciteproc-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzotero%2Fciteproc-rs/lists"}