{"id":26903202,"url":"https://github.com/stardog-union/millan","last_synced_at":"2025-04-01T10:05:23.697Z","repository":{"id":44552210,"uuid":"130265279","full_name":"stardog-union/millan","owner":"stardog-union","description":"The Stardog Whisperer: TypeScript/JS parsers for Stardog languages","archived":false,"fork":false,"pushed_at":"2024-07-29T14:46:41.000Z","size":11995,"stargazers_count":16,"open_issues_count":3,"forks_count":6,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-22T23:02:02.979Z","etag":null,"topics":["error-tolerant","knowledge-graph","linked-data","parser","rdf","sparql","stardog","turtle"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stardog-union.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-19T19:48:46.000Z","updated_at":"2024-10-04T16:24:24.000Z","dependencies_parsed_at":"2024-06-19T01:32:42.424Z","dependency_job_id":"a0410c3d-e701-493e-afe4-2639963f6c19","html_url":"https://github.com/stardog-union/millan","commit_stats":{"total_commits":152,"total_committers":11,"mean_commits":"13.818181818181818","dds":"0.35526315789473684","last_synced_commit":"4285c754104d4eb79e962f398013fd4381c59e9d"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stardog-union%2Fmillan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stardog-union%2Fmillan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stardog-union%2Fmillan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stardog-union%2Fmillan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stardog-union","download_url":"https://codeload.github.com/stardog-union/millan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246290555,"owners_count":20753723,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["error-tolerant","knowledge-graph","linked-data","parser","rdf","sparql","stardog","turtle"],"created_at":"2025-04-01T10:05:23.122Z","updated_at":"2025-04-01T10:05:23.681Z","avatar_url":"https://github.com/stardog-union.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Millan\n\nMillan is the [Stardog](https://www.stardog.com) whisperer -- a set of parsers\nfor languages used with Stardog (currently [SPARQL](https://en.wikipedia.org/wiki/SPARQL),\n[Turtle](https://en.wikipedia.org/wiki/Turtle_(syntax)),\n[TriG](https://en.wikipedia.org/wiki/TriG_(syntax)),\n[Stardog Mapping Syntax 2 (SMS)](https://www.stardog.com/docs/#_stardog_mapping_syntax_2),\n[Stardog Rules Syntax](https://www.stardog.com/docs/#_stardog_rules_syntax), the Turtle serialization of [SHACL](https://www.w3.org/TR/shacl/), and [GraphQL](https://graphql.github.io/graphql-spec/)).\n\n## Features\n\n- Error-tolerant parsing for W3C-compliant SPARQL and SPARQL with\n[Stardog extensions](https://www.stardog.com/docs/#_path_queries).\n- Error-tolerant parsing for W3C-compliant Turtle\n- Error-tolerant parsing for W3C-compliant TriG\n- Error-tolerant parsing for Stardog Mapping Syntax 2 (SMS)\n- Error-tolerant parsing for Stardog Rules Syntax (SRS)\n- Error-tolerant parsing for the Turtle serialization of the W3C Shapes Constraints Language (SHACL)\n- Error-tolerant parsing for GraphQL (both the Facebook standard and with [Stardog's GraphQL extensions](https://www.stardog.com/docs/#_graphql_queries))\n- Exported token matchers (regular expressions) and token sets for all of the\nabove languages\n- Universally usable (both in nodejs and the browser), either as a single bundle or as only the specific parts you need\n- Tested against W3C test suites\n- Small and fast\n- Written in [TypeScript](https://www.typescriptlang.org/), compiled to JS\n(with type declarations provided TypeScript)\n- Used in production at Stardog in support of [Stardog Studio](https://www.stardog.com/studio/)\nlanguage servers\n\n## Installation\n\nSimply run:\n\n```bash\nnpm install --save millan\n```\n\nYou can then import Millan in Node.js or in the browser, using any preferred\nmodule format or bundler.\n\nIn Node/CommonJS:\n\n```javascript\nconst millan = require('millan');\n// or, if you only need some part of millan (e.g., only _one_ of the parsers):\nconst graphql = require('millan/dist/standalone/millan.graphql');\n```\n\nAs an ES module:\n\n```javascript\nimport * as millan from 'millan';\n// or, if you only need some part of millan (e.g., only _one_ of the parsers):\nimport { StardogSparqlParser } from 'millan/dist/standalone/millan.sparql';\n```\n\nIn the browser, importing everything at once:\n\n```html\n\u003cscript src=\"path/to/millan/millan.js\"\u003e\u003c/script\u003e\n```\n\nIn the browser, importing only what is needed:\n\n```html\n\u003c!-- Here, we first load any shared chunks needed by 'sms' (as indicated by the filename), then the SMS-related Javacript only --\u003e\n\u003cscript src=\"path/to/millan/dist/browser/millan.vendors~graphql~shacl~sms~sparql~srs~turtle.js\"\u003e\u003c/script\u003e\n\u003cscript src=\"path/to/millan/dist/browser/millan.sms.js\"\u003e\u003c/script\u003e\n```\n\nWhen used in the browser via `script` tags, the API will be exposed on a global `millan` variable.\n\n## Basic Usage\n\nImport Millan or the specific parts of Millan that you need, e.g.:\n\n```javascript\nimport { W3SpecSparqlParser } from 'millan';\n```\n\nParsing a document requires a parser instance, so you should get one:\n\n```javascript\nconst sparqlParser = new W3SpecSparqlParser();\n```\n\nEvery parser instance has essentially the same API. They are all instances of the\nParser base class from [chevrotain](https://github.com/SAP/chevrotain), and they all implement the [IStardogParser](https://stardog-union.github.io/millan/interfaces/istardogparser.html) interface.\n\nThe two parser methods you'll use most often are `parse` and `tokenize`. Use\n`tokenize` only when you want the parser instance to _lex_ (tokenize) the\nprovided text:\n\n```javascript\nsparqlParser.tokenize(`\n  PREFIX : \u003chttp://example.org/ns#\u003e\n  SELECT *\n  WHERE { ?x ?y ?z }\n`);\n```\n\nResult:\n\n```\n[\n  {\n    \"image\": \"PREFIX\",\n    \"startOffset\": 3,\n    \"endOffset\": 8,\n    \"startLine\": 2,\n    \"endLine\": 2,\n    \"startColumn\": 3,\n    \"endColumn\": 8,\n    \"tokenTypeIdx\": 43,\n    \"tokenType\": {\n      \"PATTERN\": {},\n      \"tokenTypeIdx\": 43,\n      \"CATEGORIES\": [],\n      \"categoryMatches\": [],\n      \"categoryMatchesMap\": {},\n      \"tokenName\": \"PREFIX\",\n      \"isParent\": false\n    }\n  },\n  /* . . . */\n]\n```\n\nUse `parse` when you want the full parsing experience. Note that, because\nMillan's parsers are error-tolerant, the `parse` method returns an object\nthat contains both a [concrete syntax tree (CST)](https://sap.github.io/chevrotain/docs/guide/concrete_syntax_tree.html#ast-vs-cst)\n_and_ an array of any errors:\n\n```javascript\nsparqlParser.parse(`\n  PREFIX : \u003chttp://example.org/ns#\u003e\n  SELECT *\n  WHERE { ?x ?y ?z }\n`);\n```\n\nResult:\n\n```\n{\n  \"errors\": [],\n  \"cst\": {\n    \"name\": \"SparqlDoc\",\n    \"children\": {\n      \"Prologue\": [\n        {\n          \"name\": \"Prologue\",\n          \"children\": {\n            \"PrefixDecl\": [\n              {\n                \"name\": \"PrefixDecl\",\n                \"children\": {\n                  \"PREFIX\": [\n                    {\n                      \"image\": \"PREFIX\",\n                      \"startOffset\": 3,\n                      \"endOffset\": 8,\n                      \"startLine\": 2,\n                      \"endLine\": 2,\n                      \"startColumn\": 3,\n                      \"endColumn\": 8,\n                      \"tokenTypeIdx\": 43,\n                      \"tokenType\": {\n                        \"PATTERN\": {},\n                        \"tokenTypeIdx\": 43,\n                        \"CATEGORIES\": [],\n                        \"categoryMatches\": [],\n                        \"categoryMatchesMap\": {},\n                        \"tokenName\": \"PREFIX\",\n                        \"isParent\": false\n                      }\n                    }\n                  ],\n                  \"PNAME_NS\": [ /* . . . */ ],\n                  /* . . . etc. . . . */\n```\n\nHelpfully, the `name` fields in the CSTs returned by Millan's parsers exactly\nmatch, in _nearly_ every case (we've removed some redundancy), the names of the\nrules in the official EBNF for the corresponding grammar (e.g., `SparqlDoc`,\n`Prologue`, `PrefixDecl`, etc.).\n\nWhen there are errors in parsing, Millan's parsers still return a best-effort\nCST, as well as information including the spot where the error occurred, what\nthe parser expected to find at that spot, what the parser actually found in\nthat spot, the \"context\" (i.e., the position in the grammar's \"rule stack\"),\nand more:\n\n```javascript\nsparqlParser.parse(`\n  PREFIX : \u003chttp://example.org/ns#\u003e\n  SELECT 1\n  WHERE { ?x ?y ?z }\n`);\n```\n\nResult:\n\n```\n{\n  \"errors\": [\n    {\n      \"name\": \"NoViableAltException\",\n      \"message\": \"Expecting: one of these possible Token sequences:\\n  1. [?foo]\\n  2. [?bar]\\n  3. [LParen]\\n  4. [Star]\\nbut found: '1'\",\n      \"token\": {\n        \"image\": \"1\",\n        \"startOffset\": 47,\n        \"endOffset\": 47,\n        \"startLine\": 3,\n        \"endLine\": 3,\n        \"startColumn\": 10,\n        \"endColumn\": 10,\n        \"tokenTypeIdx\": 7,\n        \"tokenType\": {\n          \"PATTERN\": {},\n          \"tokenTypeIdx\": 7,\n          \"CATEGORIES\": [],\n          \"categoryMatches\": [],\n          \"categoryMatchesMap\": {},\n          \"tokenName\": \"INTEGER\",\n          \"isParent\": false\n        }\n      },\n      \"resyncedTokens\": [],\n      \"context\": {\n        \"ruleStack\": [\n          \"SparqlDoc\",\n          \"QueryUnit\",\n          \"Query\",\n          \"SelectQuery\",\n          \"SelectClause\"\n        ],\n        \"ruleOccurrenceStack\": [\n          0,\n          0,\n          0,\n          0,\n          0\n        ]\n      }\n    }\n  ],\n  \"cst\": {\n    \"name\": \"SparqlDoc\",\n    \"children\": {\n      \"Prologue\": [\n        {\n          \"name\": \"Prologue\",\n          \"children\": {\n            \"PrefixDecl\": [\n              {\n                \"name\": \"PrefixDecl\",\n                /* . . . etc. . . . */\n```\n\n## Full API\n\nCheck out our comprehensive [API docs](https://stardog-union.github.io/millan/) for more information.\n\n## Contributing\n\n### Prerequisites\n\n- [Node.js](https://nodejs.org/) 8+.\n- [Yarn](https://yarnpkg.com/en/) 1.0.0+.\n\n### Installing\n\nAfter cloning the repo, in the project's root, run:\n\n```bash\nyarn install\n```\n\n### Developing\n\nAll development-relevant code, including tests, is located in the repo's `src/`\ndirectory. All of this code should be written in TypeScript. After making\nchanges, you will want to run our test scripts (run by Jest), like so:\n\n```bash\nyarn test\n```\n\nStylistic conventions and formatting are enforced via `tslint` and `prettier`,\nwhich means you don't really need to worry about those things yourself\n(both the linter and `prettier` will check and format your code automatically\non any commit). If you'd like to force `prettier` to format your code (a la\n`gofmt`), just run:\n\n```bash\nyarn format\n```\n\nTo have the project run a lint check, tests, and compilation while you make\nchanges, simply run (and leave running):\n\n```bash\nyarn dev\n```\n\n### Pull Requests\n\nThe preferred way of making a pull request is as follows:\n\n1. Create an issue for the PR, if there isn't one already.\n2. Branch off of master with a branch name that begins with either 'feature/'\nor 'bug/', and make the rest of the branch name include the issue number and\na brief description of the branch (e.g., `bug/104-turtle-prefix-error`).\n3. Do your work on that branch, push the branch when it is ready, and create\na PR.\n\nIf you don't have contributor rights, you will need to fork the repo before\ndoing steps 2 and 3, and then submit the PR from your fork.\n\nBe sure to run `yarn docs` before submitting your PR, so that our type docs are\nupdated accordingly.\n\n## License\n\nCopyright 2018 Stardog Union\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstardog-union%2Fmillan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstardog-union%2Fmillan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstardog-union%2Fmillan/lists"}