{"id":26190106,"url":"https://github.com/zazuko/xrm-xml-workflow","last_synced_at":"2026-03-10T17:11:15.715Z","repository":{"id":74818033,"uuid":"573337386","full_name":"zazuko/xrm-xml-workflow","owner":"zazuko","description":null,"archived":false,"fork":false,"pushed_at":"2024-09-02T14:45:19.000Z","size":270,"stargazers_count":1,"open_issues_count":2,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-12T00:52:30.354Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zazuko.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-02T08:17:50.000Z","updated_at":"2025-02-25T11:49:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"b845068c-5783-4e8e-9ad0-d5bd7349d7e4","html_url":"https://github.com/zazuko/xrm-xml-workflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zazuko/xrm-xml-workflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zazuko%2Fxrm-xml-workflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zazuko%2Fxrm-xml-workflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zazuko%2Fxrm-xml-workflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zazuko%2Fxrm-xml-workflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zazuko","download_url":"https://codeload.github.com/zazuko/xrm-xml-workflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zazuko%2Fxrm-xml-workflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30344131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T15:55:29.454Z","status":"ssl_error","status_checked_at":"2026-03-10T15:54:58.440Z","response_time":106,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-12T00:52:33.012Z","updated_at":"2026-03-10T17:11:15.682Z","avatar_url":"https://github.com/zazuko.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Template to transform JSON, XML or CSV to RDF\n\nThis is a running example of a [Barnard59](#about-barnard59) pipeline that convert JSON, XML or CSV files to RDF and (optionally) upload it into a store.\n\nThis repository contains:\n\n- Sample [XRM mappings](./mappings) for JSON, XML and CSV and some [example files](./input)\n- A [pipeline](./pipelines/main.ttl) that converts the input file to RDF\n- A default GitHub Action configuration that runs the pipeline and creates an artifact for download\n\n## Quick start\n\n* Install the dependencies\n\n```markdown\nnpm install\n```\n\n* Run the barnard59 workflow to produce a file, for example:\n\n```sh\nnpm run json-to-file\n```\n\nYou will see the output of the pipeline process, if all went well, the RDF file was generated at `./output/transformed.nt`\n\nWhat the pipeline did is to use a [mapping](./src-gen/mapping-json.carml.ttl) to transform an [input file](./input/example.json) using a [CARML service](https://github.com/zazuko/carml-service) instance.\n\nFrom now own you can [write your mapping](#writing-your-mappings) and start producing RDF. You can choose if you produce local files, or you upload them [into a store](#upload-to-the-store). \n\n## Using the template\n\nThis is a GitHub template repository. It will not be declared as \"fork\" once you click on the `Use this template` button above. Simply do that, start adding your data sources and create the mappings accordingly.\n\n## Writing your mappings\n\nYou can choose to write the mappings by hand in the `src-gen` directory, or write XRM through a [plugin](https://zazuko.com/products/expressive-rdf-mapper/). This is by far a better experience since it provides autocomplete, code-lookup and a type-safety. The XRM will be easier to maintain in the future.\n\nSet up a mapping to do a transformation:\n\n1. Copy the source file to the `input` directory\n2. Create/adjust the XRM files in the [mappings](./mappings) directory, the plugin will produce a CARML or RML mapping in the [src-gen](./src-gen) to transform the data.\n3. Execute one of the run-scripts to convert your data.\n\nMake sure to commit the `input` and `src-gen` directories if you want to build it using GitHub Actions. Also is convenient to keep track of the `mappings` directory.\n\nTip: When developing a mapping to RDF sometimes is easier to start with a small sample of what you want to transform. When the mapping is ok you can use it to convert the whole dataset.\n\nSee [Further reading](#further-reading) for more information about the XRM mapping language.\n\n## Run the pipeline\n\nThe default pipeline can be run with `npm run json-to-file`. It will:\n\n- Read the JSON input file (default: `input/example.json`)\n- Convert it to RDF\n- Write it into a file as N-Triples (default: `output/transformed.nt`)\n\nTo convert XML input, run `npm run xml-to-file`.\n\nTo convert CSV input, run `npm run csv-to-file`.\n\n\n### Upload to the store\n\nThere are additional pipelines configured in `package.json`:\n\n* `file-to-store(-dev)`: Uploads the generated output file to an RDF store via SPARQL Graph Store Protocol\n* `json-to-store(-dev)`: Directly uploads to an RDF store (direct streaming in the pipeline) via SPARQL Graph Store Protocol\n\nIf you want to test the upload to an RDF store, a default [Apache Jena Fuseki](https://jena.apache.org/index.html) installation with a database `data` on port `3030` should work out of the box.\n\n### Configure the pipeline\n\nPipeline configuration is done via environment variables and/or adjusting default variables in the pipeline itself. If you want to pass another default, have a look at the `--variable=XYZ` samples in `package.json` or consult the [barnard59 documentation](https://github.com/zazuko/barnard59/blob/master/packages/cli/README.md#passing-arguments-to-the-pipeline). If you want to adjust it in the pipeline, open the file [pipelines/main.ttl](pipelines/main.ttl) and edit `\u003cdefaultVars\u003e ...`.\n\n\n## Watch script (dev-mode)\n\nWhile developing or changing the mapping, it might be useful to run the transformation automatically whenever the mapping file changes. This can be helpful in order to get quick feedback and validate modifications incrementally.\n\nThere is a simple script that shows how this could be done: `src-gen/watch.sh`\n\nRunning that script is alternative to running the pipeline as described above. The watch script requires that `inotifywait` is installed on the system. The script downloads and invokes carml-jar directly (different from the pipeline which is using carml-service).\n\n\n## About barnard59\n\nThis template is built on top of our [Zazuko](https://zazuko.com/) [barnard59](https://github.com/zazuko/barnard59) pipelining system. It is a [Node.js](https://nodejs.org) based, fully configurable pipeline framework aimed at creating RDF data out of various data sources. Unlike many other data pipelining systems, barnard59 is configured instead of programmed. In case you need to do pre- or post-processing, you can implement additional pipeline steps written in JavaScript.\n\nbarnard59 is streaming and can be used to convert very large data sets with a small memory footprint.\n\n## Other template repositories\n\nWe provide additional template repositories:\n\n* [xrm-r2rml-workflow](https://github.com/zazuko/xrm-r2rml-workflow):  A template repository for converting complete relational databases to RDF using the R2RML specification and Ontop as mapper.\n* [xrm-csvw-workflow](https://github.com/zazuko/xrm-csvw-workflow):  A template repository for converting CSV files to RDF using a CARML mapping.\n\n## Further reading\n\n* [Expressive RDF Mapping Language (XRM)](https://zazuko.com/products/expressive-rdf-mapper/) and the [documentation](https://github.com/zazuko/expressive-rdf-mapper) for details about the domain-specific language (DSL).\n* [CSV on the Web: A Primer](https://www.w3.org/TR/tabular-data-primer/): Introduction to the CSVW mapping language, which is generated by XRM and consumed by barnard59. This is only as a reference, you do not have to learn about it, XRM generates that for you.\n* [SPARQL 1.1 Graph Store HTTP Protocol](https://www.w3.org/TR/sparql11-http-rdf-update/): The SPARQL Graph Store specification used to upload data to an RDF store like Apache Jena Fuseki\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzazuko%2Fxrm-xml-workflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzazuko%2Fxrm-xml-workflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzazuko%2Fxrm-xml-workflow/lists"}