{"id":27608000,"url":"https://github.com/jindrichmynarz/sparql-to-csv","last_synced_at":"2025-07-25T16:35:46.894Z","repository":{"id":137387223,"uuid":"77366542","full_name":"jindrichmynarz/sparql-to-csv","owner":"jindrichmynarz","description":"Stream SPARQL results to CSV","archived":false,"fork":false,"pushed_at":"2017-11-03T12:50:46.000Z","size":46,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-22T22:23:38.432Z","etag":null,"topics":["csv","sparql"],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jindrichmynarz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2016-12-26T07:27:58.000Z","updated_at":"2021-04-13T07:02:37.000Z","dependencies_parsed_at":"2023-05-04T17:40:57.215Z","dependency_job_id":null,"html_url":"https://github.com/jindrichmynarz/sparql-to-csv","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/jindrichmynarz/sparql-to-csv","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jindrichmynarz%2Fsparql-to-csv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jindrichmynarz%2Fsparql-to-csv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jindrichmynarz%2Fsparql-to-csv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jindrichmynarz%2Fsparql-to-csv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jindrichmynarz","download_url":"https://codeload.github.com/jindrichmynarz/sparql-to-csv/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jindrichmynarz%2Fsparql-to-csv/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267031089,"owners_count":24024280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-25T02:00:09.625Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","sparql"],"created_at":"2025-04-22T22:19:57.040Z","updated_at":"2025-07-25T16:35:46.585Z","avatar_url":"https://github.com/jindrichmynarz.png","language":"Clojure","funding_links":[],"categories":["Other Awesome","Misc"],"sub_categories":["BBedit"],"readme":"# sparql-to-csv\n\nA command-line tool to stream SPARQL results to CSV. The tool is primarily intended to support data preparation for analyses that require tabular input. It helps you avoid writing ad hoc scripts to piece larger tabular datasets out of results of many SPARQL queries. It allows to generate queries from [Mustache](https://mustache.github.io) templates, either to execute paged queries or to execute queries based on results of other queries. \n\n## Usage\n\nUse a [released executable](https://github.com/jindrichmynarz/sparql-to-csv/releases) or compile using [Leiningen](http://leiningen.org) and [lein-binplus](https://github.com/BrunoBonacci/lein-binplus):\n\n```sh\ngit clone https://github.com/jindrichmynarz/sparql-to-csv.git\ncd sparql-to-csv\nlein bin\n```\n\nThen you can run the created executable file to find out about the configuration options:\n \n```sh\ntarget/sparql_to_csv --help\n```\n\nExample of use:\n\n```sh\ntarget/sparql_to_csv --endpoint http://localhost:8890/sparql \\\n                     --page-size 1000 \\\n                     paged_query.mustache \u003e results.csv\n```\n\nThere are two main use cases for this tool: paged queries and piped queries.\n\n### Paged queries\n\nThe first one is paged execution of SPARQL `SELECT` queries. RDF stores often limit the number of rows a SPARQL `SELECT` query can retrieve in one go and thus avoid the load such queries impose on the store. For queries that select more results than the limit per one request their execution must be split into several requests if complete results need to be obtained. One way to partition such queries is to split them into pages delimited by `LIMIT` and `OFFSET`, indicating the size and the start index, respectively, of a page. Paging requires the results to have a deterministic order, which can be achieved by using an `ORDER BY` clause. Due to limitations of some RDF stores (see [Virtuoso's documentation on this topic](https://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksHowToHandleBandwidthLimitExceed)), the paged queries may need to contain an inner sub-`SELECT` that with an `ORDER BY` clause wrapped by an outer `SELECT` that slices a page from the ordered results using `LIMIT` and `OFFSET`, like this:\n\n```sparql\nPREFIX dbo: \u003chttp://dbpedia.org/ontology/\u003e\n\nSELECT ?person \nWHERE {\n  {\n    SELECT DISTINCT ?person\n    WHERE {\n      ?person a dbo:Person .\n    }\n    ORDER BY ?person\n  }\n}\nLIMIT 10000\nOFFSET 40000\n```\n\nIn order to run paged queries you need to provide the tool with a Mustache template to generate the queries for the individual pages. These queries must contain a `{{limit}}` and `{{offset}}` parameters, like so:\n\n```sparql\nPREFIX dbo: \u003chttp://dbpedia.org/ontology/\u003e\n\nSELECT ?person \nWHERE {\n  {\n    SELECT DISTINCT ?person\n    WHERE {\n      ?person a dbo:Person .\n    }\n    ORDER BY ?person\n  }\n}\nLIMIT {{limit}}\nOFFSET {{offset}}\n```\n\nThe `limit` is set by the `--page-size` parameter. The offset is incremented by the page size in each successive request. The execution of paged queries stops when an individual query returns empty results.\n\n### Piped queries \n\nIt may be desirable to decompose complex queries into several simpler queries to avoid limit on demanding queries due to performance. For example, for each person in a dataset we may want to retrieve its complex description. While this may be possible to achieve by using a sub-`SELECT` to page through the individual persons and an outer `SELECT` to compose their descriptions, such query would be more demanding since it both sorts the persons and selects their descriptions. Consequently, it may not be possible to run such query since it would end with a time-out. Instead, this query can be decomposed into two queries. The first one selects persons in the paged manner described above, while the second one receives results of the first query one by one and fetches their descriptions.\n\nThis approach is also useful when you need to query one SPARQL endpoint using data from another SPARQL endpoint. While this is feasible using [federated queries](https://www.w3.org/TR/sparql11-federated-query), they too suffer from performance problems.\n\nPiped queries take CSV input and for each line they execute a query generated from a Mustache template that is provided with the line's data as parameters. For example, the CSV generated by the query above contains a column `person`, which can be used in a query template as `{{person}}`:\n\n```sparql\nPREFIX dbo: \u003chttp://dbpedia.org/ontology/\u003e\nPREFIX dbp: \u003chttp://dbpedia.org/property/\u003e\n\nSELECT (\u003c{{person}}\u003e AS ?person) ?name ?birthDate ?deathDate\nWHERE {\n  \u003c{{person}}\u003e dbp:name ?name ;\n    dbo:birthDate ?birthDate .\n  OPTIONAL {\n    \u003c{{person}}\u003e dbo:deathDate ?deathDate .\n  }\n}\n```\n\nThe input CSV must have a header with column names. In order to be usable in Mustache template, the column names in the input CSV can contain only ASCII characters, `?`, `!`, `/`, `.`, or `-`. For example, `right!` is allowed, while `mélangé` is not.\n\nPiped queries enable to create data processing pipelines. For instance, if the first query is stored in the `persons.mustache` file and the second query is stored as `describe_person.mustache`, then we can run them in pipeline using the following command using `--piped` to indicate that it is a piped query: \n\n```sh\nsparql_to_csv -e http://dbpedia.org/sparql persons.mustache |\n  sparql_to_csv -e http://dbpedia.org/sparql --piped describe_person.mustache\n```\n\nBy default the piped input is replaced by the output query results. However, using the `--extend` parameter extends the input with the results. Each result row is append to its input row. This allows you to combine data from multiple queries. Piped queries can be arbitrarily chained and allow joining data across many SPARQL endpoints.\n\n## License\n\nCopyright © 2016 Jindřich Mynarz\n\nDistributed under the Eclipse Public License version 1.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjindrichmynarz%2Fsparql-to-csv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjindrichmynarz%2Fsparql-to-csv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjindrichmynarz%2Fsparql-to-csv/lists"}