{"id":25389985,"url":"https://github.com/jvalue/jayvee","last_synced_at":"2025-10-23T22:05:56.802Z","repository":{"id":150671333,"uuid":"546589214","full_name":"jvalue/jayvee","owner":"jvalue","description":"Jayvee is a domain-specific language and runtime for automated processing of data pipelines","archived":false,"fork":false,"pushed_at":"2025-03-11T10:25:47.000Z","size":11030,"stargazers_count":176,"open_issues_count":50,"forks_count":15,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-03T05:13:01.260Z","etag":null,"topics":["data-engineering","data-pipeline","data-science","domain-specific-language","etl-pipeline","typescript"],"latest_commit_sha":null,"homepage":"https://jvalue.github.io/jayvee/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jvalue.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSES/AGPL-3.0-only.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-06T10:15:54.000Z","updated_at":"2025-03-26T21:40:11.000Z","dependencies_parsed_at":"2023-10-02T05:15:32.440Z","dependency_job_id":"95df17fe-313c-41e1-8977-08250dcbc959","html_url":"https://github.com/jvalue/jayvee","commit_stats":null,"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvalue%2Fjayvee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvalue%2Fjayvee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvalue%2Fjayvee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jvalue%2Fjayvee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jvalue","download_url":"https://codeload.github.com/jvalue/jayvee/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248493034,"owners_count":21113187,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","data-pipeline","data-science","domain-specific-language","etl-pipeline","typescript"],"created_at":"2025-02-15T14:23:42.476Z","updated_at":"2025-10-23T22:05:51.758Z","avatar_url":"https://github.com/jvalue.png","language":"TypeScript","readme":"\u003c!--\nSPDX-FileCopyrightText: 2023 Friedrich-Alexander-Universitat Erlangen-Nurnberg\n\nSPDX-License-Identifier: AGPL-3.0-only\n--\u003e\n\n![Jayvee Mascot](./assets/mascot1.png)\n\n# Jayvee\n\nJayvee is a domain-specific language (DSL) for automated processing of data pipelines.\nThe Jayvee interpreter allows executing such data pipelines on local machines.\nData engineers can use Jayvee and its interpreter to clean and preprocess data for later activities like data science or machine learning.\n\n[![Official Docs](assets/docs-banner.png)](https://jvalue.github.io/jayvee)\n\n## Roadmap\n\nExplore a glimpse of our upcoming features in the following list. This overview is broad and subject to evolution. We're excited to share our vision of the exciting journey ahead, and we invite you to accompany us on this adventure!\n\n- \u0026#9989; Blocks and pipes \n- \u0026#9989; Simple value types and constraints\n- \u0026#9989; Natively support table-based data\n- \u0026#9989; Column-based transformations\n- \u0026#9989; Describe blocks via builtin block types in Jayvee\n- \u0026#9989; Compose logic of multiple blocks via composite block types\n- \u0026#9989; Multi-file Jayvee to distribute programs over multiple files\n- \u0026#9989; Jayvee formatter\n- \u0026#8987; Improve the syntax of value types (see [RFC 0014](https://github.com/jvalue/jayvee/pull/409))\n- \u0026#8987; Reusable libraries (with a package manager)\n- \u0026#8987; Further extractors and sinks\n- \u0026#129300; Composite value types (with multiple fields)\n- \u0026#129300; Natively support tree data (XML, JSON)\n- \u0026#129300; Valuetypes parsers (to read and write different formats)\n- \u0026#129300; Customizable invalid value handling (default value, average, median, interpolation, ...)\n- \u0026#129300; VSCode Debugger\n- \u0026#129300; Block types with multiple ports (e.g., for merging different data)\n\n\nAnything missing, or you have ideas how some of the items on the list could be approached?\nFeel free to create and issue and share your thoughts with us!\n\nYou like the project and our vision? Then we'd appreciate your star! \u0026#11088;\n\n\n## Contribute\n\nIn case you would like to contribute to Jayvee, please have a look at our [contribution guide](CONTRIBUTING.md).\n\n## Development Quickstart\n\n1. Run `npm ci` to install the dependencies.\n2. Run `npm run generate` to generate TypeScript code from the Jayvee grammar definition and the standard library.\n3. Run `npm run build` to compile all projects.\n4. In Visual Studio Code, press `F5` to open a new window with the Jayvee extension loaded.\n5. Create a new file with a `.jv` file name suffix or open an existing file in the directory `example`.\n6. Verify that syntax highlighting, validation, completion etc. are working as expected.\n7. Run `node dist/apps/interpreter/main.js` to see options for the CLI of the interpreter; `node dist/apps/interpreter/main.js \u003cfile\u003e` interprets a given `.jv` file.\n\nIn case you run into problems, make sure to use the current LTS version of Node.js and npm.\n\n## Projects overview\n\n| Name                                                              | Description                                                                                                                      | NPM package                                                                                                          |\n| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |\n| [`language-server`](./libs/language-server)                       | Jayvee language definition and language server implementation                                                                    | [@jvalue/jayvee-language-server](https://www.npmjs.com/package/@jvalue/jayvee-language-server)                       |\n| [`interpreter`](./apps/interpreter)                               | Command line tool for interpreting Jayvee files                                                                                  | [@jvalue/jayvee-interpreter](https://www.npmjs.com/package/@jvalue/jayvee-interpreter)                               |\n| [`language-server-web-worker`](./apps/language-server-web-worker) | Ready-to-use Jayvee language server, bundled as a [Web Worker](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API) | [@jvalue/jayvee-language-server-web-worker](https://www.npmjs.com/package/@jvalue/jayvee-language-server-web-worker) |\n| [`vs-code-extension`](./apps/vs-code-extension)                   | Visual Studio Code extension for editing Jayvee files                                                                            | -                                                                                                                    |\n| [`docs`](./apps/docs)                                             | Website for Jayvee user documentation                                                                                            | -                                                                                                                    |\n| [`monaco-editor`](./libs/monaco-editor)                           | React component for editing Jayvee files                                                                                         | [@jvalue/jayvee-monaco](https://www.npmjs.com/package/@jvalue/jayvee-monaco)                                         |\n| [`execution`](./libs/execution)                                   | Shared code for Jayvee extensions and the interpreter                                                                            | -                                                                                                                    |\n| [`extensions/std`](./libs/extensions/std)                         | Standard Jayvee extension consisting of the extensions below                                                                     | -                                                                                                                    |\n| [`extensions/rdbms`](./libs/extensions/rdbms)                     | Jayvee extension for relational databases                                                                                        | -                                                                                                                    |\n| [`extensions/tabular`](./libs/extensions/tabular)                 | Jayvee extension for tabular data                                                                                                | -                                                                                                                    |\n\n## Scripts\n\n### Building all projects\n\n```bash\nnpm run build\n```\n\n### Linting all projects\n\n```bash\nnpm run lint\n```\n\n### Formatting project files via Nx\n\n```bash\nnpm run format\n```\n\n### Testing all projects\n\n```bash\nnpm run test\n```\n\n### Generating TypeScript code from the grammar definition\n\n```bash\nnpm run generate\n```\n\n### Examples\n\n#### Load data about cars into a local SQLite DB\n\n```bash\nnpm run example:cars\n```\n\n#### Load GTFS data into a local SQLite DB\n\n```bash\nnpm run example:gtfs\n```\n\n#### Load data about electric vehicles into local Postgres and SQLite DBs\n\n1. Start the Postgres database locally\n\n```bash\ndocker compose -f ./example/docker-compose.example.yml up\n```\n\n2. Run the Jayvee model\n\n```bash\nnpm run example:vehicles\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjvalue%2Fjayvee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjvalue%2Fjayvee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjvalue%2Fjayvee/lists"}