{"id":13503310,"url":"https://github.com/evinism/mistql","last_synced_at":"2025-03-31T02:04:02.622Z","repository":{"id":39666462,"uuid":"392826077","full_name":"evinism/mistql","owner":"evinism","description":"A query / expression language for performing computations on JSON-like structures. Tuned for clientside ML feature extraction.","archived":false,"fork":false,"pushed_at":"2024-10-08T18:40:14.000Z","size":3564,"stargazers_count":363,"open_issues_count":22,"forks_count":17,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-24T00:11:21.944Z","etag":null,"topics":["expression-language","feature-engineering","feature-extraction","hacktoberfest","javascript","json","machine-learning","mistql","python","query","typescript"],"latest_commit_sha":null,"homepage":"https://mistql.com","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/evinism.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-04T21:14:11.000Z","updated_at":"2025-02-26T13:56:15.000Z","dependencies_parsed_at":"2024-10-31T23:30:36.033Z","dependency_job_id":"c3ab5631-13ab-4766-9d16-a2fb4e8798cc","html_url":"https://github.com/evinism/mistql","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evinism%2Fmistql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evinism%2Fmistql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evinism%2Fmistql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evinism%2Fmistql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/evinism","download_url":"https://codeload.github.com/evinism/mistql/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246403911,"owners_count":20771530,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expression-language","feature-engineering","feature-extraction","hacktoberfest","javascript","json","machine-learning","mistql","python","query","typescript"],"created_at":"2024-07-31T22:02:45.951Z","updated_at":"2025-03-31T02:04:02.598Z","avatar_url":"https://github.com/evinism.png","language":"TypeScript","readme":"\u003cimg width=\"1022\" alt=\"Screen Shot 2022-05-28 at 11 46 17 AM\" src=\"https://user-images.githubusercontent.com/1979887/170838934-0553c383-d517-4158-8d29-589cd089ec28.png\"\u003e\n\n[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/evinism/mistql/blob/main/LICENSE) \n![Python](https://github.com/evinism/mistql/actions/workflows/python.yml/badge.svg) \n![Node.js](https://github.com/evinism/mistql/actions/workflows/node.js.yml/badge.svg) \n[![npm version](https://img.shields.io/npm/v/mistql.svg?style=flat)](https://www.npmjs.com/package/mistql)\n[![npm version](https://img.shields.io/pypi/v/mistql.svg?style=flat)](https://pypi.org/project/mistql/)\n\nMistQL is a query language for JSON-like structures, built for embedding across multiple domains. It supports\nlogic for querying and manipulating JSON-like data in a simple, readable manner. MistQL serves as a powerful \ncommon expression language across multiple implementations.\n\nFor more detailed usage information, please visit MistQL's docs site.\n\n[Join the Discord!!](https://discord.gg/YupxqvE5Jk)\n\n### Links\n\n- [MistQL's doc site](https://www.mistql.com/)\n- [Getting Started](https://www.mistql.com/docs/intro)\n- [Try it out!](https://www.mistql.com/tryitout)\n- [Functions](https://www.mistql.com/docs/reference/functions)\n- [Reference](https://www.mistql.com/docs/reference/overview)\n\n# At A Glance\n\nMistQL is an embedded query language.\n\n```js\nimport mistql from 'mistql';\n\nconst animals = [\n  {name: 'charlie', species: 'dog'},\n  {name: 'mulberry', species: 'cat'},\n]\n\nconst cats = mistql.query('@ | filter species == \"dog\" | count')\n```\n\nThe primary power of MistQL comes from its strong cross-platform behavior semantics. For example, the following have the same behavior:\n\n```py \n# Python\nimport mistql\nimport json\n\nquery = 'events | groupby type | keys'\nprint(mistql.query(query, json.loads(data)))\n```\n\n```js\n// JavaScript\nimport mistql from 'mistql'\n\nconst query = 'events | groupby type | keys';\nconsole.log(mistql.query(query, JSON.parse(data)))\n```\n\n# Developing MistQL\n\nContributions to MistQL are very welcome!\n\nAs MistQL is a small project, there are no formatting requirements for either issues or pull requests.\n\nIf you're planning on making a new implementation, [ping the discord](https://discord.gg/YupxqvE5Jk) and we'll coordinate!\n\nFor an example PR that adds a function to MistQL, refer to [this PR](https://github.com/evinism/mistql/pull/175) as an example of a new function with moderate complexity. \n\n### Code workflow\n\nCode contributions to MistQL should roughly follow standard open source workflows:\n\n1. Fork the project\n2. Make code changes on your fork of the project.\n3. (if necessary) Pull upstream to bring in new changes\n4. Submit a pull request to MistQL's `main` branch.\n5. (if necessary) Implement changes requested by maintainers.\n6. Wait for the branch to be accepted and merged by maintainers!\n\n### MistQL standard\n\nNo MistQL standard yet exists, but we're aiming for the `0.5.0` release of `mistql` as a standardizable language. After the `0.5.0` release, we will create a language specification, separate from any implementation.\n\nIn the meantime, we're actually pretty close.\n\nWe have a [Lark grammar](https://github.com/evinism/mistql/blob/main/py/mistql/grammar.lark) \nwhich defines the language's syntax. This is likely the final grammar that will be formalized \ninto ABNF, although it is possible that we may need to fix minor issues before `0.5.0`. I\nexpect this to barely change, if at all.\n\nAdditionally, our language-independent test suite is rather extensive and forms the de-facto\nstandard of behaviors, as shared by both Python and JavaScript. While not strictly formalized, \nthe tests and the docs together form a cohesive body of behaviors, that, except for a few \nminuitae, is of sufficient detail to be standardized.\n\n## Directory Structure\n\nMistQL's directory structure is a monorepo, currently consisting of these main directories:\n\n1. `/docs`: Documentation Site (hosted at [mistql.com](https://www.mistql.com/))\n2. `/js`: MistQL's browser implementation (e.g. `mistql` on npm).\n3. `/py`: MistQL's python implementation (e.g. `mistql` on pypi).\n4. `/shared`: Shared assets between all implementation. Contains the language-independent test suite.\n\n## Developing for the docs site\n\nDocs are built via a fairly standard [Docusaurus 2](https://docusaurus.io/) implementation. Please follow Docusaurus's docs for developing for the Docs site.\n\n## Developing for `mistql` on npm\n\n`mistql` is written exclusively using typescript. Additionally, `mistql` uses `yarn` for dependency management, versioning, and uploading. JS-specific tests are stored alongside their implementation, using the suffix `.spec.ts`. Tests that describe the language itself are written in a language agnostic JSON format in the `/shared` directory. Writing tests for all feature additions and bug fixes is strongly encouraged.\n\nFor all major improvements, it is strongly encouraged to run `yarn bundlesize` to estimate gzipped impact of MistQL on a browser. MistQL for the browser should, in general, remain relatively close to 5kb.\n\nThe directory structure is relatively flat, except for the single `src/builtins` folder, which contains the implementation of all of MistQL's internal functions.\n\n## Developing for `mistql` on pypi\n\n`mistql` is a fairly standard python package managed with [poetry](https://python-poetry.org/).\n\nTests can be run using pytest, e.g. `poetry run pytest` from within the `/py` directory.\n","funding_links":[],"categories":["TypeScript","Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevinism%2Fmistql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevinism%2Fmistql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevinism%2Fmistql/lists"}