{"id":23449455,"url":"https://github.com/1biot/FiQueLa","last_synced_at":"2025-08-26T09:31:45.535Z","repository":{"id":268853605,"uuid":"905652958","full_name":"1biot/uniquel","owner":"1biot","description":"UniQueL (Universal Query Language) is a PHP library for easy manipulation of JSON or Yaml or Neon data.","archived":false,"fork":false,"pushed_at":"2024-12-23T09:32:38.000Z","size":78,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-23T10:27:51.648Z","etag":null,"topics":["json","neon","parser","php","query","sql","yaml"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/1biot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-19T09:03:30.000Z","updated_at":"2024-12-23T09:32:42.000Z","dependencies_parsed_at":"2024-12-23T10:27:59.708Z","dependency_job_id":null,"html_url":"https://github.com/1biot/uniquel","commit_stats":null,"previous_names":["1biot/jql","1biot/uniquel"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2Funiquel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2Funiquel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2Funiquel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2Funiquel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/1biot","download_url":"https://codeload.github.com/1biot/uniquel/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230985440,"owners_count":18310838,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","neon","parser","php","query","sql","yaml"],"created_at":"2024-12-23T23:18:36.647Z","updated_at":"2025-08-26T09:31:45.526Z","avatar_url":"https://github.com/1biot.png","language":"PHP","readme":"# FiQueLa: File Query Language \n\n\u003e _[fi-kju-ela]_\n\n![Packagist Version](https://img.shields.io/packagist/v/1biot/fiquela)\n![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/1biot/fiquela/ci.yml)\n![Packagist Dependency Version](https://img.shields.io/packagist/dependency-v/1biot/fiquela/php)\n![Packagist License](https://img.shields.io/packagist/l/1biot/fiquela)\n\n![Static Badge](https://img.shields.io/badge/PHPUnit-tests%3A_228-lightgreen)\n![Static Badge](https://img.shields.io/badge/PHPUnit-asserts%3A_801-lightgreen)\n![Static Badge](https://img.shields.io/badge/PHPStan_6-OK-lightgreen)\n![Static Badge](https://img.shields.io/badge/PHPStan_7|8-15_errors-orange)\n\n**F**i**Q**ue**L**a is a powerful PHP library that brings SQL-inspired querying capabilities to structured data formats\nlike **XML**, **CSV**, **JSON**, **NDJSON**, **YAML** and **NEON**. Designed for simplicity and modularity, it allows you to filter,\njoin, and aggregate data with a familiar and efficient syntax. Whether you're working with large datasets or integrating\nvarious sources, **F**i**Q**ue**L**a provides a seamless way to manipulate and explore your data.\n\n**Features**:\n\n- 📂 **Supports multiple formats**: Work seamlessly with XML, CSV, JSON, NDJSON, YAML, and NEON.\n- 🛠️ **SQL-inspired syntax**: Perform `SELECT`, `JOIN`, `WHERE`, `GROUP BY`, `ORDER BY` and more.\n- ✍️ **Flexible Querying**: Write SQL-like strings or use the fluent API for maximum flexibility.\n- 📊 **Advanced functions**: Access features like `SUM`, `COUNT`, `GROUP_CONCAT`, `ARRAY_MERGE`, `DATE_FORMAT` and many more.\n- 🚀 **Efficient with Large Files**: Optimized for processing JSON, XML, and CSV files with tens of thousands of rows using stream processing.\n- 🧑‍💻 **Developer-Friendly**: Map results to DTOs for easier data manipulation.\n- ⭐ **Unified API across all supported formats**: Use a consistent API for all your data needs.\n\n**Table of Contents**:\n\n- [Overview](#1-overview)\n- [Installation](#2-installation)\n- [Supported Formats](#3-supported-formats)\n- [Getting Started](#4-getting-started)\n- [Documentation](#5-documentation)\n  - [Opening Files](docs/opening-files.md)\n  - [Fluent API](docs/fluent-api.md)\n  - [File Query Language](docs/file-query-language.md)\n  - [Fetching Data](docs/fetching-data.md)\n  - [Query Life Cycle](docs/query-life-cycle.md)\n  - [Query Inspection and Benchmarking](docs/query-inspection-and-benchmarking.md)\n- [Examples](#6-examples)\n- [Knowing issues](#7-knowing-issues)\n- [Planning Features](#8-planning-features)\n- [Contributions](#9-contributions)\n\n## 1. Overview\n\nWhy limit SQL to databases when it can be just as effective for querying structured data? **F**i**Q**ue**L**a (File Query Language)\nbrings the power of SQL to your files. Whether you're working with **XML**, **CSV**, **JSON**, **NDJSON**, **YAML** or **NEON**,\n**F**i**Q**ue**L**a enables you to interact with these formats using familiar SQL syntax.\n\nKey highlights:\n- **Universal Querying**: Use SQL-like queries to filter, sort, join, and aggregate data across multiple file types.\n- **Data Formats Support**: Seamlessly work with JSON, XML, CSV, YAML, and more.\n- **Powerful Features**: Access advanced SQL features like `GROUP BY`, `HAVING`, and functions for data transformation directly on your file-based datasets.\n- **Developer-Friendly**: Whether you're a beginner or an experienced developer, FiQueLa offers a simple and consistent API for all your data needs.\n- **Flexible Integration**: Ideal for scenarios where data lives in files rather than traditional databases.\n- **SQL-Like Strings**: Write and execute SQL-like string queries directly, providing an alternative to fluent syntax for greater flexibility and familiarity.\n\nUse **F**i**Q**ue**L**a to:\n- Simplify data extraction and analysis from structured files.\n- Combine data from multiple sources with ease.\n- Create lightweight data processing pipelines without a full-fledged database.\n\n**F**i**Q**ue**L**a empowers developers to unlock the potential of file-based data with the familiar and expressive language of SQL.\n\n## 2. Installation\n\nInstall via [Composer](https://getcomposer.org/):\n\n```bash\ncomposer require 1biot/fiquela\n```\n\nInstall packages for optional features:\n\n```bash\ncomposer require league/csv halaxa/json-machine symfony/yaml nette/neon tracy/tracy\n```\n\n- **`league/csv`**: Required for CSV file support.\n- **`halaxa/json-machine`**: Required for JSON stream support.\n- **`symfony/yaml`**: Required for YAML file support.\n- **`nette/neon`**: Required for NEON file support.\n- **`tracy/tracy`**: Optional for using Debugger\n\n## 3. Supported Formats\n\n| Format      | Name                    | Class                   | File Support | String Support | Dependencies                                    |\n|-------------|-------------------------|-------------------------|--------------|----------------|-------------------------------------------------|\n| `csv`       | CSV                     | `FQL\\Stream\\Csv`        | ✅            | ❌              | `league/csv`                                    |\n| `xml`       | XML                     | `FQL\\Stream\\Xml`        | ✅            | ❌              | `ext-xmlreader`, `ext-simplexml`, `ext-libxml`  |\n| `jsonFile`  | JSON Stream             | `FQL\\Stream\\JsonStream` | ✅            | ❌              | `halaxa/json-machine`                           |\n| `json`      | JSON (json_decode)      | `FQL\\Stream\\Json`       | ✅            | ✅              | `ext-json`                                      |\n| `ndJson`    | Newline Delimited JSON  | `FQL\\Stream\\NDJson`     | ✅            | ❌              | `ext-fileinfo`                                  |\n| `yaml`      | YAML                    | `FQL\\Stream\\Yaml`       | ✅            | ✅              | `symfony/yaml`                                  |\n| `neon`      | NEON                    | `FQL\\Stream\\Neon`       | ✅            | ✅              | `nette/neon`                                    |\n\n### Directory provider\n\nIs special provider `FQL\\Stream\\Dir` class. It allows you to use directory as a source.\nYou can query all files recursively by queries.\n\n## 4. Getting Started\n\nHere’s a quick example of how **F**i**Q**ue**L**a can simplify your data queries:\n\n```php\nuse FQL\\Enum;\nuse FQL\\Query;\n\n$results = Query\\Provider::fromFileQuery('(./path/to/file.xml).SHOP.SHOPITEM')\n    -\u003eselectAll()\n    -\u003ewhere('EAN', Enum\\Operator::EQUAL, '1234567891011')\n    -\u003eor('PRICE', Enum\\Operator::LESS_THAN_OR_EQUAL, 200)\n    -\u003eorderBy('PRICE')-\u003edesc()\n    -\u003elimit(10)\n    -\u003eexecute()\n    -\u003efetchAll();\n\nprint_r(iterator_to_array($results));\n```\n\nOr using the FQL syntax:\n\n```php\nuse FQL\\Query;\n\n$query = \u003c\u003c\u003cFQL\n    SELECT *\n    FROM (./path/to/file.xml).SHOP.SHOPITEM\n    WHERE\n        EAN = \"1234567891011\"\n        OR PRICE \u003c= 200\n    ORDER BY PRICE DESC\n    LIMIT 10\nFQL;\n$results = Query\\Provider::fql($query)\n    -\u003eexecute()\n    -\u003efetchAll();\n\nprint_r(iterator_to_array($results));\n````\n\n\nOutput:\n\n```php\nArray\n(\n    [0] =\u003e Array\n        (\n            [NAME] =\u003e \"Product 1\"\n            [EAN] =\u003e \"1234567891011\"\n            [PRICE] =\u003e 300.00\n        )\n    [1] =\u003e Array\n        (\n            [NAME] =\u003e \"Product 2\"\n            [EAN] =\u003e \"1234567891012\"\n            [PRICE] =\u003e 150.00\n        )\n    [2] =\u003e Array\n        (\n            [NAME] =\u003e \"Product 3\"\n            [EAN] =\u003e \"1234567891013\"\n            [PRICE] =\u003e 150.00\n        )\n    ...\n)\n```\n\n## 5. Documentation\n\nFor more details about **F**i**Q**ue**L**a and her capabilities, explore the documentation sections.\n\n- [Opening Files](docs/opening-files.md)\n- [Fluent API](docs/fluent-api.md)\n- [File Query Language](docs/file-query-language.md)\n- [Fetching Data](docs/fetching-data.md)\n- [Query Life Cycle](docs/query-life-cycle.md)\n- [Query Inspection and Benchmarking](docs/query-inspection-and-benchmarking.md)\n\n\n## 6. Examples\n\nCheck the examples and run them using Composer. All examples uses `\\FQL\\Query\\Debugger` and methods\n`inspectQuery`, `inspectSql`, `inspectStreamSql` or `benchmarkQuery` to show the results.\n\n```bash\ncomposer examples\n# or\ncomposer example:csv\ncomposer example:join\ncomposer example:json\ncomposer example:neon\ncomposer example:sql\ncomposer example:xml\ncomposer example:yaml\n```\n\nCheck step **Examples** at [actions](https://github.com/1biot/fiquela/actions/runs/12992585648/job/36232767074) or run\n`composer example:csv` and output will look like this:\n\n```\n=========================\n### Debugger started: ###\n=========================\n\u003e Memory usage (MB): 1.196 (emalloc)\n\u003e Memory peak usage (MB): 1.5637 (emalloc)\n------------------------------\n\u003e Execution time (s): 5.9E-5\n\u003e Execution time (ms): 0.059\n\u003e Execution time (µs): 59\n\u003e Execution memory peak usage (MB): 0\n=========================\n### Inspecting query: ###\n=========================\n==================\n### SQL query: ###\n==================\n\u003e SELECT\n\u003e   ean ,\n\u003e   defaultCategory ,\n\u003e   EXPLODE(defaultCategory, \" \u003e \") AS categoryArray ,\n\u003e   price ,\n\u003e   ROUND(price, 2) AS price_rounded ,\n\u003e   MOD(price, 100) AS modulo_100 ,\n\u003e   MOD(price, 54) AS modulo_54\n\u003e FROM [csv](products-w-1250.csv, windows-1250, \";\").*\n\u003e GROUP BY defaultCategory\n\u003e ORDER BY defaultCategory DESC\n================\n### Results: ###\n================\n\u003e Result class: FQL\\Results\\InMemory\n\u003e Results size memory (KB): 3.55\n\u003e Result exists: true\n\u003e Result count: 15\n========================\n### Fetch first row: ###\n========================\narray (7)\n   'ean' =\u003e 5010232964877\n   'defaultCategory' =\u003e 'Testování \u003e Drogerie'\n   'categoryArray' =\u003e array (2)\n   |  0 =\u003e 'Testování'\n   |  1 =\u003e 'Drogerie'\n   'price' =\u003e 121.0\n   'price_rounded' =\u003e 121.0\n   'modulo_100' =\u003e 21.0\n   'modulo_54' =\u003e 13.0\n\n\u003e\u003e\u003e SPLIT TIME \u003c\u003c\u003c\n\u003e Memory usage (MB): 2.5525 (emalloc)\n\u003e Memory peak usage (MB): 2.6933 (emalloc)\n------------------------------\n\u003e Execution time (s): 0.035236\n\u003e Execution time (ms): 35.236\n\u003e Execution time (µs): 35236\n\u003e Execution memory peak usage (MB): 1.1296\n========================\n### Benchmark Query: ###\n========================\n\u003e 2 500 iterations\n==================\n### SQL query: ###\n==================\n\u003e SELECT\n\u003e   ean ,\n\u003e   defaultCategory ,\n\u003e   EXPLODE(defaultCategory, \" \u003e \") AS categoryArray ,\n\u003e   price ,\n\u003e   ROUND(price, 2) AS price_rounded ,\n\u003e   MOD(price, 100) AS modulo_100 ,\n\u003e   MOD(price, 54) AS modulo_54\n\u003e FROM [csv](products-w-1250.csv, windows-1250, \";\").*\n\u003e GROUP BY defaultCategory\n\u003e ORDER BY defaultCategory DESC\n=========================\n### STREAM BENCHMARK: ###\n=========================\n\u003e Size (KB): 2.65\n\u003e Count: 15\n\u003e Iterated results: 37 500\n\u003e\u003e\u003e SPLIT TIME \u003c\u003c\u003c\n\u003e Memory usage (MB): 2.5419 (emalloc)\n\u003e Memory peak usage (MB): 2.7163 (emalloc)\n------------------------------\n\u003e Execution time (s): 15.815553\n\u003e Execution time (ms): 15815.553\n\u003e Execution time (µs): 15815553\n\u003e Execution memory peak usage (MB): 0.023\n============================\n### IN_MEMORY BENCHMARK: ###\n============================\n\u003e Size (KB): 3.55\n\u003e Count: 15\n\u003e Iterated results: 37 500\n\u003e\u003e\u003e SPLIT TIME \u003c\u003c\u003c\n\u003e Memory usage (MB): 2.5525 (emalloc)\n\u003e Memory peak usage (MB): 2.7163 (emalloc)\n------------------------------\n\u003e Execution time (s): 0.009213\n\u003e Execution time (ms): 9.213\n\u003e Execution time (µs): 9213\n\u003e Execution memory peak usage (MB): 0\n=======================\n### Debugger ended: ###\n=======================\n\u003e Memory usage (MB): 2.5416 (emalloc)\n\u003e Memory peak usage (MB): 2.7163 (emalloc)\n------------------------------\n\u003e Final execution time (s): 15.860123\n\u003e Final execution time (ms): 15860.123\n\u003e Final execution time (µs): 15860123\n```\n\n## 7. Knowing issues\n\n- ⚠️ Functions `JOIN`, `ORDER BY` and `GROUP BY` are not memory efficient, because joining data or sorting data requires \nto load all data into memory. It may cause memory issues for large datasets. But everything else is like ⚡️.\n\n## 8. Planning Features\n\n- [x] ~~**Operator BETWEEN**: Add operator `BETWEEN` for filtering data and add support for dates and ranges.~~\n- [ ] **Next file formats**: Add next file formats [MessagePack](https://msgpack.org/), [XLSX](https://en.wikipedia.org/wiki/Excel_Open_XML_file_formats), [Parquet](https://parquet.apache.org/docs/file-format/), [INI](https://en.wikipedia.org/wiki/INI_file) and [TOML](https://toml.io/en/)\n- [ ] **Custom cast type**: Add support for custom cast type for `SELECT` clause.\n- [ ] **Documentation**: Create detailed guides and examples for advanced use cases.\n- [ ] **Add explain method**: Add method `explain()` for explaining query execution from actual query debugger and provide more complex information about query.\n- [ ] **PHPStan 8**: Fix all PHPStan 8 errors.\n- [ ] **Tests**: Increase test coverage.\n- [ ] **Optimize GROUP BY**: Optimize `GROUP BY` for more memory efficient data processing.\n- [ ] **Hashmap cache**: Add hashmap cache (Redis, Memcache) for more memory efficient data processing.\n- [ ] ~~**DELETE, UPDATE, INSERT**: Support for manipulating data in files.~~ - Instead of this, it will comes support\nfor exporting data to files (CSV, NDJson, MessagePack, and more...) by `INTO` clause.\n\n## 9. Contributions\n\nIf you have suggestions or would like to contribute to these features, feel free to open an issue or a pull request!\n\n**How to contribute:**\n- Fork the repository\n- Create a new branch\n- Make your changes\n- Create a pull request\n- All tests must pass\n- Wait for approval\n- 🚀\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1biot%2FFiQueLa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F1biot%2FFiQueLa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1biot%2FFiQueLa/lists"}