{"id":24886330,"url":"https://github.com/1biot/fiquela","last_synced_at":"2026-04-05T14:02:21.003Z","repository":{"id":268853605,"uuid":"905652958","full_name":"1biot/FiQueLa","owner":"1biot","description":null,"archived":false,"fork":false,"pushed_at":"2025-01-28T11:07:29.000Z","size":5514,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-28T12:22:14.380Z","etag":null,"topics":["json","json-parser","neon","parser","php","query","sql","xml","xml-parser","yaml","yaml-parser"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/1biot.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-19T09:03:30.000Z","updated_at":"2025-01-28T11:07:34.000Z","dependencies_parsed_at":"2025-01-15T02:50:30.934Z","dependency_job_id":"7aa01256-6fbb-4c8e-ad93-c3c393800172","html_url":"https://github.com/1biot/FiQueLa","commit_stats":null,"previous_names":["1biot/jql","1biot/uniquel","1biot/fiquela"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2FFiQueLa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2FFiQueLa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2FFiQueLa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1biot%2FFiQueLa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/1biot","download_url":"https://codeload.github.com/1biot/FiQueLa/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236677573,"owners_count":19187477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","json-parser","neon","parser","php","query","sql","xml","xml-parser","yaml","yaml-parser"],"created_at":"2025-02-01T15:14:40.084Z","updated_at":"2026-04-05T14:02:20.997Z","avatar_url":"https://github.com/1biot.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FiQueLa: File Query Language \n\n\u003e _[fi-kju-ela]_\n\n![Packagist Version](https://img.shields.io/packagist/v/1biot/fiquela)\n[![CI](https://github.com/1biot/fiquela/actions/workflows/ci.yml/badge.svg)](https://github.com/1biot/fiquela/actions/workflows/ci.yml)\n![Packagist Dependency Version](https://img.shields.io/packagist/dependency-v/1biot/fiquela/php)\n![Packagist License](https://img.shields.io/packagist/l/1biot/fiquela)\n\n![Coverage](https://img.shields.io/badge/coverage-83.62%25-yellow)\n![PHPUnit Tests](https://img.shields.io/badge/PHPUnit-tests%3A_745-lightgreen)\n![PHPStan](https://img.shields.io/badge/phpstan-level_8-lightgreen)\n\n**F**i**Q**ue**L**a lets you query files like a database, with SQL-like syntax or a fluent PHP API.\nFilter, join, group, aggregate, and transform data from **XML**, **XLS**, **ODS**, **CSV**, **JSON**, **NDJSON**,\n**YAML**, **NEON**, and **HTTP access logs** without setting up a database. It is built for real-world data processing\nwith streaming support, explain/debug tooling, and strongly typed operators.\n\n**Features**:\n\n- 📂 **Supports multiple formats**: Work seamlessly with XML, CSV, JSON, NDJSON, YAML, NEON, XLS, and HTTP access logs.\n- 🛠️ **SQL-inspired syntax**: Perform `SELECT`, `JOIN`, `WHERE`, `GROUP BY`, `HAVING`, `ORDER BY` and more.\n- ✍️ **Flexible querying**: Write SQL-like strings or use the fluent API.\n- 📊 **Powerful expressions and functions**: Use `CASE WHEN`, `IF`, grouped conditions, `XOR`, `REGEXP`, aggregates, and utility functions.\n- 🚀 **Stream-first processing**: Optimized for large JSON, XML, and CSV files with low memory pressure where possible.\n- 🧑‍💻 **Developer-Friendly**: Map results to DTOs for easier data manipulation.\n- ⭐ **Unified API across all supported formats**: Use a consistent API for all your data needs.\n\n### Why FiQueLa\n\n- Query files with familiar SQL concepts while keeping everything in PHP.\n- Join data across sources and formats in one query.\n- Handle advanced logic with nested condition groups and statement functions.\n- Inspect execution using explain output and debugger tooling.\n\n**Table of Contents**:\n\n- [Overview](#1-overview)\n- [Installation](#2-installation)\n- [Supported Formats](#3-supported-formats)\n- [Getting Started](#4-getting-started)\n- [Documentation](#5-documentation)\n  - [Opening Files](docs/opening-files.md)\n  - [Fluent API](docs/fluent-api.md)\n  - [File Query Language](docs/file-query-language.md)\n  - [Fetching Data](docs/fetching-data.md)\n  - [Query Life Cycle](docs/query-life-cycle.md)\n  - [Query Inspection and Benchmarking](docs/query-inspection-and-benchmarking.md)\n  - [API Reference](docs/api-reference.md)\n- [Examples](#6-examples)\n- [Ecosystem](#7-ecosystem)\n  - [FiQueLa CLI](#fiquela-cli)\n  - [FiQueLa API](#fiquela-api)\n  - [FiQueLa Studio](#fiquela-studio)\n- [Known issues](#8-known-issues)\n- [Roadmap](#9-roadmap)\n- [Contributions](#10-contributions)\n\n## 1. Overview\n\nWhy limit SQL to databases when it can be just as effective for structured files?\n**F**i**Q**ue**L**a brings SQL-like querying to file-based data and keeps your workflow fully in PHP.\n\nKey highlights:\n- **Universal querying**: Filter, sort, join, and aggregate data across multiple file formats.\n- **Real SQL-like behavior**: Use `GROUP BY`, `HAVING`, nested conditions, `CASE WHEN`, `IF`, and many built-in functions.\n- **Flexible integration**: Query through fluent API or SQL-like strings, whichever matches your use case.\n- **Operational tooling**: Use debugger and explain plans to understand performance and execution.\n\nUse **F**i**Q**ue**L**a to:\n- Simplify data extraction and analysis from structured files.\n- Combine data from multiple sources with ease.\n- Create lightweight data processing pipelines without a full-fledged database.\n\n**F**i**Q**ue**L**a empowers developers to unlock the potential of file-based data with the familiar and expressive language of SQL.\n\n## 2. Installation\n\nInstall via [Composer](https://getcomposer.org/):\n\n```bash\ncomposer require 1biot/fiquela\n```\n\nInstall packages for optional features:\n\n```bash\ncomposer require tracy/tracy\n```\n\n### Dependencies\n\n- **`league/csv`**: Required for CSV file support.\n- **`halaxa/json-machine`**: Required for JSON stream support.\n- **`symfony/yaml`**: Required for YAML file support.\n- **`nette/neon`**: Required for NEON file support.\n- **`openspout/openspout`**: Required for XLSX and ODS file support.\n- **`tracy/tracy`**: Optional for using Debugger\n\n## 3. Supported Formats\n\n| Format      | Name                    | Class                   | File Support | String Support |\n|-------------|-------------------------|-------------------------|--------------|----------------|\n| `csv`       | CSV                     | `FQL\\Stream\\Csv`        | ✅            | ❌              |\n| `xml`       | XML                     | `FQL\\Stream\\Xml`        | ✅            | ❌              |\n| `xls`       | XLS/XLSX                | `FQL\\Stream\\Xls`        | ✅            | ❌              |\n| `ods`       | ODS                     | `FQL\\Stream\\Ods`        | ✅            | ❌              |\n| `jsonFile`  | JSON Stream             | `FQL\\Stream\\JsonStream` | ✅            | ❌              |\n| `ndJson`    | Newline Delimited JSON  | `FQL\\Stream\\NDJson`     | ✅            | ❌              |\n| `json`      | JSON (json_decode)      | `FQL\\Stream\\Json`       | ✅            | ✅              |\n| `yaml`      | YAML                    | `FQL\\Stream\\Yaml`       | ✅            | ✅              |\n| `neon`      | NEON                    | `FQL\\Stream\\Neon`       | ✅            | ✅              |\n| `log`       | HTTP Access Log         | `FQL\\Stream\\AccessLog`  | ✅            | ❌              |\n\n\n### Directory provider\n\nIs special provider `FQL\\Stream\\Dir` class. It allows you to use directory as a source.\nYou can query all files recursively by queries.\n\n## 4. Getting Started\n\nHere’s a quick example of how **F**i**Q**ue**L**a can simplify your data queries:\n\n```php\nuse FQL\\Enum;\nuse FQL\\Query;\n\n$results = Query\\Provider::fromFileQuery('(./path/to/file.xml).SHOP.SHOPITEM')\n    -\u003eselectAll()\n    -\u003ewhere('EAN', Enum\\Operator::EQUAL, '1234567891011')\n    -\u003eor('PRICE', Enum\\Operator::LESS_THAN_OR_EQUAL, 200)\n    -\u003eorderBy('PRICE')-\u003edesc()\n    -\u003elimit(10)\n    -\u003eexecute()\n    -\u003efetchAll();\n\nprint_r(iterator_to_array($results));\n```\n\nThis query returns rows that match either a specific EAN or a price threshold, sorted by price and limited to 10 records.\n\nOr using the FQL syntax:\n\n```php\nuse FQL\\Query;\n\n$query = \u003c\u003c\u003cFQL\n    SELECT *\n    FROM xml(./path/to/file.xml).SHOP.SHOPITEM\n    WHERE\n        EAN = \"1234567891011\"\n        OR PRICE \u003c= 200\n    ORDER BY PRICE DESC\n    LIMIT 10\nFQL;\n$results = Query\\Provider::fql($query)\n    -\u003eexecute()\n    -\u003efetchAll();\n\nprint_r(iterator_to_array($results));\n````\n\n\nOutput:\n\n```php\nArray\n(\n    [0] =\u003e Array\n        (\n            [NAME] =\u003e \"Product 1\"\n            [EAN] =\u003e \"1234567891011\"\n            [PRICE] =\u003e 300.00\n        )\n    [1] =\u003e Array\n        (\n            [NAME] =\u003e \"Product 2\"\n            [EAN] =\u003e \"1234567891012\"\n            [PRICE] =\u003e 150.00\n        )\n    [2] =\u003e Array\n        (\n            [NAME] =\u003e \"Product 3\"\n            [EAN] =\u003e \"1234567891013\"\n            [PRICE] =\u003e 150.00\n        )\n    ...\n)\n```\n\n## 5. Documentation\n\nFor more details about **F**i**Q**ue**L**a and her capabilities, explore the documentation sections.\n\n- [Opening Files](docs/opening-files.md)\n- [Fluent API](docs/fluent-api.md)\n- [File Query Language](docs/file-query-language.md)\n- [Fetching Data](docs/fetching-data.md)\n- [Query Life Cycle](docs/query-life-cycle.md)\n- [Query Inspection and Benchmarking](docs/query-inspection-and-benchmarking.md)\n- [API Reference](docs/api-reference.md)\n\n## 6. Examples\n\nCheck the examples and run them using Composer. All examples uses `\\FQL\\Query\\Debugger` and methods\n`inspectQuery`, `inspectSql`, `inspectStreamSql` or `benchmarkQuery` to show the results.\n\n```bash\ncomposer examples\n# or\ncomposer example:csv\ncomposer example:join\ncomposer example:json\ncomposer example:neon\ncomposer example:sql\ncomposer example:xml\ncomposer example:yaml\ncomposer example:explain\n```\n\nCheck step **Examples** at [actions](https://github.com/1biot/fiquela/actions/runs/12992585648/job/36232767074) or run\n`composer example:csv` and output will look like this:\n\n```\n=========================\n### Debugger started: ###\n=========================\n\u003e Memory usage (MB): 1.3191 (emalloc)\n\u003e Memory peak usage (MB): 1.7326 (emalloc)\n------------------------------\n\u003e Execution time (s): 8.5E-5\n\u003e Execution time (ms): 0.085\n\u003e Execution time (µs): 85\n\u003e Execution memory peak usage (MB): 0\n=========================\n### Inspecting query: ###\n=========================\n==================\n### SQL query: ###\n==================\n\u003e SELECT\n\u003e   ean ,\n\u003e   defaultCategory ,\n\u003e   EXPLODE(defaultCategory, \" \u003e \") AS categoryArray ,\n\u003e   price ,\n\u003e   ROUND(price, 2) AS price_rounded ,\n\u003e   MOD(price, 100) AS modulo_100 ,\n\u003e   MOD(price, 54) AS modulo_54\n\u003e FROM csv(products-w-1250.csv, \"windows-1250\", \";\").*\n\u003e GROUP BY defaultCategory\n\u003e ORDER BY defaultCategory DESC\n================\n### Results: ###\n================\n\u003e Result class: FQL\\Results\\InMemory\n\u003e Results size memory (KB): 3.55\n\u003e Result exists: true\n\u003e Result count: 15\n========================\n### Fetch first row: ###\n========================\narray (7)\n   'ean' =\u003e 5010232964877\n   'defaultCategory' =\u003e 'Testování \u003e Drogerie'\n   'categoryArray' =\u003e array (2)\n   |  0 =\u003e 'Testování'\n   |  1 =\u003e 'Drogerie'\n   'price' =\u003e 121.0\n   'price_rounded' =\u003e 121.0\n   'modulo_100' =\u003e 21.0\n   'modulo_54' =\u003e 13.0\n\n\u003e\u003e\u003e SPLIT TIME \u003c\u003c\u003c\n\u003e Memory usage (MB): 3.1451 (emalloc)\n\u003e Memory peak usage (MB): 3.2262 (emalloc)\n------------------------------\n\u003e Execution time (s): 0.040016\n\u003e Execution time (ms): 40.016\n\u003e Execution time (µs): 40016\n\u003e Execution memory peak usage (MB): 1.4936\n========================\n### Benchmark Query: ###\n========================\n\u003e 2 500 iterations\n==================\n### SQL query: ###\n==================\n\u003e SELECT\n\u003e   ean ,\n\u003e   defaultCategory ,\n\u003e   EXPLODE(defaultCategory, \" \u003e \") AS categoryArray ,\n\u003e   price ,\n\u003e   ROUND(price, 2) AS price_rounded ,\n\u003e   MOD(price, 100) AS modulo_100 ,\n\u003e   MOD(price, 54) AS modulo_54\n\u003e FROM csv(products-w-1250.csv, \"windows-1250\", \";\").*\n\u003e GROUP BY defaultCategory\n\u003e ORDER BY defaultCategory DESC\n=========================\n### STREAM BENCHMARK: ###\n=========================\n\u003e Size (KB): 2.78\n\u003e Count: 15\n\u003e Iterated results: 37 500\n\u003e\u003e\u003e SPLIT TIME \u003c\u003c\u003c\n\u003e Memory usage (MB): 3.1347 (emalloc)\n\u003e Memory peak usage (MB): 3.2262 (emalloc)\n------------------------------\n\u003e Execution time (s): 36.402098\n\u003e Execution time (ms): 36402.098\n\u003e Execution time (µs): 36402098\n\u003e Execution memory peak usage (MB): 0\n============================\n### IN_MEMORY BENCHMARK: ###\n============================\n\u003e Size (KB): 3.55\n\u003e Count: 15\n\u003e Iterated results: 37 500\n\u003e\u003e\u003e SPLIT TIME \u003c\u003c\u003c\n\u003e Memory usage (MB): 3.1451 (emalloc)\n\u003e Memory peak usage (MB): 3.2262 (emalloc)\n------------------------------\n\u003e Execution time (s): 0.01743\n\u003e Execution time (ms): 17.43\n\u003e Execution time (µs): 17430\n\u003e Execution memory peak usage (MB): 0\n=======================\n### Debugger ended: ###\n=======================\n\u003e Memory usage (MB): 3.1343 (emalloc)\n\u003e Memory peak usage (MB): 3.2262 (emalloc)\n------------------------------\n\u003e Final execution time (s): 36.459756\n\u003e Final execution time (ms): 36459.756\n\u003e Final execution time (µs): 36459756\n```\n\n## 7. Ecosystem\n\nFiQueLa is more than just a PHP library. It comes with a CLI tool, a REST API server, and a web-based query explorer.\n\n### FiQueLa CLI\n\n[**fiquela-cli**](https://github.com/1biot/fiquela-cli) is a command-line tool for querying structured files directly from the terminal. It supports local file querying, remote API connections, and an interactive REPL mode with paginated table output.\n\n```bash\n# Install\ncurl -fsSL https://raw.githubusercontent.com/1biot/fiquela-cli/main/install.sh | bash\n\n# Query a local file\nfiquela-cli --file=data.csv \"SELECT name, price FROM * WHERE price \u003e 100;\"\n\n# Interactive mode\nfiquela-cli --file=data.csv\n```\n\nRequires PHP 8.2+ with readline, curl, and zlib extensions.\n\n### FiQueLa API\n\n[**fiquela-api**](https://github.com/1biot/fiquela-api) is a RESTful server for querying structured files over HTTP. It provides file management, query execution, result export (CSV, TSV, JSON), and query history tracking with JWT authentication.\n\n[![Deploy to DO](https://www.deploytodo.com/do-btn-blue.svg)](https://cloud.digitalocean.com/apps/new?repo=https://github.com/1biot/fiquela-api/tree/main?refcode=92025543cb9f)\n\nRequires credentials configuration via environment variables. Optionally enable S3 backup (Cloudflare R2) for file storage. For more information visit the [repository](https://github.com/1biot/fiquela-api?tab=readme-ov-file#-credentials).\n\nKey endpoints: `POST /api/auth/login` for JWT authentication, `POST /api/v1/query` for executing queries, `GET /api/v1/files` for file management, `GET /api/v1/export/{hash}` for downloading results. All endpoints except login require `Authorization: Bearer \u003ctoken\u003e`.\n\n### FiQueLa Studio\n\n[**studio.fiquela.io**](https://studio.fiquela.io) is a web-based visual query explorer for building and running FQL queries interactively. Requires a running [FiQueLa API](#fiquela-api) instance to connect to.\n\n## 8. Known issues\n\n- ⚠️ Functions `JOIN`, and `ORDER BY` are not memory efficient, because joining data or sorting data requires \nto load all data into memory. It may cause memory issues for large datasets. But everything else is like ⚡️.\n\n## 9. Roadmap\n\n- [x] ~~**Operator BETWEEN**: Add operator `BETWEEN` for filtering data and add support for dates and ranges.~~\n- [x] ~~**XLS/XLSX**: Add Excel file support.~~\n- [x] ~~**Custom cast type**: Add support for custom cast type for `SELECT` clause.~~\n- [x] ~~**Add explain method**: Add method `explain()` for explaining query execution from actual query debugger and provide more complex information about query.~~\n- [x] ~~**PHPStan 8**: Fix all PHPStan 8 errors.~~\n- [x] ~~**Tests**: Increase test coverage (80%+).~~\n- [x] ~~**Optimize GROUP BY**: Optimize `GROUP BY` for more memory efficient data processing.~~\n- [x] ~~**DELETE, UPDATE, INSERT**: Support for manipulating data in files.~~ ~~- Instead of this, it will comes support\nfor exporting data to files (CSV, NDJson, MessagePack, and more...) by `INTO` clause.~~\n- [ ] **Next file formats**: Add next file formats [MessagePack](https://msgpack.org/), [Parquet](https://parquet.apache.org/docs/file-format/), [INI](https://en.wikipedia.org/wiki/INI_file) and [TOML](https://toml.io/en/)\n- [ ] **Documentation**: Create detailed guides and examples for advanced use cases.\n- [ ] **Tests**: Increase test coverage (90%+).\n- [ ] **Hashmap cache**: Add hashmap cache (Redis, Memcache) for more memory efficient data processing.\n\n\n## 10. Contributions\n\nIf you have suggestions or would like to contribute to these features, feel free to open an issue or a pull request!\n\n**How to contribute:**\n- Fork the repository\n- Create a new branch\n- Make your changes\n- Create a pull request\n- All tests must pass\n- Wait for approval\n- 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1biot%2Ffiquela","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F1biot%2Ffiquela","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1biot%2Ffiquela/lists"}