{"id":14966329,"url":"https://github.com/yiisoft/data","last_synced_at":"2025-04-13T00:47:42.110Z","repository":{"id":35004100,"uuid":"195634919","full_name":"yiisoft/data","owner":"yiisoft","description":"Data providers","archived":false,"fork":false,"pushed_at":"2025-03-17T11:49:55.000Z","size":573,"stargazers_count":44,"open_issues_count":5,"forks_count":18,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-04-13T00:47:32.175Z","etag":null,"topics":["data-reader","hacktoberfest","pagination","yii3"],"latest_commit_sha":null,"homepage":"https://www.yiiframework.com/","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yiisoft.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":".github/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"open_collective":"yiisoft","github":["yiisoft"]}},"created_at":"2019-07-07T09:50:23.000Z","updated_at":"2025-03-18T15:43:07.000Z","dependencies_parsed_at":"2023-02-14T09:40:13.744Z","dependency_job_id":"6fedcb30-2d9f-4bae-95fa-ca8f3d4328f2","html_url":"https://github.com/yiisoft/data","commit_stats":{"total_commits":229,"total_committers":23,"mean_commits":9.956521739130435,"dds":0.6681222707423581,"last_synced_commit":"a080786b9df3788b7cc376eaa3fb80babc28392a"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":"yiisoft/package-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yiisoft%2Fdata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yiisoft%2Fdata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yiisoft%2Fdata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yiisoft%2Fdata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yiisoft","download_url":"https://codeload.github.com/yiisoft/data/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650434,"owners_count":21139672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-reader","hacktoberfest","pagination","yii3"],"created_at":"2024-09-24T13:36:13.607Z","updated_at":"2025-04-13T00:47:42.091Z","avatar_url":"https://github.com/yiisoft.png","language":"PHP","funding_links":["https://opencollective.com/yiisoft","https://github.com/sponsors/yiisoft"],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/yiisoft\" target=\"_blank\"\u003e\n        \u003cimg src=\"https://yiisoft.github.io/docs/images/yii_logo.svg\" height=\"100px\" alt=\"Yii\"\u003e\n    \u003c/a\u003e\n    \u003ch1 align=\"center\"\u003eYii Data\u003c/h1\u003e\n    \u003cbr\u003e\n\u003c/p\u003e\n\n[![Latest Stable Version](https://poser.pugx.org/yiisoft/data/v)](https://packagist.org/packages/yiisoft/data)\n[![Total Downloads](https://poser.pugx.org/yiisoft/data/downloads)](https://packagist.org/packages/yiisoft/data)\n[![Build status](https://github.com/yiisoft/data/actions/workflows/build.yml/badge.svg)](https://github.com/yiisoft/data/actions/workflows/build.yml)\n[![Code Coverage](https://codecov.io/gh/yiisoft/data/branch/master/graph/badge.svg)](https://codecov.io/gh/yiisoft/data)\n[![Mutation testing badge](https://img.shields.io/endpoint?style=flat\u0026url=https%3A%2F%2Fbadge-api.stryker-mutator.io%2Fgithub.com%2Fyiisoft%2Fdata%2Fmaster)](https://dashboard.stryker-mutator.io/reports/github.com/yiisoft/data/master)\n[![static analysis](https://github.com/yiisoft/data/workflows/static%20analysis/badge.svg)](https://github.com/yiisoft/data/actions?query=workflow%3A%22static+analysis%22)\n[![type-coverage](https://shepherd.dev/github/yiisoft/data/coverage.svg)](https://shepherd.dev/github/yiisoft/data)\n[![psalm-level](https://shepherd.dev/github/yiisoft/data/level.svg)](https://shepherd.dev/github/yiisoft/data)\n\nThe package provides generic data abstractions. The aim is to hide a storage aspect from the operations of reading,\nwriting and processing data.\n\nFeatures are:\n\n- Data reader abstraction with counting, sorting, limiting and offsetting, reading criteria filter and post-filter.\n- Pagination abstraction along with offset and keyset implementations.\n- Data writer abstraction.\n- Data processor abstraction.\n\n## Requirements\n\n- PHP 8.1 or higher.\n\n## Installation\n\nThe package could be installed with [Composer](https://getcomposer.org):\n\n```shell\ncomposer require yiisoft/data\n```\n\n## General usage\n\n## Concepts\n\n- Each data set consists of items.\n- Each item has multiple named fields.\n- All items in a data set have the same structure.\n\n## Reading data\n\nData reader aim is to read data from a storage such as database, array or API and convert it to a simple iterator of\nfield =\u003e value items.\n\n```php\n$reader = new MyDataReader(...);\n$result = $reader-\u003eread(); \n```\n\nResult is `iterable` so you can use `foreach` on it. If you need an array, it could be achieved the following way:\n\n```php\n// using is foreach\nforeach ($result as $item) {\n    // ...\n}\n\n// preparing array\n$dataArray = $result instanceof \\Traversable ? iterator_to_array($result, true) : (array)$result;\n```\n\n### Limiting the number of items to read\n\nYou can limit the number of items in an iterator:\n\n```php\n$reader = (new MyDataReader(...))-\u003ewithLimit(10);\nforeach ($reader-\u003eread() as $item) {\n    // ...\n}\n```\n\n### Counting the total number of items\n\nTo know total number of items in a data reader implementing `CountableDataInterface`:\n\n```php\n$reader = new MyDataReader(...);\n$total = count($reader);\n```\n\n### Filtering\n\nFiltering of data could be done in two steps:\n\n1. Forming a criteria for getting the data. That is done by \"filter.\"\n2. Post-filtering data by iteration and checking each item.\n   That is done by `IterableDataReader` with filters.\n\nWhenever possible, it is best to stick to using criteria because usually it gives much better performance.\n\nTo filter data in a data reader implementing `FilterableDataInterface` you need to supply filter to\n`withFilter()` method:\n\n```php\n$filter = new All(\n    new GreaterThan('id', 3),\n    new Like('name', 'agent')\n);\n\n$reader = (new MyDataReader(...))\n    -\u003ewithFilter($filter);\n\n$data = $reader-\u003eread();\n```\n\nFilter could be composed with:\n\n- `All`\n- `Any`\n- `Between`\n- `Equals`\n- `EqualsNull`\n- `GreaterThan`\n- `GreaterThanOrEqual`\n- `ILike`\n- `In`\n- `LessThan`\n- `LessThanOrEqual`\n- `Like`\n- `Not`\n\n#### Filtering with arrays\n\nThe `All` and `Any` filters have a `withCriteriaArray()` method, which allows you to define filters with arrays.\n\n```php\n$dataReader-\u003ewithFilter((new All())-\u003ewithCriteriaArray([\n    ['=', 'id', 88],\n    [\n       'or',\n       [\n          ['=', 'color', 'red'],\n          ['=', 'state', 1],\n       ]\n    ]\n]));\n```\n\n#### Implementing your own filter\n\nTo have your own filter:\n\n- Implement at least `FilterInterface`, which includes:\n  - `getOperator()` method that returns a string that represents a filter operation.\n  - `toArray()` method that returns an array with filtering parameters.\n- If you want to create a filter handler for a specific data reader type, then you need to implement at least\n`FilterHandlerInterface`. It has a single `getOperator()` method that returns a string representing a filter operation.\nAlso, each data reader specifies an extended interface required for handling or building the operation.\n*For example, `IterableDataFilter` defines `IterableFilterHandlerInterface`, which contains additional `match()`\nmethod to execute a filter on PHP variables.*\n\nYou can add your own filter handlers to the data reader using the `withFilterHandlers()` method. You can add any filter\nhandler to the reader. If a reader is not able to use a filter, filter is ignored.\n\n```php\n// own filter for filtering\nclass OwnNotTwoFilter implements FilterInterface\n{\n    private $field;\n\n    public function __construct($field)\n    {\n        $this-\u003efield = $field;\n    }\n    public static function getOperator(): string\n    {\n        return 'my!2';\n    }\n    public function toArray(): array\n    {\n        return [static::getOperator(), $this-\u003efield];\n    }\n}\n\n// own iterable filter handler for matching\nclass OwnIterableNotTwoFilterHandler implements IterableFilterHandlerInterface\n{\n    public function getOperator(): string\n    {\n        return OwnNotTwoFilter::getOperator();\n    }\n\n    public function match(array $item, array $arguments, array $filterHandlers): bool\n    {\n        [$field] = $arguments;\n        return $item[$field] != 2;\n    }\n}\n\n// and using it on a data reader\n$filter = new All(\n    new LessThan('id', 8),\n    new OwnNotTwoFilter('id'),\n);\n\n$reader = (new MyDataReader(...))\n    -\u003ewithFilter($filter)\n    -\u003ewithFilterHandlers(\n        new OwnIterableNotTwoFilter()\n        new OwnSqlNotTwoFilter()    // for SQL\n        // and for any supported readers...\n    );\n\n$data = $reader-\u003eread();\n```\n\n### Sorting\n\nTo sort data in a data reader implementing `SortableDataInterface` you need to supply a sort object to\n`withSort()` method:\n\n```php\n$sorting = Sort::only([\n    'id',\n    'name'\n]);\n\n$sorting = $sorting-\u003ewithOrder(['name' =\u003e 'asc']);\n// or $sorting = $sorting-\u003ewithOrderString('name');\n\n$reader = (new MyDataReader(...))\n    -\u003ewithSort($sorting);\n\n$data = $reader-\u003eread();\n```\n\nThe goal of the `Sort` is to map logical fields sorting to real data set fields sorting and form a criteria for the data\nreader. Logical fields are the ones user operates with. Real fields are the ones actually present in a data set.\nSuch a mapping helps when you need to sort by a single logical field that, in fact, consists of multiple fields\nin the underlying data set. For example, you provide a user with a username which consists of first name and last name\nfields in the actual data set.\n\nTo get a `Sort` instance, you can use either `Sort::only()` or `Sort::any()`. `Sort::only()` ignores user-specified order\nfor logical fields that have no configuration. `Sort::any()` uses user-specified logical field name and order directly\nfor fields that have no configuration.\n\nEither way, you pass a config array that specifies which logical fields should be order-able and, optionally, details on\nhow these should map to real fields order.\n\nThe current order to apply is specified via `withOrder()` where you supply an array with keys corresponding to logical\nfield names and values correspond to order (`asc` or `desc`). Alternatively `withOrderString()` can be used. In this case, \nordering is represented as a single string containing comma separate logical field names. If the name is prefixed by `-`,\nordering direction is set to `desc`.\n\n### Skipping some items\n\nIn case you need to skip some items from the beginning of data reader implementing `OffsetableDataInterface`:\n\n```php\n$reader = (new MyDataReader(...))-\u003ewithOffset(10);\n```\n\n### Implementing your own data reader\n\nTo have your own data reader, you need to implement at least `DataReaderInteface`. It has a single `read()`\nmethod that returns iterable representing a set of items.\n\nAdditional interfaces could be implemented to support different pagination types, ordering and filtering:\n\n- `CountableDataInterface` - allows getting total number of items in data reader.\n- `FilterableDataInterface` - allows returning subset of items based on criteria.\n- `LimitableDataInterface` - allows returning limited subset of items.\n- `SortableDataInterface` - allows sorting by one or multiple fields.\n- `OffsetableDataInterface` - allows skipping first N items when reading data.\n\nNote that when implementing these, methods, instead of modifying data, should only define criteria that is later used\nin `read()` to affect what data is returned.\n\n## Pagination\n\nPagination allows getting a limited subset of data that is both handy for displaying items page by page and for getting\nacceptable performance on big data sets.\n\nThere are two types of pagination provided: traditional offset pagination and keyset pagination.\n\n### Offset pagination\n\nOffset pagination is a common pagination method that selects OFFSET + LIMIT items and then skips OFFSET items.  \n\nAdvantages:\n\n- The total number of pages is available\n- Can get to specific page\n- Data can be unordered\n- The limit set in the data reader is taken into account\n\nDisadvantages:\n\n- Performance degrades with page number increase\n- Insertions or deletions in the middle of the data are making results inconsistent\n\nUsage is the following:\n\n```php\n$reader = (new MyDataReader(...));\n\n$paginator = (new OffsetPaginator($dataReader))\n            -\u003ewithPageSize(10)\n            -\u003ewithCurrentPage(2);\n\n\n$total = $paginator-\u003egetTotalPages();\n$data = $paginator-\u003eread();\n```\n\n### Keyset pagination\n\nKeyset pagination is an alternative pagination method that is good for infinite scrolling and \"load more.\" It is selecting\nLIMIT items that have key field greater or lesser (depending on the sorting) than the value specified.\n\nAdvantages:\n\n- Performance doesn't depend on page number\n- Consistent results regardless of insertions and deletions\n\nDisadvantages:\n\n- The total number of pages is not available\n- Can't get to specific page, only \"previous\" and \"next\"\n- Data cannot be unordered\n- The limit set in the data reader leads to an exception\n\nUsage is the following:\n\n```php\n$sort = Sort::only(['id', 'name'])-\u003ewithOrderString('id');\n\n$dataReader = (new MyDataReader(...))\n    -\u003ewithSort($sort);\n\n$paginator = (new KeysetPaginator($dataReader))\n    -\u003ewithPageSize(10)\n    -\u003ewithToken(PageToken::next('13'));\n```\n\nWhen displaying first page ID (or another field name to paginate by) of the item displayed last is used with `withNextPageToken()`\nto get next page.\n\n## Writing data\n\n```php\n$writer = new MyDataWriter(...);\n$writer-\u003ewrite($arrayOfItems);\n```\n\n## Processing data\n\n```php\n$processor = new MyDataProcessor(...);\n$processor-\u003eprocess($arrayOfItems);\n```\n\n## Documentation\n\n- Guide: [Português - Brasil](docs/guide/pt-BR/README.md)\n- [Internals](docs/internals.md)\n\nIf you need help or have a question, the [Yii Forum](https://forum.yiiframework.com/c/yii-3-0/63) is a good place for\nthat. You may also check out other [Yii Community Resources](https://www.yiiframework.com/community).\n\n## License\n\nThe Yii Data is free software. It is released under the terms of the BSD License.\nPlease see [`LICENSE`](./LICENSE.md) for more information.\n\nMaintained by [Yii Software](https://www.yiiframework.com/).\n\n## Support the project\n\n[![Open Collective](https://img.shields.io/badge/Open%20Collective-sponsor-7eadf1?logo=open%20collective\u0026logoColor=7eadf1\u0026labelColor=555555)](https://opencollective.com/yiisoft)\n\n## Follow updates\n\n[![Official website](https://img.shields.io/badge/Powered_by-Yii_Framework-green.svg?style=flat)](https://www.yiiframework.com/)\n[![Twitter](https://img.shields.io/badge/twitter-follow-1DA1F2?logo=twitter\u0026logoColor=1DA1F2\u0026labelColor=555555?style=flat)](https://twitter.com/yiiframework)\n[![Telegram](https://img.shields.io/badge/telegram-join-1DA1F2?style=flat\u0026logo=telegram)](https://t.me/yii3en)\n[![Facebook](https://img.shields.io/badge/facebook-join-1DA1F2?style=flat\u0026logo=facebook\u0026logoColor=ffffff)](https://www.facebook.com/groups/yiitalk)\n[![Slack](https://img.shields.io/badge/slack-join-1DA1F2?style=flat\u0026logo=slack)](https://yiiframework.com/go/slack)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyiisoft%2Fdata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyiisoft%2Fdata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyiisoft%2Fdata/lists"}