{"id":27373431,"url":"https://github.com/awesomized/crc-fast-php-ext","last_synced_at":"2025-04-19T14:30:12.313Z","repository":{"id":287293730,"uuid":"964268904","full_name":"awesomized/crc-fast-php-ext","owner":"awesomized","description":"Fast, hardware-accelerated CRC calculation in PHP for all known CRC-32 and CRC-64 variants using SIMD intrinsics, which can exceed 100GiB/s for CRC-32, and 50GiB/s for CRC-64, on modern systems. Much faster (up to \u003e200X) than PHP's internals.","archived":false,"fork":false,"pushed_at":"2025-04-14T20:05:34.000Z","size":24,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-17T22:04:41.253Z","etag":null,"topics":["checksum","checksum-calculation","crc","crc32","crc64","extension","hardware-acceleration","php","php-extension","simd"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/awesomized.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-Apache","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-11T00:26:10.000Z","updated_at":"2025-04-14T20:05:38.000Z","dependencies_parsed_at":"2025-04-18T01:18:05.757Z","dependency_job_id":"1d73f9df-bcca-4a41-bbbb-4f2151927330","html_url":"https://github.com/awesomized/crc-fast-php-ext","commit_stats":null,"previous_names":["awesomized/crc-fast-php-ext"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awesomized%2Fcrc-fast-php-ext","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awesomized%2Fcrc-fast-php-ext/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awesomized%2Fcrc-fast-php-ext/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/awesomized%2Fcrc-fast-php-ext/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/awesomized","download_url":"https://codeload.github.com/awesomized/crc-fast-php-ext/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249489609,"owners_count":21280812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["checksum","checksum-calculation","crc","crc32","crc64","extension","hardware-acceleration","php","php-extension","simd"],"created_at":"2025-04-13T11:14:49.692Z","updated_at":"2025-04-19T14:30:12.295Z","avatar_url":"https://github.com/awesomized.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"`awesome/crc_fast`\n===========\n\n[![Test status](https://github.com/awesomized/crc-fast-php-ext/workflows/Tests/badge.svg)](https://github.com/awesomized/crc-fast-php-ext/actions?query=workflow%3ATests)\n[![Latest Stable Version](https://img.shields.io/packagist/v/awesome/crc-fast)](https://packagist.org/packages/awesome/crc-fast)\n\nFast, hardware-accelerated CRC calculation in `PHP` for [all known CRC-32 and CRC-64 variants](https://reveng.sourceforge.io/crc-catalogue/all.htm) using `SIMD` \nintrinsics, which can exceed _100GiB/s_ for `CRC-32`, and _50GiB/s_ for `CRC-64`, on modern systems.\n\nIt is much, much faster ([up to \u003e200X](#performance)) than the native [crc32](https://www.php.net/manual/en/function.crc32.php), `crc32b`, and `crc32c` \n[implementations](https://www.php.net/manual/en/function.hash-algos.php) in `PHP`, plus adds many more variants \n(particularly `CRC-64/NVME`). \n\nThe performance gains are especially pronounced on `aarch64` (Arm) systems, since PHP doesn't\ncurrently use hardware acceleration there.\n\n## CRC-64/NVME\n\n[CRC-64/NVME](https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat.crc-64-nvme) is in use in a variety of\nlarge-scale and mission-critical systems, software, and hardware, such as:\n- The `AWS S3` [recommended checksum](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html)\n- The [Linux kernel](https://github.com/torvalds/linux/blob/786c8248dbd33a5a7a07f7c6e55a7bfc68d2ca48/lib/crc64.c#L66-L73)\n- The [NVMe specification](https://nvmexpress.org/wp-content/uploads/NVM-Express-NVM-Command-Set-Specification-1.0d-2023.12.28-Ratified.pdf)\n\n## CRC-32/PHP, one of PHP's special flowers 🌼\n\n`hash('crc32')` in `PHP` is not the same as `crc32()` in `PHP`, and doesn't match the `crc32` definition or output in \nmany other programming languages and implementations (which is typically \n[CRC-32/ISO-HDLC](https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat.crc-32-iso-hdlc))\n\nInstead, it's actually [CRC-32/BZIP2](https://reveng.sourceforge.io/crc-catalogue/all.htm#crc.cat.crc-32-bzip2) which \nis then byte-reversed. This extension provides a `CrcFast\\CRC_32_PHP` algorithm `const` that performs the same calculation, \nbut accelerated.\n\n```php \n$checksum = CrcFast\\checksum(\n    algorithm: CrcFast\\CRC_32_PHP,\n    string: '123456789'\n); // 181989fc\n```\n\n## Requirements\nUses the [crc_fast](https://github.com/awesomized/crc-fast-rust) `Rust` package and its `C`-compatible shared library\nunder the hood, so you'll need to have built and installed it. (See [Usage](#Usage), below).\n\n## Changes\n\nSee the [change log](CHANGELOG.md).\n\n## Installing\n\nUse [Composer](https://getcomposer.org) to install this library using [PIE](https://github.com/php/pie) (note the \n[Requirements](#Requirements) above):\n\n```bash\ncomposer install awesome/crc-fast\n```\n\nIf you're using a non-standard installation location for the [crc_fast](https://github.com/awesomized/crc-fast-rust) \nlibrary, you may need to specify where the `crc_fast` header (in `include`) and shared library (in `lib`) can be found:\n\n```bash\ncomposer install awesome/crc-fast --with-crc-fast=/path/to/crc-fast\n```\n\n## Building\n\nLike most `PHP` extensions, you can also build yourself:\n\n```bash\nphpize\n./configure\nmake\n```\n\nor with a custom [crc_fast](https://github.com/awesomized/crc-fast-rust) location where the `crc_fast` header and shared\nlibrary can be found:\n\n```bash\nphpize\n./configure --with-crc-fast=/path/to/crc-fast\nmake\n```\n\n## Usage\n\nExamples are for `CRC-64/NVME`, but you can use any [supported algorithm](#get-a-list-of-supported-algorithm-variants) \nvariant.\n\n### Calculate CRC-64/NVME checksums:\n\n```php\n\n// calculate the checksum of a string\n$checksum = CrcFast\\hash(\n    algorithm: CrcFast\\CRC_64_NVME,\n    string: '123456789'\n); // ae8b14860a799888\n\n// calculate the checksum of a file, which will chunk through the file optimally,\n// limiting RAM usage and maximizing throughput\n$checksum = CrcFast\\hash_file(\n    algorithm: CrcFast\\CRC_64_NVME,\n    filename: 'path/to/123456789.txt',\n); // ae8b14860a799888\n```\n\n### Calculate CRC-64/NVME checksums with a Digest for intermittent / streaming / etc workloads:\n\n```php\n$crc64Digest = CrcFast\\Digest::new(\n    algorithm: CrcFast\\CRC_64_NVME,\n);\n\n// write some data to the digest\n$crc64Digest-\u003ewrite('1234');\n\n// write some more data to the digest\n$crc64Digest-\u003ewrite('56789');\n\n// calculate the entire digest\n$checksum = $crc64Digest-\u003efinalize(); // ae8b14860a799888\n```\n\n### Get a list of supported algorithm variants\n```php\n$algorithms = get_supported_algorithms();\n\nvar_dump($algorithms);\n```\n\n## Equivalents to PHP functions\n\n### crc32()\nCalculates `CRC-32/ISO-HDLC` as an integer.\n\n```php \n$checksum = CrcFast\\crc32(\n    data: '123456789'\n); // 3421780262\n```\n\n### hash('crc32') 🌼\nCalculates [CRC-32/PHP](#crc-32php-one-of-phps-special-flowers-) as binary or hex.\n\n```php \n$checksum = CrcFast\\hash(\n    algorithm: CrcFast\\CRC_32_PHP,\n    string: '123456789'\n    binary: false,\n); // 181989fc\n```\n\n### hash('crc32b') \nCalculates `CRC-32/ISO-HDLC` as binary or hex.\n\n```php \n$checksum = CrcFast\\hash(\n    algorithm: CrcFast\\CRC_32_ISO_HDLC,\n    string: '123456789'\n    binary: false,\n); // cbf43926\n```\n\n### hash('crc32c')\nCalculates `CRC-32/ISCSI` as binary or hex.\n\n```php \n$checksum = CrcFast\\hash(\n    algorithm: CrcFast\\CRC_32_ISCSI,\n    string: '123456789'\n    binary: false,\n); // b798b438\n```\n\n## IDE Stubs\n\nThis extension comes with IDE [stubs](crc_fast.stub.php) for use with your favorite development environment.\n\n## Tests\nSee the [tests](tests) directory for test coverage, which also double as useful examples.\n\n```bash\nmake test\n```\n\n## Platform support\n\nThis extension has been extensively tested on `macOS` and `Linux`, on both `aarch64` and `x86_64`. \n\nAt [Awesome](https://awesome.co) we use it in production at very large scale on `Linux` on both \n[Flickr](https://flickr.com) and [SmugMug](https://smugmug.com).\n\nThis extension is not currently supported on `Windows`. :(\n\nThe underlying [crc_fast](https://github.com/awesomized/crc-fast-rust) library (same authors) builds and works on \n`Windows`, so this is likely just a build issue with creating a working `config.w32` implementation. (I took a quick \nstab, failed, and moved on since we don't use `Windows` in production.)\n\nIf you want to help, please open a working PR. I'd love to merge it.\n\n## Performance\n\n`PHP` already uses `SIMD` intrinsics for `CRC-32` calculations on `x86_64` but not on `aarch64`. Even on `x86_64`, this\nlibrary provides considerable improvements, in addition to supporting many more variants.\n\nTested using the maximum settings for [crc_fast](https://github.com/awesomized/crc-fast-rust) for each platform, using\n1MiB random payloads.\n\n### CRC-32/ISCSI and CRC-32/ISO-HDLC\n\n| Arch    | Brand | CPU             | System               |         PHP |      crc_fast | Speedup |\n|:--------|:------|-----------------|----------------------|------------:|--------------:|--------:|\n| x86_64  | Intel | Sapphire Rapids | EC2 c7i.metal-48xl   | ~27.0 GiB/s |  ~108.1 GiB/s |     ~4X |\n| x86_64  | AMD   | Genoa           | EC2 c7a.metal-48xl   | ~13.6 GiB/s |   ~53.7 GiB/s |     ~4X |\n| aarch64 | AWS   | Graviton4       | EC2 c8g.metal-48xl   |  ~0.4 GiB/s |   ~52.3 GiB/s |   ~141X |\n| aarch64 | Apple | M3 Ultra        | Mac Studio (32 core) |  ~0.4 GiB/s |   ~99.6 GiB/s |   ~233X |\n\n### CRC-32/PHP 🌼\n\n| Arch    | Brand | CPU             | System               |          PHP |    crc_fast | Speedup |\n|:--------|:------|-----------------|----------------------|-------------:|------------:|--------:|\n| x86_64  | Intel | Sapphire Rapids | EC2 c7i.metal-48xl   |  ~26.6 GiB/s | ~27.4 GiB/s |     n/a |\n| x86_64  | AMD   | Genoa           | EC2 c7a.metal-48xl   |  ~13.6 GiB/s | ~25.4 GiB/s |     ~2X |\n| aarch64 | AWS   | Graviton4       | EC2 c8g.metal-48xl   |   ~0.4 GiB/s | ~31.5 GiB/s |    ~73X |\n| aarch64 | Apple | M3 Ultra        | Mac Studio (32 core) |   ~0.4 GiB/s | ~57.8 GiB/s |   ~134X |\n\n### CRC-64/NVME\n\nNote that PHP has no native equivalent.\n\n| Arch    | Brand | CPU             | System               |    crc_fast | \n|:--------|:------|-----------------|----------------------|------------:|\n| x86_64  | Intel | Sapphire Rapids | EC2 c7i.metal-48xl   | ~54.6 GiB/s | \n| x86_64  | AMD   | Genoa           | EC2 c7a.metal-48xl   | ~27.0 GiB/s | \n| aarch64 | AWS   | Graviton4       | EC2 c8g.metal-48xl   | ~37.0 GiB/s |\n| aarch64 | Apple | M3 Ultra        | Mac Studio (32 core) | ~70.0 GiB/s | \n\n## License\n\n`cfc-fast` is dual-licensed under\n\n* Apache 2.0 license ([LICENSE-Apache](./LICENSE-Apache) or \u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e)\n* MIT license ([LICENSE-MIT](./LICENSE-MIT) or \u003chttps://opensource.org/licenses/MIT\u003e)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawesomized%2Fcrc-fast-php-ext","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fawesomized%2Fcrc-fast-php-ext","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fawesomized%2Fcrc-fast-php-ext/lists"}