{"id":20906558,"url":"https://github.com/xia-mc/pyfastutil","last_synced_at":"2025-10-25T11:39:01.838Z","repository":{"id":262598417,"uuid":"887741688","full_name":"xia-mc/PyFastUtil","owner":"xia-mc","description":"Make Python Fast Again","archived":false,"fork":false,"pushed_at":"2025-02-05T09:51:39.000Z","size":1890,"stargazers_count":18,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-11T06:48:07.013Z","etag":null,"topics":["asm","binding","cpp","cpython","faster-cpython","fastutil","library","multi-platform","performance","python","simd"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xia-mc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-13T07:50:51.000Z","updated_at":"2025-01-08T20:51:17.000Z","dependencies_parsed_at":"2024-12-08T17:27:25.415Z","dependency_job_id":"073d14af-f880-407b-b0fb-f86b203052c7","html_url":"https://github.com/xia-mc/PyFastUtil","commit_stats":null,"previous_names":["xia-mc/pyfastutil"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/xia-mc/PyFastUtil","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xia-mc%2FPyFastUtil","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xia-mc%2FPyFastUtil/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xia-mc%2FPyFastUtil/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xia-mc%2FPyFastUtil/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xia-mc","download_url":"https://codeload.github.com/xia-mc/PyFastUtil/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xia-mc%2FPyFastUtil/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259914417,"owners_count":22931303,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asm","binding","cpp","cpython","faster-cpython","fastutil","library","multi-platform","performance","python","simd"],"created_at":"2024-11-18T13:34:09.307Z","updated_at":"2025-10-25T11:39:01.784Z","avatar_url":"https://github.com/xia-mc.png","language":"C++","readme":"\u003cdiv align=\"center\"\u003e\n\nEnglish | [简体中文](./README_CN.md)\n\n# PyFastUtil\n\n\u003cp\u003e\n   \u003cimg src=\"./mascot.png\" alt=\"Project Mascot\" width=\"500\"\u003e\n\u003c/p\u003e\n\n## Make Python Fast Again\n\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)\n[![Issues](https://img.shields.io/github/issues/xia-mc/PyFastUtil)](https://img.shields.io/github/issues/xia-mc/PyFastUtil)\n![Version](https://img.shields.io/badge/CPython-3.9_and_later-blue)\n[![Build](https://img.shields.io/github/actions/workflow/status/xia-mc/PyFastUtil/python-package.yml)](https://github.com/xia-mc/PyFastUtil/actions)\n\n\u003c/div\u003e\n\n## Introduction\n\n**PyFastUtil** is a high-performance utility library for Python, inspired by Java's [FastUtil](https://fastutil.di.unimi.it/) library. However, **PyFastUtil is NOT a Python binding for FastUtil**. It is a library built from the ground up, designed to bring C-like efficiency and functionality to CPython.\n\n### Features\n\n- Implements all corresponding Python data structure interfaces while significantly improving performance through targeted optimizations. Users can choose the most suitable data structure type based on their needs.\n- Fully implemented in C/C++ with hardware-level optimizations such as SIMD, aiming to maximize the performance of data structures.\n- Provides efficient Python bindings for some C APIs, allowing advanced users to perform \"unsafe\" low-level operations.\n- Enables inline assembly, allowing advanced users to dynamically generate, invoke, and destroy C functions at runtime.\n\n### Benchmark\n\n\u003e **Note**: For extremely fast \\(O(1)\\) operations (e.g., `pop`, `extend`), PyFastUtil's performance may be slightly inferior to Python's native implementation due to the unavoidable overhead of calling C extensions from CPython. We are actively working to optimize this.\n\n\u003e CPU: AMD Ryzen 7 5700G (AVX2)\n\u003e \n\u003e Windows 11 23H2, Python 3.12, MSVC 19.41.34120\n\n#### Type-Specialized List (e.g., `IntArrayList`)\n\n```text\nPreparing data...\n---Python list \u0026 IntArrayList Benchmark---\nBatch size: 10000\nRepeat: 3\n\nPython list init time: 0.02 ms\nPyFastUtil IntArrayList init time: 0.08 ms\nPyFastUtil speed of Python list (init): 29.788 %\n\nPython list copy time: 0.02 ms\nPyFastUtil IntArrayList copy time: 0.00 ms\nPyFastUtil speed of Python list (copy): 766.102 %\n\nPython list to_python time: 0.02 ms\nPyFastUtil IntArrayList to_python time: 0.27 ms\nPyFastUtil speed of Python list (to_python): 5.821 %\n\nPython list sequential_access time: 0.00 ms\nPyFastUtil IntArrayList sequential_access time: 0.00 ms\nPyFastUtil speed of Python list (sequential_access): 126.667 %\n\nPython list random_access time: 0.39 ms\nPyFastUtil IntArrayList random_access time: 0.62 ms\nPyFastUtil speed of Python list (random_access): 62.423 %\n\nPython list sort time: 1.29 ms\nPyFastUtil IntArrayList sort time: 0.04 ms\nPyFastUtil speed of Python list (sort): 3344.483 %\n\nPython list append time: 0.23 ms\nPyFastUtil IntArrayList append time: 0.32 ms\nPyFastUtil speed of Python list (append): 72.517 %\n\nPython list insert time: 70.74 ms\nPyFastUtil IntArrayList insert time: 10.93 ms\nPyFastUtil speed of Python list (insert): 647.349 %\n\nPython list pop time: 0.29 ms\nPyFastUtil IntArrayList pop time: 0.35 ms\nPyFastUtil speed of Python list (pop): 85.143 %\n\nPython list remove time: 5.50 ms\nPyFastUtil IntArrayList remove time: 2.54 ms\nPyFastUtil speed of Python list (remove): 216.749 %\n\nPython list contains time: 258.02 ms\nPyFastUtil IntArrayList contains time: 2.61 ms\nPyFastUtil speed of Python list (contains): 9896.599 %\n\nPython list index time: 438.12 ms\nPyFastUtil IntArrayList index time: 3.77 ms\nPyFastUtil speed of Python list (index): 11618.038 %\n\nPython list extend time: 0.08 ms\nPyFastUtil IntArrayList extend time: 0.14 ms\nPyFastUtil speed of Python list (extend): 58.433 %\n\nPython list reverse time: 0.00 ms\nPyFastUtil IntArrayList reverse time: 0.00 ms\nPyFastUtil speed of Python list (reverse): 378.948 %\n\n\nAvg speed of PyFastUtil compared to Python list: 1950.647 %\n```\n\n#### Generic Type List (e.g., `ObjectArrayList`)\n\n```text\nPreparing data...\n---Python list \u0026 ObjectArrayList Benchmark---\nBatch size: 10000\nRepeat: 3\n\nPython list init time: 0.06 ms\nPyFastUtil ObjectArrayList init time: 0.05 ms\nPyFastUtil speed of Python list (init): 129.484 %\n\nPython list copy time: 0.07 ms\nPyFastUtil ObjectArrayList copy time: 0.03 ms\nPyFastUtil speed of Python list (copy): 268.376 %\n\nPython list to_python time: 0.03 ms\nPyFastUtil ObjectArrayList to_python time: 0.03 ms\nPyFastUtil speed of Python list (to_python): 99.325 %\n\nPython list sequential_access time: 0.00 ms\nPyFastUtil ObjectArrayList sequential_access time: 0.00 ms\nPyFastUtil speed of Python list (sequential_access): 139.130 %\n\nPython list random_access time: 0.22 ms\nPyFastUtil ObjectArrayList random_access time: 0.31 ms\nPyFastUtil speed of Python list (random_access): 71.053 %\n\nPython list sort time: 1039.91 ms\nPyFastUtil ObjectArrayList sort time: 1003.55 ms\nPyFastUtil speed of Python list (sort): 103.623 %\n\nPython list append time: 0.27 ms\nPyFastUtil ObjectArrayList append time: 0.33 ms\nPyFastUtil speed of Python list (append): 81.091 %\n\nPython list insert time: 72.43 ms\nPyFastUtil ObjectArrayList insert time: 20.82 ms\nPyFastUtil speed of Python list (insert): 347.893 %\n\nPython list pop time: 0.36 ms\nPyFastUtil ObjectArrayList pop time: 0.27 ms\nPyFastUtil speed of Python list (pop): 130.323 %\n\nPython list remove time: 5.17 ms\nPyFastUtil ObjectArrayList remove time: 5.17 ms\nPyFastUtil speed of Python list (remove): 100.102 %\n\nPython list contains time: 665.64 ms\nPyFastUtil ObjectArrayList contains time: 3.77 ms\nPyFastUtil speed of Python list (contains): 17634.553 %\n\nPython list index time: 985.46 ms\nPyFastUtil ObjectArrayList index time: 6.80 ms\nPyFastUtil speed of Python list (index): 14501.919 %\n\nPython list extend time: 0.08 ms\nPyFastUtil ObjectArrayList extend time: 0.12 ms\nPyFastUtil speed of Python list (extend): 73.264 %\n\nPython list reverse time: 0.00 ms\nPyFastUtil ObjectArrayList reverse time: 0.00 ms\nPyFastUtil speed of Python list (reverse): 106.000 %\n\n\nAvg speed of PyFastUtil compared to Python list: 2413.296 %\n```\n\n## Installation\n\n### Install PyFastUtil from PyPI:\n```shell\npip install PyFastUtil\n```\n\n### Or, build from source\n\nIf you'd like to build the project from source, follow these steps:\n\n1. Clone the repository:\n    ```shell\n    git clone https://github.com/yourusername/PyFastUtil.git\n    cd PyFastUtil\n    ```\n\n2. Build the project:\n    - On **Windows**:\n      ```shell\n      ./build.cmd\n      ```\n    - On **Linux/macOS**:\n      ```shell\n      ./build.sh\n      ```\n\n## License\n\nThis project is licensed under the **Apache License 2.0**. For more details, see the [LICENSE](LICENSE) file.\n\n## Roadmap\n\n- [ ] Implement `int`, `float`, and `double` ArrayList and LinkedList.\n- [x] Add Numpy support.\n- [x] Provide bindings for SIMD utility functions.\n- [x] Provide raw AVX512 bindings.\n- [x] Perform comprehensive testing and benchmarking.\n- [x] Publish to PyPI.\n\n## Contribution\n\nContributions are welcome! Feel free to submit issues or pull requests. Please note that the project is in its early stages, and we greatly appreciate any feedback or suggestions.\n\n## Acknowledgements\n\nThis project is partially based on the following excellent open-source projects:\n\n- [CPython](https://github.com/python/cpython): The official implementation of the Python interpreter, licensed under the [Python Software Foundation License](https://docs.python.org/3/license.html).\n- [C++ Standard Library (STL)](https://en.cppreference.com/w/cpp): The standard library for C++, providing essential data structures, algorithms, and utilities, licensed under the [ISO C++ Standard](https://isocpp.org/).\n- [cpp-TimSort](https://github.com/timsort/cpp-TimSort): A C++ implementation of the TimSort algorithm, licensed under the [MIT License](https://github.com/timsort/cpp-TimSort/blob/master/LICENSE).\n- [ankerl::unordered_dense](https://github.com/martinus/unordered_dense): A modern, high-performance, low-memory hash table implementation in C++, licensed under the [MIT License](https://github.com/martinus/unordered_dense/blob/main/LICENSE).\n- [qReverse](https://github.com/Wunkolo/qreverse): A high-performance, architecture-optimized array reversal algorithm, released under the [MIT License](https://github.com/martinus/unordered_dense/blob/main/LICENSE).\n\nSpecial thanks to the contributors of these open-source projects!\n\nThe mascot image of the project was created by the artist [kokola](https://x.com/kokola10032) and is used with permission. You can view the original artwork on [X](https://x.com/kokola10032/status/1812480707643506704).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxia-mc%2Fpyfastutil","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxia-mc%2Fpyfastutil","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxia-mc%2Fpyfastutil/lists"}