{"id":29296294,"url":"https://github.com/mghalix/storix","last_synced_at":"2026-06-02T01:00:57.578Z","repository":{"id":303149460,"uuid":"1007992277","full_name":"mghalix/storix","owner":"mghalix","description":"Storage Unix Style for Developers","archived":false,"fork":false,"pushed_at":"2025-12-14T20:33:55.000Z","size":4776,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-20T23:26:30.460Z","etag":null,"topics":["azure","cli","python","sdk","storage"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/storix/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mghalix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-24T21:20:23.000Z","updated_at":"2025-12-14T20:33:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"e2f26ac9-6cf3-42d6-bb37-d747350ab7be","html_url":"https://github.com/mghalix/storix","commit_stats":null,"previous_names":["mghalix/storix"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/mghalix/storix","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mghalix%2Fstorix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mghalix%2Fstorix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mghalix%2Fstorix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mghalix%2Fstorix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mghalix","download_url":"https://codeload.github.com/mghalix/storix/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mghalix%2Fstorix/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33800676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","cli","python","sdk","storage"],"created_at":"2025-07-06T15:10:16.470Z","updated_at":"2026-06-02T01:00:57.559Z","avatar_url":"https://github.com/mghalix.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Storix: Storage for Unix Lovers\n\n**A blazing-fast, secure, and developer-friendly storage abstraction for\nPython.**\n\n[![PyPI version](https://badge.fury.io/py/storix.svg)](https://pypi.org/project/storix/)\n[![GitHub stars](https://img.shields.io/github/stars/mghalix/storix.svg?style=social)](https://github.com/mghalix/storix)\n[![License](https://img.shields.io/github/license/mghalix/storix.svg)](https://github.com/mghalix/storix/blob/main/LICENSE)\n\n![storix-icon](./.github/assets/storix-icon.png)\n\n---\n\n## 🎬 Demo\n\n![Storix Interactive Shell Demo](./.github/assets/demo.gif)\n\n_Watch storix in action: navigating files, creating directories, and using the\ninteractive shell mode._\n\n---\n\n## 🚀 Getting Started\n\n### 1. Install Storix\n\nChoose the installation that fits your needs:\n\n```bash\n# Basic (local filesystem only)\nuv add storix\n\n# With CLI tools\nuv add \"storix[cli]\"\n\n# With Azure support\nuv add \"storix[azure]\"\n\n# Everything included\nuv add \"storix[all]\"\n```\n\n### 2. Configure (Optional)\n\nCreate a `.env` file for custom settings:\n\n```bash\ncp env.example .env\n# Edit .env with your preferences\n```\n\n### 3. Start Using\n\n```python\nfrom storix import get_storage\n\nfs = get_storage()\nfs.touch(\"hello.txt\", \"Hello, Storix!\")\n```\n\n---\n\n## 💡 Quick Examples\n\n```python\n# Basic file operations\nfrom storix import get_storage\n\nfs = get_storage()\nfs.touch(\"hello.txt\", \"Hello, World!\")\ncontent = fs.cat(\"hello.txt\").decode()\nprint(content)  # Hello, World!\n\n# List files\nfiles = fs.ls(\"/\")\nprint(files)  # [StorixPath('hello.txt')]\n\n# Create directories\nfs.mkdir(\"mydata\", parents=True)\nfs.touch(\"mydata/config.json\", '{\"key\": \"value\"}')\n```\n\n```python\n# Async operations\nimport asyncio\n\nfrom storix.aio import get_storage\n\nasync def main():\n    fs = get_storage()\n    await fs.touch(\"async.txt\", \"Async is easy!\")\n    content = await fs.cat(\"async.txt\")\n    print(content.decode())  # Async is easy!\n\nasyncio.run(main())\n```\n\n### More operations\n\n```python\nfrom storix import get_storage\n\nfs = get_storage()\nfs.mkdir(\"logs\", parents=True)\n\n# Stream writes: pass any iterable/generator of chunks\nfs.echo((f\"line {i}\\n\" for i in range(3)), \"logs/app.log\")\nfs.echo(\"another line\\n\", \"logs/app.log\", mode=\"a\")\n\n# Metadata and sizes\ninfo = fs.stat(\"logs/app.log\")\nprint(info.file_kind, info.size)  # file  ...\nprint(fs.du(\"logs/app.log\"))      # bytes\n\n# Tree view\nprint(fs.tree(\"/\", abs=False))\n```\n\n---\n\n## 🛠️ Quickstart\n\n### Install\n\n```bash\n# Basic installation (local filesystem only)\nuv add storix\n# or\npip install storix\n\n# With CLI support\nuv add \"storix[cli]\"\n# or\npip install \"storix[cli]\"\n\n# With Azure support\nuv add \"storix[azure]\"\n# or\npip install \"storix[azure]\"\n\n# With everything (CLI + Azure)\nuv add \"storix[all]\"\n# or\npip install \"storix[all]\"\n```\n\n### Development Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/mghalix/storix.git\ncd storix\n\n# Install dependencies and setup development environment\nuv sync\nuv run pip install -e .\n```\n\n### CLI Configuration (Optional)\n\nCreate a `.env` file to customize your storage settings. Storix will\nautomatically find your `.env` file in:\n\n1. **Current directory** (where you run the command)\n2. **Parent directories** (searches up the directory tree)\n3. **Home directory** (`~/.storix/.env`)\n\n```bash\n# Copy the example configuration\ncp env.example .env\n```\n\nThen edit `.env` with your preferences:\n\n```env\n# Storage Provider (local or azure)\nSTORAGE_PROVIDER=local\n\n# Initial paths for different providers\nSTORAGE_INITIAL_PATH=.\nSTORAGE_INITIAL_PATH_LOCAL=/path/to/your/data\nSTORAGE_INITIAL_PATH_AZURE=/your/azure/path\n\n# Azure credentials (only needed for azure provider)\nADLSG2_CONTAINER_NAME=my-container\nADLSG2_ACCOUNT_NAME=my-storage-account\nADLSG2_TOKEN=your-sas-token-or-account-key\n```\n\n**Smart Discovery**: You can run `sx` from any subdirectory of your project and\nit will find your `.env` file automatically!\n\n**Path Configuration**:\n\n- `STORAGE_INITIAL_PATH` is the shared default path for all providers\n- `STORAGE_INITIAL_PATH_LOCAL` and `STORAGE_INITIAL_PATH_AZURE` override the\n  shared path for specific providers\n- Provider-specific paths take precedence over the shared path\n\n### CLI Usage\n\n```bash\n# List files in current directory\nsx ls\n\n# Interactive shell\nsx\n\n# Switch providers\nsx -p local ls\n\n# Create directory\nsx mkdir mydata\n\n# Upload file\nsx upload local_file.txt remote_file.txt\n```\n\n### Basic Usage\n\n```python\nfrom storix import LocalFilesystem\n\nfs = LocalFilesystem(\"/tmp/mydata\")\nfs.touch(\"hello.txt\", \"Hello, world!\")\nprint(fs.cat(\"hello.txt\").decode())\n```\n\n### Sandboxed Usage\n\n```python\nfrom storix import LocalFilesystem\n\nfs = LocalFilesystem(\"/tmp/sandbox\", sandboxed=True)\nfs.touch(\"/secret.txt\", \"sandboxed!\")\nprint(fs.ls(\"/\"))  # [StorixPath('secret.txt')]\n```\n\n### Async Usage\n\n```python\nfrom storix.aio import LocalFilesystem\n\nfs = LocalFilesystem(\"/tmp/mydata\")\nawait fs.touch(\"async.txt\", \"Async is easy!\")\nprint((await fs.cat(\"async.txt\")).decode())\n```\n\n---\n\n## 🗄️ Supported Backends\n\n| Backend                          | Sync | Async | Sandboxed | Status  |\n| -------------------------------- | ---- | ----- | --------- | ------- |\n| **Local Filesystem**             | ✅   | ✅    | ✅        | Beta    |\n| **Azure Data Lake Storage Gen2** | ✅   | ✅    | ✅        | Beta    |\n| **AWS S3**                       | ❌   | ❌    | ❌        | Planned |\n| **Google Cloud Storage**         | ❌   | ❌    | ❌        | Planned |\n\n## 🔌 Integrations\n\n### FastAPI Integration\n\nStorix integrates seamlessly with FastAPI for dependency injection and async\nfile operations:\n\n```python\nfrom typing import AsyncGenerator\nfrom fastapi import Depends, FastAPI\nfrom typing_extensions import Annotated\n\nimport storix as sx\n\napp = FastAPI()\n\nasync def get_fs() -\u003e AsyncGenerator[sx.Storage, None]:\n    \"\"\"Dependency to provide storage service.\"\"\"\n    async with sx.get_storage() as fs:\n        yield fs\n\nStorageDep = Annotated[sx.Storage, Depends(get_fs)]\n\n@app.get(\"/files/{file_path:path}\")\nasync def read_file(file_path: str, fs: StorageDep):\n    \"\"\"Read a file from storage.\"\"\"\n    content = await fs.cat(file_path)\n    return {\"content\": content.decode()}\n\n@app.post(\"/files/{file_path:path}\")\nasync def write_file(file_path: str, content: str, fs: StorageDep):\n    \"\"\"Write content to a file.\"\"\"\n    success = await fs.touch(file_path, content)\n    return {\"success\": success}\n```\n\n**Configuration**: Set `STORAGE_PROVIDER` in your `.env` file to choose your backend:\n\n```sh\n# For local filesystem\nSTORAGE_PROVIDER=local\nSTORAGE_INITIAL_PATH_LOCAL=/path/to/local/data\n\n# For Azure Data Lake\nSTORAGE_PROVIDER=azure\nSTORAGE_INITIAL_PATH_AZURE=/your/azure/path\nADLSG2_ACCOUNT_NAME=your_account\nADLSG2_TOKEN=your_token\nADLSG2_CONTAINER_NAME=your_container\n```\n\n\u003e Notes\n\u003e\n\u003e \u003e **Local:**\n\u003e \u003e\n\u003e \u003e - if `STORAGE_INITIAL_PATH` set to \"~\", gets mapped to your home directory\n\u003e \u003e - if `STORAGE_INITIAL_PATH` set to \".\", gets mapped to your current working directory\n\u003e\n\u003e \u003e **Azure:**\n\u003e \u003e\n\u003e \u003e - if `STORAGE_INITIAL_PATH` set to \"~\", gets mapped to root \"/\"\n\u003e \u003e - if `STORAGE_INITIAL_PATH` set to \".\", gets mapped to root \"/\"\n\n## 🖼️ Media Support\n\nStorix makes it effortless to store and retrieve images or any binary media\nfiles—just use Python's `bytes` type.\n\n### Write an Image to Storage\n\n```python\nfrom storix import get_storage\n\nfs = get_storage()\nwith open(\"my_photo.jpg\", \"rb\") as img_file:\n    img_bytes = img_file.read()\nfs.touch(\"photos/profile.jpg\", img_bytes)\n```\n\n### Read an Image from Storage\n\n```python\nfrom storix import get_storage\n\nfs = get_storage()\nimg_bytes = fs.cat(\"photos/profile.jpg\")\n# Now you can use img_bytes with PIL, OpenCV, or send it in a web response...\n```\n\n### Async Example\n\n```python\nfrom storix.aio import get_storage\n\nfs = get_storage()\nasync with fs:\n    img_bytes = await fs.cat(\"photos/profile.jpg\")\n    await fs.touch(\"photos/backup.jpg\", img_bytes)\n```\n\n**It's that simple:**\n\n- No special APIs for media—just use `bytes` for any file type.\n- Works for images, audio, video, or any binary data.\n- Seamless support for both sync and async code.\n\n## ✨ Features\n\n- **Sync \u0026 Async APIs:**\n  Use `from storix import LocalFilesystem` or `from storix.aio import\nLocalFilesystem` — just add `await` for async!\n- **Sandboxing:**\n  Restrict all file operations to a virtual root, blocking path traversal and\n  symlink escapes.\n- **Consistent Path Handling:**\n  Absolute and relative paths, `cd`, `ls`, and more — just like a shell.\n- **Decorator Support:**\n  Automatic path conversion for your own functions.\n- **Easy Migration:**\n  Switch from sync to async in seconds.\n- **CLI Tool:**\n  Manage files and sandboxes from your terminal.\n\n---\n\n## 🚀 Why storix\n\n- **Unified API:** Seamless sync and async support with identical interfaces.\n- **Rock-solid sandboxing:** Secure your file operations with robust path\n  traversal and symlink protection.\n- **Plug-and-play:** Instantly switch between local and cloud (Azure) backends.\n- **CLI included:** Script and automate storage tasks from the command line.\n- **Extensible:** Clean, modern codebase ready for new providers and features.\n- **Tested \u0026 Secure:** Comprehensive test suite and security-first design.\n\n---\n\n## 🔒 Security\n\n- **Path Traversal Protection:**\n  All `../` and symlink escapes are blocked.\n- **Virtual Root:**\n  Sandboxed mode makes `/` map to your chosen directory.\n- **Symlink Safety:**\n  Symlinks are resolved and validated before access.\n\n---\n\n## 🙋 Need Help or Found a Bug?\n\nIf you run into any issues, have questions, or spot a bug, please [open an\nissue](https://github.com/mghalix/storix/issues) on the GitHub repository.\n\nWe welcome all feedback and contributions—your input helps make Storix better\nfor everyone!\n\n---\n\n## 🧑‍💻 Contributing\n\nContributions are welcome and appreciated. If you have ideas for improvements,\nnew features, or spot something that could be better, feel free to open an\nissue or a pull request.\n\nStorix aims to be robust, secure, and easy to use. We value clear code, good\ndocumentation, and thoughtful discussion. Whether you want to add a new\nprovider, enhance the CLI, share real-world examples, or review security, your\ninput is valued.\n\nTo get started:\n\n- Star and fork the repository\n- Open an issue to discuss your idea or report a bug\n- Submit a pull request with your proposed changes\n\nFor more details, see the documentation or join the discussion on GitHub.\n\n---\n\n## 📣 Spread the Word\n\nIf you like `storix` ⭐ star the repo, share it, and help us grow the community!\n\n---\n\n## 📚 Documentation\n\n- [Sandbox Implementation Details](docs/SANDBOX_IMPLEMENTATION.md)\n- [Async Migration Guide](docs/ASYNC_MIGRATION.md)\n- [Release Notes](release-notes.md)\n\n---\n\n## 🧪 Testing\n\n```bash\n# Run all tests\nuv run pytest\n\n# Run with coverage\nuv run pytest --cov=storix\n\n# Run specific test categories\nuv run pytest -m \"not slow\"  # Skip slow tests\nuv run pytest -m integration # Run only integration tests\n```\n\n## 🗺️ Roadmap\n\nPlanned and upcoming features:\n\n- **`mv` support for directories in Azure sync/async providers**\n- **`du` (Disk Usage):**\n  - Calculate and display the size of files and directories, similar to the\n    Unix `du` command.\n- **`stat` (File Metadata):**\n  - Retrieve and display detailed file metadata (size, permissions, timestamps, etc.),\n    similar to the Unix `stat` command.\n  - Normalize the result using a shared model between the sync/async interfaces.\n- `touch`, `mkdir`, `rm`, `rmdir` should take **`*paths`** instead of a single\n  path arg\n- **Storage Tree Structure (`tree`):**\n  - Not just for visualization—enables programmatic operations such as copying,\n    iterating, or transforming entire directory trees easily.\n  - Will likely use a tree data structure (e.g.,\n    [anytree](https://anytree.readthedocs.io/), TBD) to represent and manipulate\n    storage hierarchies efficiently.\n  - Should be streaming for performance (leverage generators)\n- **Advanced CLI Features:**\n  - Enhanced command-line tools for scripting and automation.\n- **Performance Improvements:**\n  - Further optimize for speed and scalability.\n- **Additional Cloud Providers:**\n  - Add support for more cloud storage backends (e.g., S3, GCS). Local\n    filesystem support is already included.\n- **Auto Completions in the storix REPL (Interactive Shell)**\n- **Improve validation strategy**\n  - Reduce code duplication in validation logic between sync/async providers\n  - Evaluate shared utility functions, mixins, or decorator patterns\n  - Focus on common operations like `rm`, `rmdir`, `touch` validation\n  - Ensure consistent error messages and behavior across providers\n- **Storage Connection Pool**\n  - Implement connection pooling for cloud storage providers to improve performance\n  - Reuse connections across requests to reduce latency and overhead\n  - Configurable pool sizes and connection timeouts\n\n---\n\n## 📝 License\n\nStorix is licensed under the Apache 2.0 License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmghalix%2Fstorix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmghalix%2Fstorix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmghalix%2Fstorix/lists"}