{"id":50115088,"url":"https://github.com/nightmarewalker/d-memfs","last_synced_at":"2026-05-23T14:04:38.879Z","repository":{"id":341247661,"uuid":"1169416730","full_name":"nightmarewalker/D-MemFS","owner":"nightmarewalker","description":"In-process virtual filesystem with hard quota for Python","archived":false,"fork":false,"pushed_at":"2026-03-09T21:26:54.000Z","size":322,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-10T03:37:26.282Z","etag":null,"topics":["asyncio","etl","filesystem","free-threaded-python","hard-quota","in-memory","in-process","memory-filesystem","pure-python","python","quota","resource-management","sandbox","standard-library","temporary-filesystem","testing-tools","thread-safe","vfs","virtual-filesystem","zero-dependencies"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nightmarewalker.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-28T16:53:39.000Z","updated_at":"2026-03-09T21:26:58.000Z","dependencies_parsed_at":"2026-02-28T20:01:31.080Z","dependency_job_id":null,"html_url":"https://github.com/nightmarewalker/D-MemFS","commit_stats":null,"previous_names":["nightmarewalker/d-memfs"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nightmarewalker/D-MemFS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nightmarewalker%2FD-MemFS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nightmarewalker%2FD-MemFS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nightmarewalker%2FD-MemFS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nightmarewalker%2FD-MemFS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nightmarewalker","download_url":"https://codeload.github.com/nightmarewalker/D-MemFS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nightmarewalker%2FD-MemFS/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33398419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T04:15:53.637Z","status":"ssl_error","status_checked_at":"2026-05-23T04:15:53.242Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","etl","filesystem","free-threaded-python","hard-quota","in-memory","in-process","memory-filesystem","pure-python","python","quota","resource-management","sandbox","standard-library","temporary-filesystem","testing-tools","thread-safe","vfs","virtual-filesystem","zero-dependencies"],"created_at":"2026-05-23T14:04:32.562Z","updated_at":"2026-05-23T14:04:38.866Z","avatar_url":"https://github.com/nightmarewalker.png","language":"Python","funding_links":["https://github.com/sponsors/nightmarewalker"],"categories":[],"sub_categories":[],"readme":"# D-MemFS\n\n**An in-process virtual filesystem with hard quota enforcement for Python.**\n\n[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)\n[![Tests](https://github.com/nightmarewalker/D-MemFS/actions/workflows/test.yml/badge.svg)](https://github.com/nightmarewalker/D-MemFS/actions/workflows/test.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/nightmarewalker/D-MemFS/blob/main/LICENSE)\n[![Zero dependencies (runtime)](https://img.shields.io/badge/runtime_deps-none-brightgreen.svg)]()\n[![PyPI version](https://img.shields.io/pypi/v/D-MemFS.svg)](https://pypi.org/project/D-MemFS/)\n[![Socket Badge](https://socket.dev/api/badge/pypi/package/D-MemFS)](https://socket.dev/pypi/package/D-MemFS)\n\nLanguages: [English](https://github.com/nightmarewalker/D-MemFS/blob/main/README.md) | [Japanese](https://github.com/nightmarewalker/D-MemFS/blob/main/README_ja.md)\n\n---\n\n## Proven Quality\n\n| Metric | Details |\n|---|---|\n| 🧪 **Robustness** | 436 tests with 97% code coverage |\n| 🔒 **Verified Safety** | 98, 100×4 — top scores across all security categories (Socket.dev) |\n| 🌟 **Community** | [Discussed on `r/Python`](https://www.reddit.com/r/Python/comments/1rrqr8z/i_built_an_inmemory_virtual_filesystem_for_python/) with highly positive reception |\n---\n\n## Why MFS?\n\n`MemoryFileSystem` gives you a fully isolated filesystem-like workspace inside a Python process.\n\n- Hard quota (`MFSQuotaExceededError`) to reject oversized writes before OOM\n- Memory Guard to detect physical RAM exhaustion before it causes OOM kills\n- **Full filesystem semantics**: Hierarchical directories and multi-file operations (`import_tree`, `copy_tree`, `move`)\n- File-level RW locking + global structure lock for thread-safe operations\n- Free-threaded Python compatible (`PYTHON_GIL=0`) — stress-tested under 50-thread contention\n- Async wrapper (`AsyncMemoryFileSystem`) powered by `asyncio.to_thread`\n- Zero runtime dependencies (standard library only)\n- **No admin/root privileges required** — works on locked-down CI runners, containers, and shared machines where OS-level RAM disks are not an option\n- **436 tests, 97% coverage** across 3 OS (Linux / Windows / macOS) × 3 Python versions (3.11–3.13, including free-threaded 3.13t)\n\nThis is useful when `io.BytesIO` is too primitive (single buffer), and OS-level RAM disks/tmpfs are impractical (permissions, container policy, Windows driver friction). Ideal for **CI pipeline acceleration** — eliminate disk I/O from test suites and data processing without any infrastructure changes.\n\n**Note on Architectural Boundary:** This is strictly an in-process tool. External subprocesses (CLI tools) cannot access these files via standard OS paths. If your pipeline relies heavily on passing files to external binaries, an OS-level RAM disk (`tmpfs`) is the correct tool. D-MemFS shines when accelerating Python-native test suites or internal data pipelines.\n\n---\n\n### Archive Extraction\nExtract ZIP/TAR archives directly into D-MemFS using the built-in `expand_archive()` (atomic, all-or-nothing) or `expand_archive_streaming()` (low-memory, incremental). Custom archive formats are supported via the pluggable `ArchiveAdapter` interface. A low-level manual extraction example using `open()`/`write()` is also included as a reference for advanced use cases.\n* 📝 **Tutorial:** [`examples/archive_extraction.md`](examples/archive_extraction.md)\n\n### CI/CD Pipelines \u0026 Test Debugging\nSpeed up your pipeline by running heavy file I/O tests entirely in memory. If a test fails, export the complete virtual filesystem state to a physical directory (`export_tree`) for easy post-mortem debugging.\n* 📝 **Tutorial:** [`examples/ci_debug_export.md`](examples/ci_debug_export.md)\n\n### High-Speed SQLite Test Fixtures\nEliminate disk I/O bottlenecks in your database test suites. Generate a master SQLite database state once, store it in D-MemFS, and load it instantly for each individual test. Ensure perfect test isolation with zero disk wear and zero cleanup.\n* 📝 **Tutorial:** [`examples/sqlite_test_fixtures.md`](examples/sqlite_test_fixtures.md)\n\n### SQLite Shared In-Memory DB Auto-Persistence\nCombine SQLite's shared-cache in-memory databases (`mode=memory\u0026cache=shared`) with D-MemFS. This allows multiple concurrent connections to share a single live database, while automatically serializing its state to D-MemFS when the last connection closes and restoring it upon the next connection. Ideal for dynamic applications and ETL pipelines.\n* 📝 **Tutorial:** [`examples/sqlite_shared_store.md`](examples/sqlite_shared_store.md)\n\n### Multi-threaded Data Staging (ETL)\nUse D-MemFS as a volatile, high-speed staging area for ETL pipelines. It features built-in, thread-safe file locking, ensuring safe concurrent data processing.\n* 📝 **Tutorial:** [`examples/etl_staging_multithread.md`](examples/etl_staging_multithread.md)\n\n### Safe Large File Processing (Serverless/Sandboxed)\nProcess massive files chunk-by-chunk using our Memory Guard. Safely raise an exception *before* the host OS hits an Out-Of-Memory (OOM) crash, which is crucial for environments without OS-level RAM disks.\n* 📝 **Tutorial:** [`examples/memory_guard_streaming.md`](examples/memory_guard_streaming.md)\n\n---\n\n## Installation\n\n```bash\npip install D-MemFS\n```\n\nRequirements: Python 3.11+\n\n---\n\n## Quick Start\n\n```python\nfrom dmemfs import MemoryFileSystem, MFSQuotaExceededError\n\nmfs = MemoryFileSystem(max_quota=64 * 1024 * 1024)\n\nmfs.mkdir(\"/data\")\nwith mfs.open(\"/data/hello.bin\", \"wb\") as f:\n    f.write(b\"hello\")\n\nwith mfs.open(\"/data/hello.bin\", \"rb\") as f:\n    print(f.read())  # b\"hello\"\n\nprint(mfs.listdir(\"/data\"))\nprint(mfs.is_file(\"/data/hello.bin\"))  # True\n\ntry:\n    with mfs.open(\"/huge.bin\", \"wb\") as f:\n        f.write(bytes(512 * 1024 * 1024))\nexcept MFSQuotaExceededError as e:\n    print(e)\n```\n\n---\n\n## API Highlights\n\n### `MemoryFileSystem`\n\n- `open(path, mode, *, preallocate=0, lock_timeout=None)`\n- `mkdir`, `remove`, `rmtree`, `rename`, `move`, `copy`, `copy_tree`\n- `listdir`, `exists`, `is_dir`, `is_file`, `walk`, `glob`\n- `stat`, `stats`, `get_size`\n- `export_as_bytesio`, `export_tree`, `iter_export_tree`, `import_tree`\n\n### Archive Extraction Functions\n\n- `expand_archive(mfs, source, dest, *, on_conflict, adapter, adapters)` — atomic extraction via `import_tree()`\n- `expand_archive_streaming(mfs, source, dest, *, on_conflict, adapter, adapters)` — streaming extraction, returns write count\n- `ArchiveAdapter` — base class for pluggable archive format support (built-in: `ZipAdapter`, `TarAdapter`)\n\n**Constructor parameters:**\n- `max_quota` (default `256 MiB`): byte quota for file data\n- `max_nodes` (default `None`): optional cap on total node count (files + directories). Raises `MFSNodeLimitExceededError` when exceeded.\n- `default_storage` (default `\"auto\"`): storage backend for new files — `\"auto\"` / `\"sequential\"` / `\"random_access\"`\n- `promotion_hard_limit` (default `None`): byte threshold above which Sequential→RandomAccess auto-promotion is suppressed (`None` uses the built-in 512 MiB limit)\n- `chunk_overhead_override` (default `None`): override the per-chunk overhead estimate used for quota accounting\n- `default_lock_timeout` (default `30.0`): default timeout in seconds for file-lock acquisition during `open()`. Use `None` to wait indefinitely.\n- `memory_guard` (default `\"none\"`): physical memory protection mode — `\"none\"` / `\"init\"` / `\"per_write\"`\n- `memory_guard_action` (default `\"warn\"`): action when the guard triggers — `\"warn\"` (`ResourceWarning`) / `\"raise\"` (`MemoryError`)\n- `memory_guard_interval` (default `1.0`): minimum seconds between OS memory queries (`\"per_write\"` only)\n\n\u003e **Note:** The `BytesIO` returned by `export_as_bytesio()` is outside quota management.\n\u003e Exporting large files may consume significant process memory beyond the configured quota limit.\n\n\u003e **Note — Quota and free-threaded Python:**\n\u003e The per-chunk overhead estimate used for quota accounting is calibrated at import time\n\u003e via `sys.getsizeof()`. Free-threaded Python (3.13t, `PYTHON_GIL=0`) has larger object\n\u003e headers than the standard build, so `CHUNK_OVERHEAD_ESTIMATE` is higher (~117 bytes vs\n\u003e ~93 bytes on CPython 3.13). This means the same `max_quota` yields slightly less\n\u003e effective storage capacity on free-threaded builds, especially for workloads with many\n\u003e small files or small appends. This is not a bug — it reflects real memory consumption.\n\u003e To ensure consistent behaviour across builds, use `chunk_overhead_override` to pin the\n\u003e value, or inspect `stats()[\"overhead_per_chunk_estimate\"]` at runtime.\n\nSupported binary modes: `rb`, `wb`, `ab`, `r+b`, `xb`\n\n## Memory Guard\n\nMFS enforces a logical quota, but that quota can still be configured larger than the\ncurrently available physical RAM. `memory_guard` provides an optional safety net.\n\n```python\nfrom dmemfs import MemoryFileSystem\n\n# Warn if max_quota exceeds available RAM\nmfs = MemoryFileSystem(max_quota=8 * 1024**3, memory_guard=\"init\")\n\n# Raise MemoryError before writes when RAM is insufficient\nmfs = MemoryFileSystem(\n    max_quota=8 * 1024**3,\n    memory_guard=\"per_write\",\n    memory_guard_action=\"raise\",\n)\n```\n\n| Mode | Initialization | Each Write | Overhead |\n|---|---|---|---|\n| `\"none\"` | — | — | Zero |\n| `\"init\"` | Check once | — | Negligible |\n| `\"per_write\"` | Check once | Cached check | About 1 OS call/sec |\n\nWhen `memory_guard_action=\"warn\"`, the guard emits `ResourceWarning` and allows the operation to continue.\nWhen `memory_guard_action=\"raise\"`, the guard rejects the operation with `MemoryError` before the actual allocation path.\n\n`AsyncMemoryFileSystem` accepts the same constructor parameters and forwards them to the synchronous implementation.\n\n### `MemoryFileHandle`\n\n- `io.RawIOBase`-compatible binary handle\n- `read`, `write`, `seek`, `tell`, `truncate`, `flush`, `close`\n- `readinto`\n- file-like capability checks: `readable`, `writable`, `seekable`\n\n`flush()` is intentionally a no-op (compatibility API for file-like integrations).\n\n### `stat()` return (`MFSStatResult`)\n\n`size`, `created_at`, `modified_at`, `generation`, `is_dir`\n\n- Supports both files and directories\n- For directories: `size=0`, `generation=0`, `is_dir=True`\n\n---\n\n## Text Mode\n\nD-MemFS natively operates in binary mode. For text I/O, use `MFSTextHandle`:\n\n```python\nfrom dmemfs import MemoryFileSystem, MFSTextHandle\n\nmfs = MemoryFileSystem()\nmfs.mkdir(\"/data\")\n\n# Write text\nwith mfs.open(\"/data/hello.bin\", \"wb\") as f:\n    th = MFSTextHandle(f, encoding=\"utf-8\")\n    th.write(\"こんにちは世界\\n\")\n    th.write(\"Hello, World!\\n\")\n\n# Read text line by line\nwith mfs.open(\"/data/hello.bin\", \"rb\") as f:\n    th = MFSTextHandle(f, encoding=\"utf-8\")\n    for line in th:\n        print(line, end=\"\")\n```\n\n`MFSTextHandle` is a thin, bufferless wrapper. It encodes on `write()` and decodes on `read()` / `readline()`. `read(size)` counts characters, not bytes, so multibyte text can be read safely without splitting code points. Unlike `io.TextIOWrapper`, it introduces no buffering issues when used with `MemoryFileHandle`.\n\n---\n\n## Async Usage\n\n```python\nfrom dmemfs import AsyncMemoryFileSystem\n\nasync def run() -\u003e None:\n    mfs = AsyncMemoryFileSystem(max_quota=64 * 1024 * 1024)\n    await mfs.mkdir(\"/a\")\n    async with await mfs.open(\"/a/f.bin\", \"wb\") as f:\n        await f.write(b\"data\")\n    async with await mfs.open(\"/a/f.bin\", \"rb\") as f:\n        print(await f.read())\n```\n\n---\n\n## Concurrency and Locking Notes\n\n- Path/tree operations are guarded by `_global_lock`.\n- File access is guarded by per-file `ReadWriteLock`.\n- `lock_timeout` behavior:\n  - `None`: block indefinitely\n  - `0.0`: try-lock (fail immediately with `BlockingIOError`)\n  - `\u003e 0`: timeout in seconds, then `BlockingIOError`\n- Current `ReadWriteLock` is non-fair: under sustained read load, writers can starve.\n\n### Thread Safety of File Handles\n\nWhile the core `MemoryFileSystem` is thread-safe, individual file handles (`MemoryFileHandle`, `MFSTextHandle`, `AsyncMemoryFileHandle`) are **not thread-safe** when shared concurrently. \n\n- **The Reason**: Like standard OS file descriptors, handles maintain internal mutable state (e.g., read/write cursors, text decode buffers). Concurrent access will corrupt this state.\n- **The Rule**: Always acquire a new handle per thread or async task (e.g., call `mfs.open()` inside your worker function). Do not pass open handles across thread boundaries.\n\n### Operational guidance\n\n- Keep lock hold duration short\n- Set an explicit `lock_timeout` in latency-sensitive code paths\n- `walk()` and `glob()` provide weak consistency: each directory level is\n  snapshotted under `_global_lock`, but the overall traversal is NOT atomic.\n  Concurrent structural changes may produce inconsistent results.\n\n---\n\n## Benchmarks\n\nMinimal benchmark tooling is included:\n\n- D-MemFS vs `io.BytesIO` vs `PyFilesystem2 (MemoryFS)` vs `tempfile(RAMDisk)` / `tempfile(SSD)`\n- Cases: many-small-files, stream write/read, random access, large stream, deep tree\n- Optional report output to `benchmarks/results/`\n\n\u003e **Note:** As of setuptools 82 (February 2026), `pyfilesystem2` fails to import due to a known upstream issue ([#597](https://github.com/PyFilesystem/pyfilesystem2/issues/597)). Benchmark results including PyFilesystem2 were measured with setuptools ≤ 81 and are valid as historical comparison data.\n\nRun:\n\n```bash\n# With explicit RAM disk and SSD directories for tempfile comparison:\nuvx --with-requirements requirements.txt --with-editable . python benchmarks/compare_backends.py --ramdisk-dir R:\\Temp --ssd-dir C:\\TempX --save-md auto --save-json auto\n```\n\nSee `BENCHMARK.md` for details.\n\nLatest benchmark snapshot:\n\n- [benchmark_current_result.md](https://github.com/nightmarewalker/D-MemFS/blob/main/benchmarks/results/benchmark_current_result.md)\n\n---\n\n## Testing and Coverage\n\nTest execution and dev flow are documented in `TESTING.md`.\n\nTypical local run:\n\n```bash\nuv pip compile requirements.in -o requirements.txt\nuvx --with-requirements requirements.txt --with-editable . pytest tests/ -v --timeout=30 --cov=dmemfs --cov-report=xml --cov-report=term-missing\n```\n\nCI (`.github/workflows/test.yml`) runs tests with coverage XML generation.\n\n---\n\n## API Docs Generation\n\nAPI docs can be generated as Markdown (viewable on GitHub) using `pydoc-markdown`:\n\n```bash\nuvx --with pydoc-markdown --with-editable . pydoc-markdown '{\n  loaders: [{type: python, search_path: [.]}],\n  processors: [{type: filter, expression: \"default()\"}],\n  renderer: {type: markdown, filename: docs/api_md/index.md}\n}'\n```\n\nOr as HTML using `pdoc` (local browsing only):\n\n```bash\nuvx --with-requirements requirements.txt pdoc dmemfs -o docs/api\n```\n\n- [API Reference (Markdown)](https://github.com/nightmarewalker/D-MemFS/blob/main/docs/api_md/index.md)\n\n---\n\n## Compatibility and Non-Goals\n\n- Core `open()` is binary-only (`rb`, `wb`, `ab`, `r+b`, `xb`). Text I/O is available via the `MFSTextHandle` wrapper.\n- No symlink/hardlink support — intentionally omitted to eliminate path traversal loops and structural complexity (same rationale as `pathlib.PurePath`).\n- No direct `pathlib.Path` / `os.PathLike` API — MFS paths are virtual and must not be confused with host filesystem paths. Accepting `os.PathLike` would allow third-party libraries or a plain `open()` call to silently treat an MFS virtual path as a real OS path, potentially issuing unintended syscalls against the host filesystem. All paths must be plain `str` with POSIX-style absolute notation (e.g. `\"/data/file.txt\"`).\n- No kernel filesystem integration (intentionally in-process only)\n- No exhaustive archive format support — core handles zip and tar (standard library) only. For other formats (7z, RAR, etc.), you can write your own adapter. See [`examples/archive_extraction.md`](examples/archive_extraction.md) for details.\n- No password-protected / encrypted archive support\n- Archive extraction functions are sync-only. Use `asyncio.to_thread()` in async code.\n\nAuto-promotion behavior:\n\n- By default (`default_storage=\"auto\"`), new files start as `SequentialMemoryFile` and auto-promote to `RandomAccessMemoryFile` when random writes are detected.\n- Promotion is one-way (no downgrade back to sequential).\n- Use `default_storage=\"sequential\"` or `\"random_access\"` to fix the backend at construction; use `promotion_hard_limit` to suppress auto-promotion above a byte threshold.\n- Storage promotion temporarily doubles memory usage for the promoted file. The quota system accounts for this, but process-level memory may spike briefly.\n\nSecurity note: In-memory data may be written to physical disk via OS swap\nor core dumps. MFS does not provide memory-locking (e.g., mlock) or\nsecure erasure. Do not rely on MFS alone for sensitive data isolation.\n\n---\n\n## Exception Reference\n\n| Exception | Typical cause |\n|---|---|\n| `MFSQuotaExceededError` | write/import/copy would exceed quota |\n| `MFSNodeLimitExceededError` | node count would exceed `max_nodes` (subclass of `MFSQuotaExceededError`) |\n| `FileNotFoundError` | path missing |\n| `FileExistsError` | creation target already exists |\n| `IsADirectoryError` | file operation on directory |\n| `NotADirectoryError` | directory operation on file |\n| `BlockingIOError` | lock timeout or open-file conflict |\n| `io.UnsupportedOperation` | mode mismatch / unsupported operation |\n| `ValueError` | invalid mode/path/seek/truncate arguments |\n\n---\n\n## Testing with pytest\n\nD-MemFS ships a pytest plugin that provides an `mfs` fixture:\n\n```python\n# conftest.py — register the plugin explicitly\npytest_plugins = [\"dmemfs._pytest_plugin\"]\n```\n\n\u003e **Note:** The plugin is **not** auto-discovered. Users must declare it in `conftest.py` to opt in.\n\n```python\n# test_example.py\ndef test_write_read(mfs):\n    mfs.mkdir(\"/tmp\")\n    with mfs.open(\"/tmp/hello.txt\", \"wb\") as f:\n        f.write(b\"hello\")\n    with mfs.open(\"/tmp/hello.txt\", \"rb\") as f:\n        assert f.read() == b\"hello\"\n```\n\n---\n\n## Development Notes\n\nDesign documents (Japanese):\n\n- [Architecture Spec v13](https://github.com/nightmarewalker/D-MemFS/blob/main/docs/design/spec_v13.md) — API design, internal structure, CI matrix\n- [Architecture Spec v14](https://github.com/nightmarewalker/D-MemFS/blob/main/docs/design/spec_v14.md) — MemoryGuard-integrated architecture spec\n- [Architecture Spec v15](https://github.com/nightmarewalker/D-MemFS/blob/main/docs/design/spec_v15.md) — MemoryGuard-integrated architecture spec\n- [Detailed Design Spec v3](https://github.com/nightmarewalker/D-MemFS/blob/main/docs/design/DetailedDesignSpec_v3.md) — component-level design and rationale\n- [Test Design Spec v3](https://github.com/nightmarewalker/D-MemFS/blob/main/docs/design/DetailedDesignSpec_test_v3.md) — test case table and pseudocode\n\n\u003e These documents are written in Japanese and serve as internal design references.\n\n---\n\n## Performance Summary\n\nKey results from the included benchmark (300 small files × 4 KiB, 16 MiB stream, 512 MiB large stream):\n\n| Case | D-MemFS (ms) | BytesIO (ms) | tempfile(RAMDisk) (ms) | tempfile(SSD) (ms) |\n|---|---:|---:|---:|---:|\n| small_files_rw | 51 | 6 | 207 | 267 |\n| stream_write_read | 81 | 62 | 20 | 21 |\n| random_access_rw | **34** | 82 | 37 | 35 |\n| large_stream_write_read | **529** | 2 258 | 514 | 541 |\n| many_files_random_read | 1 280 | 212 | 6 310 | 8 601 |\n| deep_tree_read | 224 | 3 | 346 | 361 |\n\nD-MemFS incurs a small overhead on tiny-file workloads but delivers significantly better performance on large streams and random-access patterns compared with `BytesIO`. See `BENCHMARK.md` and [benchmark_current_result.md](https://github.com/nightmarewalker/D-MemFS/blob/main/benchmarks/results/benchmark_current_result.md) for full data.\n\n\u003e **Note:** `tempfile(RAMDisk)` results were measured with the temp directory on a RAM disk; `tempfile(SSD)` results use a physical SSD. Use `--ramdisk-dir` and `--ssd-dir` options to reproduce both variants in a single run.\n\n---\n\n## Support\n\nIf you find D-MemFS useful, consider [sponsoring the project](https://github.com/sponsors/nightmarewalker).\n\n---\n\n## License\n\nMIT License\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnightmarewalker%2Fd-memfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnightmarewalker%2Fd-memfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnightmarewalker%2Fd-memfs/lists"}