{"id":14235995,"url":"https://github.com/Snowtrack/SnowFS","last_synced_at":"2025-08-11T00:33:22.995Z","repository":{"id":38391085,"uuid":"332630181","full_name":"snowtrack/snowfs","owner":"snowtrack","description":"SnowFS - a fast, scalable version control file storage for graphic files :art:","archived":false,"fork":false,"pushed_at":"2023-10-25T13:06:31.000Z","size":824,"stargazers_count":1287,"open_issues_count":28,"forks_count":42,"subscribers_count":33,"default_branch":"main","last_synced_at":"2024-08-01T17:41:03.305Z","etag":null,"topics":["apfs","binary-manager","graphic-files","storage","version-control","version-control-system","version-manager","version-tracker"],"latest_commit_sha":null,"homepage":"https://www.snowtrack.io","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snowtrack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-01-25T04:28:08.000Z","updated_at":"2024-07-28T10:33:35.000Z","dependencies_parsed_at":"2024-01-07T01:18:21.360Z","dependency_job_id":"16fb3238-8a72-43e2-a0ce-f75a12f886c0","html_url":"https://github.com/snowtrack/snowfs","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowtrack%2Fsnowfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowtrack%2Fsnowfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowtrack%2Fsnowfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowtrack%2Fsnowfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snowtrack","download_url":"https://codeload.github.com/snowtrack/snowfs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":216214852,"owners_count":15992210,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apfs","binary-manager","graphic-files","storage","version-control","version-control-system","version-manager","version-tracker"],"created_at":"2024-08-20T21:02:38.169Z","updated_at":"2024-08-20T21:02:59.864Z","avatar_url":"https://github.com/snowtrack.png","language":"TypeScript","readme":"![snowfs][snowfs_banner]\n\n# SnowFS - a fast, scalable version control file storage for graphic files\n\n[![release](https://img.shields.io/badge/Download%20CLI%20Beta-0.9.2-orange)](https://github.com/Snowtrack/SnowFS/releases/tag/v0.9.2)\n[![Coverage Status](https://coveralls.io/repos/github/Snowtrack/SnowFS/badge.svg)](https://coveralls.io/github/Snowtrack/SnowFS)\n[![Build and Test](https://github.com/Snowtrack/SnowFS/workflows/Build%20and%20Test/badge.svg)](https://github.com/snowtrack/SnowFS/actions)\n[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\n[![DeepSource](https://deepsource.io/gh/Snowtrack/SnowFS.svg/?label=active+issues\u0026show_trend=true)](https://deepsource.io/gh/Snowtrack/SnowFS/?ref=repository-badge)\n[![Codacy Badge](https://app.codacy.com/project/badge/Grade/73acdcb360414eb299f5cf0221cab4d0)](https://www.codacy.com/gh/Snowtrack/SnowFS/dashboard?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=Snowtrack/SnowFS\u0026amp;utm_campaign=Badge_Grade)\n\n## Overview\n\nSnowFS is a high-performance command-line application and node library for *Windows*, *macOS* and *Linux* with a focus on binary file versioning. It is made for the graphics industry and was initially developed for [Snowtrack](https://www.snowtrack.io).\n\n**Disclaimer:** This project is in beta and actively developed.\n\n![terminal][terminal_preview]\n\n## Feature highlights\n\n- Supports Branches\n\n- Asynchronous File Hashing\n\n- Project open to file-content awareness (e.g: `*.psd`, `*.blend`, `*.c4d`, `..`)\n\n- Super-fast-detection of modifications in large binaries\n\n- Support for instant snapshots\\*\\*\n\n- Support for instant rollback\\*\\*\n\n- Support for files bigger \u003e4TB\n\n- Block-cloning and Copy-on-Write support for [APFS](https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system), [ReFS](https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview) and [Btrfs](https://en.wikipedia.org/wiki/Btrfs)\n\n- Support for removing single versions and/or binaries\n\n- Primarily I/O bound through [libuv](https://github.com/libuv/libuv)\n\n- Feature XYZ made by you!\n\n\\*\\* If the underlying filesystem supports it (e.g. APFS, ReFS and Btrfs)\n\n## Made with SnowFS\n\n- [Snowtrack](https://www.snowtrack.io)\n- [Snow for Blender](https://snowtrack.io/snow-for-blender.html)\n- [Snow for Cinema 4D](https://snowtrack.io/snow-for-cinema-4d.html)\n- [Snow for Unity](https://github.com/daniellanner/snow-for-unity-facade)\n\n## Why not Git/Git-LFS, libgit2, or SVN?\n\nFirst and foremost, the implementations of Git - namely [Git](https://git-scm.com/)/[Git-LFS](https://github.com/git-lfs/git-lfs)\nand [libgit2](https://libgit2.org/) are excellent implementations of version control systems.\nBut due to their focus on the software development lifecycle they are not suitable to version binaries or graphic files.\n`SnowFS` addresses the technical challenges for graphic files by its core design.\n\n### Git/Git-LFS:\n\n**Advantages:**\n\n  - Support on all major platforms\n  - Supported by hosting platforms like GitHub, GitLab, and BitBucket.\n  - Fast diff-operation for text-files\n\n**Disadvantages:**\n\n  - If not properly tracked, binaries become accidentally part of \"base\" history\n  - Removing older commits is cumbersome due to Gits commit hashing integrity\n  - Complicated *rewriting history* procedure\n  - Issues with binaries \u003e4GB on Windows as reported [here](https://github.com/git-lfs/git-lfs/issues/2434), [here](https://confluence.atlassian.com/bitbucketserverkb/files-larger-than-4-gb-are-not-correctly-handled-on-windows-935385144.html), and [here](https://stackoverflow.com/questions/49018053/how-large-does-a-large-file-have-to-be-to-benefit-from-git-lfs)\n  - Slow in binary modification detection\n\n### libgit2\n\n**Advantages:**\n\n  - Faster zipping, packing, and delta-compression than the reference implementation *Git*\n  - Supports [custom backends](https://git-scm.com/book/it/v2/Appendix-B%3A-Embedding-Git-in-your-Applications-Libgit2)\n\n**Disadvantages:**\n\n  - No native support for Git-LFS without custom backends\n  - Custom backends break compatibility with Git\n\n\n## TypeScript / C++ backport\n\n`SnowFS` is currently written in TypeScript. It is a great language to write powerful and performant\nI/O bound prototypes. We are aware of the demand for a C++ implementation, and due to\nour roots as C++ developers, we are very interested in a backport as well. It would make SnowFS easier\nto integrate into other projects. If you have comments, ideas or recommendations, please let us know.\n\n## Running benchmarks\n\nWe have also implemented a comparison benchmark between **SnowFS** vs. **git-lfs**.\nAfter executing the [build instructions](#build-instructions) for a development build, the benchmarks can be executed with the following command:\n\n```bash\n$ npm run benchmark\n```\n\nExample run on a Macbook Pro (2020) with an APFS formatted SSD to check-in, delete and restore a 4GB Photoshop File.\n\n```\n...\ngit lfs track *.psd\ngit add texture.psd: 20164ms\nsnow add texture.psd: 4596ms\ngit rm texture.psd: 575ms\nsnow rm texture.psd: 111ms\ngit checkout HEAD~1: 9739ms\nsnow checkout HEAD~1: 77ms \u003c-- Yeah!\n```\n\n## Examples\n\n### Code\n\nYou can find the best and up-to-date code examples in the `test/` directory. Given below these are simply **\"Hello World!\"** examples to get you started.\n\n```typescript\nimport * as fse from \"fs-extra\";\n\nimport { join } from \"path\";\nimport { Index } from \"./src\";\nimport { Repository } from \"./src/repository\";\n\nexport async function main() {\n  const repoPath = \"/path/to/a/non/existing/directory\";\n\n  // Create a new repository.\n  let repo: Repository = Repository.initExt(repoPath);\n\n  // Copy a file called 'texture.psd' to the working directory.\n  await fse.copyFile(\"/path/to/texture.psd\", join(repoPath, \"texture.psd\"));\n\n  // Retrieve the main index.\n  const index: Index = repo.ensureMainIndex();\n\n  // Mark 'texture.psd' as a new file in the index.\n  index.addFiles([\"texture.psd\"]);\n\n  // Write all files of the index to the object database.\n  await index.writeFiles();\n\n  // Create a commit for the index.\n  await repo.createCommit(index, \"This is my first commit\");\n}\n\nmain();\n```\n\n### Command line interface\n\nThe CLI of `SnowFS` offers some basic functionality and is subject to enhancements. See [Report #90](https://github.com/Snowtrack/SnowFS/issues/90)\n\n[![release](https://img.shields.io/badge/Download%20CLI%20Beta-0.9.2-orange)](https://github.com/Snowtrack/SnowFS/releases/tag/v0.9.2)\n\n```\n$ snow init foo\n$ cp /path/to/texture.psd foo/texture.psd\n$ cd foo\n$ snow add .\n$ snow commit -m \"My first texture\"\n$ snow log\n$ snow checkout -b MyNewBranch\n$ snow log\n```\n\n## Build Instructions\n\n1. To build `SnowFS` install [node.js](https://nodejs.org/en/) for your specific platform.\n\n    1.1. On Windows, Visual Studio is required. We recommend the [Visual Studio 2019 Community Edition](https://visualstudio.microsoft.com/downloads/). During the installation, please enable \u003ca href=\"https://docs.microsoft.com/en-us/cpp/build/media/vscpp-concierge-choose-workload.gif?view=msvc-160\" target=\"_blank\"\u003eDesktop development with C++\u003c/a\u003e.\n\n    1.2. On MacOS, [XCode](https://developer.apple.com/xcode/) is required.\n\n2. To build a **development build** execute:\n\n```bash\n$ git clone https://github.com/Snowtrack/snowfs.git\n$ cd snowfs\n$ npm install\n$ npm run test\n```\n\n3. To build a **production build including the CLI**, execute the commands above, and continue with the commands below:\n\n```bash\n$ npm run tsc\n$ npm run build\n$ cd dist\n```\n\n### How To Debug\n\nFor the development of SnowFS we recommend **VSCode**. The repository contains a [launch.json](.vscode/launch.json) file with pre-defined runner configurations. For more information, please visit [this](https://github.com/Snowtrack/SnowFS/pull/49#issue-579026705) pull-request.\n\n\n## Versioning\n\nStarting with version 1.0.0 `SnowFS` follows the [semantic versioning](http://semver.org/)\nscheme. The API change and backward compatibility rules are those indicated by\nSemVer.\n\n## Licensing\n\n`SnowFS` is licensed under the **GPL v3** license, please review the [LICENSE file](LICENSE).\nExcluded from the license are images, artworks, and logos. Please file a request by mail, if you have any questions.\n\n\n## Community\n\n- [Discord](https://discord.gg/RDKPuH8dkA)\n- [Support](https://github.com/Snowtrack/snowfs/labels/question)\n- [Email](mailto:support@snowtrack.io)\n\n\n### Other resources\n\nThe [tests and benchmarks](https://github.com/snowtrack/snowfs/tree/main/test) also serve as API specification and usage examples.\n\n## Supported Platforms\n\nCurrently, Windows, macOS, and Linux are supported. `SnowFS` works on plain filesystems like FAT, NTFS, HFS+ and has extended support for APFS, ReFS and Btrfs.\n\n## How can I contribute?\n\nSee the [guidelines for contributing][].\n\n[node.js]: http://nodejs.org/\n[terminal_preview]: https://github.com/snowtrack/snowfs/raw/main/img/terminal.gif\n[guidelines for contributing]: https://github.com/snowtrack/snowfs/blob/main/CONTRIBUTING.md\n[snowfs_banner]: https://github.com/snowtrack/snowfs/raw/main/img/banner.png\n","funding_links":[],"categories":["TypeScript","Encrypted Storage"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSnowtrack%2FSnowFS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSnowtrack%2FSnowFS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSnowtrack%2FSnowFS/lists"}