{"id":22165577,"url":"https://github.com/textileio/go-foldersync","last_synced_at":"2025-07-26T11:32:07.960Z","repository":{"id":57506760,"uuid":"223293213","full_name":"textileio/go-foldersync","owner":"textileio","description":"A sample-app of file syncing using Threads V2 and IPFS Lite","archived":false,"fork":false,"pushed_at":"2020-03-18T04:59:30.000Z","size":244,"stargazers_count":15,"open_issues_count":6,"forks_count":7,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-06-20T08:18:09.640Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/textileio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-22T00:50:38.000Z","updated_at":"2023-08-19T12:16:39.000Z","dependencies_parsed_at":"2022-08-29T20:01:23.021Z","dependency_job_id":null,"html_url":"https://github.com/textileio/go-foldersync","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/textileio%2Fgo-foldersync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/textileio%2Fgo-foldersync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/textileio%2Fgo-foldersync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/textileio%2Fgo-foldersync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/textileio","download_url":"https://codeload.github.com/textileio/go-foldersync/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227673973,"owners_count":17802303,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-02T05:15:36.373Z","updated_at":"2024-12-02T05:15:37.042Z","avatar_url":"https://github.com/textileio.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Folder-sync app\nThis sample-app allows peers to join a _shared folder_, where each user will \nown a folder named as their username. The local peer can drop new files to its \nfolder, where a filewatcher will automatically detect new files and sync to \nother joined peers.\n\n\n## Design\nThe main components of the applications are:\n- A `Store` using new Threads V2.\n- IPFS Lite to download files.\n\nThe `Store` will hold all files metadata, which is a JSON document derived from \nthe following structure:\n```\ntype userFolder struct {\n\tID    core.EntityID\n\tOwner string\n\tFiles []file\n}\n\ntype file struct {\n\tID               string\n\tFileRelativePath string\n\tCID              string\n\n\tIsDirectory bool\n\tFiles       []file\n}\n```\nIn summary, each folder is represented by a `userFolder` which is owned by a \nuser. This folder has a list of `file` which has file metadata such as:\n- A unique uuid\n- A relative path from the root of the shared folder: `userXXX/file.txt`\n- The CID of the file, which can be fetched *out-of-band* with IPFS Lite.\n\nThe original intention is to allow arbitrary nested folders and files, but \ncurrently it has only one level support for files. It's on the plans to enable \nfolder discovery to allow nested trees.\n\nCurrently, when a new file is discovered by a peer, it will try to fetch it \nusing an `ipfslite` client, which will leveraged connected peers. By default, \nthe `ipfslite` peer doesn't boostrap a DHT, but it can be enabled in a single \nline. Future versions will make this optional through flags. In the same vein, \nwe can provide other sources from which fetch files such as the Filecoin \nnetwork.\n\n## Usage\nWhen a peer wants to bootstrap a new shared folder, to invite others, it has to \nrun: `go run main.go util.go client.go -name bob`. This will create a new \n`Store` with an underlying new `Thread`, and will print an invitation link that \nother peers can use to join the shared folder.\n\nJoining peers will run: \n`go run main.go util.go client.go -name alice -inviteLink xxxxxxx` to join \nthe shared folder. The invite link has format: \n`\u003cthread-addr\u003e?\u003cfollow-key\u003e\u0026\u003cread-key\u003e`.\n\nCurrent flags are:\n- `name`: to setup the peer name, which will be his sharedfolder name\n- `repo`: path to the repo that will hold data of `Store`\n- `sharedFolder`: path to where will be the shared folder containing all peer \nfolders with their files\n- `inviteLink`: if provided is used to join an existing _sharedFolder_\n\nFlags have reasonable defaults.\n\n## Tests\nThis app *now* has a pretty heavy test setup, where tunning a parameter \nsimulates:\n- A number N of peers joining a shared folder\n- How many random files will each peer generate\n- How big each random file will be\n- How many core peers will be used to join the _sharedFolder_. This means, \nX peers will be used as invitation links by others in a round-robin fashion \npossibly to avoid DoSing all joining the same peer. May also be used to \ntest how this impact syncing perf.\n- How many seconds provide to the peers to sync before asserting convergence.\n\nAfter the sync timeout triggers, the test will assert if all `Store` of peers \nare the same, and if all the files were fetched (that to say, each peer shared \nfolder contains all peers folders and their files).\n\nTake into consideration that tests configuration should be tuned to avoid \ntimeouts and high overload, since all peers are running on the same host.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftextileio%2Fgo-foldersync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftextileio%2Fgo-foldersync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftextileio%2Fgo-foldersync/lists"}