{"id":19369125,"url":"https://github.com/joystream/swg-files-comparer","last_synced_at":"2026-05-07T16:41:11.181Z","repository":{"id":214333114,"uuid":"734004350","full_name":"Joystream/swg-files-comparer","owner":"Joystream","description":"this tool allows SWG workers to check if the local files are identical to the files listed under their bucket in storage node","archived":false,"fork":false,"pushed_at":"2024-07-11T19:18:40.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-06T22:52:56.937Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Joystream.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-20T16:24:48.000Z","updated_at":"2024-07-11T19:18:43.000Z","dependencies_parsed_at":"2024-03-29T14:26:34.929Z","dependency_job_id":"b7e4635c-dcff-4e60-8dd7-693edff0841e","html_url":"https://github.com/Joystream/swg-files-comparer","commit_stats":null,"previous_names":["joystream/swg-files-comparer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Joystream%2Fswg-files-comparer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Joystream%2Fswg-files-comparer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Joystream%2Fswg-files-comparer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Joystream%2Fswg-files-comparer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Joystream","download_url":"https://codeload.github.com/Joystream/swg-files-comparer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240491826,"owners_count":19809977,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T08:09:47.206Z","updated_at":"2026-05-07T16:41:06.145Z","avatar_url":"https://github.com/Joystream.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SWG-files-comparer\n\nThis tool allows to check if some of the files for collosus were lost and if they can be found on other nodes.\n\n\u003c!-- toc --\u003e\n\n- [Installation](#installation)\n- [Usage](#usage)\n\u003c!-- tocstop --\u003e\n\n# Installation\n\n```shell\n# Install dependencies\n\nyarn install\n\n# Build the project\n\nyarn build\n\n```\n\n# Usage\n\nRun the program using yarn start. You will see the command seleciton menu.\n\n1. Index local files\n\n```sh-session\n localFiles\n @param \u003cpathToDirectory\u003e\n```\n\nThis command will index the files in the directory and create a file `localFiles.json` in the root directory of the project. It will be used\nfor comparison later.\n\n2. Index objects from bucket\n\n```sh-session\n bucketObjects\n @param \u003cbucketId\u003e\n @param \u003coptional\u003e \u003cbagId\u003e\n```\n\nThis will get all the bags from the bucket and index all the objects in them. It will create a file `remote.json` in the root directory of the project.\n\nWith provided `bagId` it will get all the objects from the bag and index them. It will create a file `remote-${bagId}.json` in the root directory of the project.\n\n3. Compare the files\n\n```sh-session\ndiff\n@param \u003coptional\u003e\u003cbagId\u003e\n```\n\nThis will compare your bags from `local.json` with the bags from `remote.json` and will output the results in `diff.json` in the root directory of the project and in the console.\n\nWith provided `bagId` it will compare your bags from `local.json` with the bags from `remote-${bagId}.json` and will output the results in `diff-${bagId}.json` in the root directory of the project and in the console.\n\n4. Check missing files\n\n```sh-session\ncheckMissing\n@param \u003cpathToDiff\u003e\n@param \u003coptional\u003e \u003cignoreSp\u003e\n```\n\nThis will check the `diff.json` file and will show if some of the other SP has the missing files.\nYou can specify the custom path to diff file and include the SP that you want to ignore in the search.\ne.g\n\n5. Download missing files\n\n```sh-session\ndownloadMissing\n```\n\nThis should be used after the checkMissing command. It will download the missing files from the remote node containing missing files.\n\n```sh-session\n$ yarn start restoreMissing ./jsons/diff.json 1,2,5\n```\n\n6. Manual HEAD request\n\n```sh-session\n head \u003cproviderUrl\u003e\n @param \u003cproviderUrl\u003e\n\n```\n\nThis will execute a HEAD request to the specified url and will indicate a presence of the file.\n\n7. Check Remote node\n\n   ```sh-session\n   remotenode\n   @param \u003cproviderUrl\u003e\n   ```\n\nThis will check the remote node and will return the same output as the localFiles command.\n\n8. Check single operator\n\n   ```sh-session\n   checknode\n   @param \u003cproviderUrl\u003e\n   ```\n\nThis is a combination of remoteNode, bucketObjects and diff commands for a single operator. It will check the operator, check all files for its bags in QN and will generate a diff.\n\n8. Check all operators\n\n   ```sh-session\n   checkalloperators\n   ```\n\nThis is a combination of remoteNode, bucketObjects and diff commands. It will check all the operators, check all files in QN and will generate a diff.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoystream%2Fswg-files-comparer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoystream%2Fswg-files-comparer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoystream%2Fswg-files-comparer/lists"}