{"id":13498996,"url":"https://github.com/kost/dvcs-ripper","last_synced_at":"2025-05-15T11:08:21.037Z","repository":{"id":5173315,"uuid":"6347029","full_name":"kost/dvcs-ripper","owner":"kost","description":"Rip web accessible (distributed) version control systems: SVN/GIT/HG...","archived":false,"fork":false,"pushed_at":"2024-07-19T12:49:07.000Z","size":48,"stargazers_count":1723,"open_issues_count":11,"forks_count":314,"subscribers_count":52,"default_branch":"master","last_synced_at":"2025-04-11T20:00:01.556Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"openfl/openfl","license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kost.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-10-23T04:55:18.000Z","updated_at":"2025-04-09T20:06:30.000Z","dependencies_parsed_at":"2024-10-15T06:20:24.992Z","dependency_job_id":null,"html_url":"https://github.com/kost/dvcs-ripper","commit_stats":{"total_commits":46,"total_committers":10,"mean_commits":4.6,"dds":"0.26086956521739135","last_synced_commit":"2c1bbc669549e64c096107f0586d748e43156a1e"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kost%2Fdvcs-ripper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kost%2Fdvcs-ripper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kost%2Fdvcs-ripper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kost%2Fdvcs-ripper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kost","download_url":"https://codeload.github.com/kost/dvcs-ripper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254328384,"owners_count":22052632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T22:00:25.036Z","updated_at":"2025-05-15T11:08:21.015Z","avatar_url":"https://github.com/kost.png","language":"Perl","readme":" [![Rawsec's CyberSecurity Inventory](https://inventory.rawsec.ml/img/badges/Rawsec-inventoried-FF5050_flat.svg)](https://inventory.rawsec.ml/tools.html#dvcs-ripper)\n [![GitHub stars](https://img.shields.io/github/stars/kost/dvcs-ripper.svg)](https://github.com/kost/dvcs-ripper/stargazers)\n [![GitHub license](https://img.shields.io/github/license/kost/dvcs-ripper.svg)](https://github.com/kost/dvcs-ripper/blob/master/LICENSE)\n\ndvcs-ripper\n===========\n\nRip web accessible (distributed) version control systems: SVN, GIT, Mercurial/hg, bzr, ...\n\nIt can rip repositories even when directory browsing is turned off. \n\nMake sure to position yourself in empty directory where you want repositories to be downloaded/cloned.\n\n## Requirements\n\n- Perl\n- Perl modules: \n  - required: LWP, IO::Socket::SSL \n  - for newer SVN: DBD::SQlite and DBI\n  - for faster GIT: Parallel::ForkManager, Redis and Algorithm::Combinatorics\n- (D)VCS client of what you want to rip (cvs, svn, git, hg, bzr, ...)\n\n### Requirements on Debian/Ubuntu\n\nYou can easily install perl requirements:\n\n`sudo apt-get install perl libio-socket-ssl-perl libdbd-sqlite3-perl libclass-dbi-perl libio-all-lwp-perl`\n\nOptional requirements (faster git rip):\n`sudo apt-get install libparallel-forkmanager-perl libredis-perl libalgorithm-combinatorics-perl`\n\nAnd if you need all clients supported:\n\n`sudo apt-get install cvs subversion git bzr mercurial`\n\n## Docker\n\nIn case you just want docker version, it is here:\n\nhttps://github.com/kost/docker-webscan/tree/master/alpine-dvcs-ripper\n\nJust say something like:\n\n`docker run --rm -it -v /path/to/host/work:/work:rw k0st/alpine-dvcs-ripper rip-git.pl -v -u http://www.example.org/.git`\n\n\nGIT\n===========\nExample run (for git):\n\n`rip-git.pl -v -u http://www.example.com/.git/`\n\nIt will automatically do `git checkout -f`\n\nor if you would like to ignore SSL certification verification (with -s):\n\n`rip-git.pl -s -v -u http://www.example.com/.git/`\n\nMercurial/HG\n===========\nExample run (for hg):\n\n`rip-hg.pl -v -u http://www.example.com/.hg/`\n\nIt will automatically do `hg revert \u003cfile\u003e`\n\nor if you would like to ignore SSL certification verification (with -s):\n\n`rip-hg.pl -s -v -u http://www.example.com/.hg/`\n\nBazaar/bzr\n===========\nExample run (for bzr):\n\n`rip-bzr.pl -v -u http://www.example.com/.bzr/`\n\nIt will automatically do `bzr revert`\n\nor if you would like to ignore SSL certification verification (with -s):\n\n`rip-bzr.pl -s -v -u http://www.example.com/.bzr/`\n\n\nSVN\n===========\nIt supports OLDER and NEWER version of svn client formats. Older is with .svn files in every directory, while\nnewer version have single .svn directory and wc.db in .svn directory. It will automatically detect which \nformat is used on the target.\n\nExample run (for SVN):\n\n`rip-svn.pl -v -u http://www.example.com/.svn/`\n\nIt will automatically do `svn revert -R .`\n\nCVS\n===========\nExample run (for CVS):\n\n`rip-cvs.pl -v -u http://www.example.com/CVS/`\n\nThis will not rip CVS, but it will display useful info.\n\n## Advance usage examples\n\nSome examples how it can be used\n\n### Output handling\n\nDownload git tree to specific output dir:\n\n`rip-git.pl -o /my/previously/made/dir -v -u http://www.example.com/.git/`\n\nDownload git tree to specific output dir (creating dir `http__www.example.com_.git_` for url):\n\n`rip-git.pl -m -o /dir -v -u http://www.example.com/.git/`\n\n### Redis usage with docker\n\nCreate Redis docker container:\n\n`docker run --rm --name myredis -it -v /my/host/dir/data:/data:rw k0st/alpine-redis`\n\nIn another terminal, just link redis container and say something like this:\n\n`docker run --rm --link=myredis:redis -it -v /path/to/host/work:/work:rw k0st/alpine-dvcs-ripper rip-git.pl -e docker -v -u http://www.example.org/.git -m -o /work`\n\n### Using redis for resuming work of ripping\n\nCreate Redis docker container:\n\n`docker run --name redisdvcs -it -v /my/host/dir/data:/data:rw k0st/alpine-redis`\n\nIn another terminal, just link redis container and say something like this:\n\n`docker run --link=redisdvcs:redis -it -v /path/to/host/work:/work:rw k0st/alpine-dvcs-ripper rip-git.pl -n -e docker -v -u http://www.example.org/.git -m -o /work`\n\n### Abusing redis for massive parallel tasks\n\nCreate global NFS and mount /work on each client. Create global Redis docker container:\n\n`docker run --name redisdvcs -it -v /my/host/dir/data:/data:rw k0st/alpine-redis`\n\nIn another terminal, just link redis container and say something like this on 1st client\n\n`docker run -it -v /path/to/host/work:/work:rw k0st/alpine-dvcs-ripper rip-git.pl -n -e global.docker.ip -v -u http://www.example.org/.git -t 10 -c -m -o /work`\n\nIn another terminal, just link redis container and say something like this on 2nd client:\n\n`docker run -it -v /path/to/host/work:/work:rw k0st/alpine-dvcs-ripper rip-git.pl -n -e global.docker.ip -v -u http://www.example.org/.git -t 10 -c -m -o /work`\n\nand so on...\n\nYou need to perform `git checkout -f` yourself on the end - of course!\n\n## Future\n\nFeel free to implement something and send pull request. Feel free to suggest any feature. Lot of features\nactually were implemented by request\n\n### ToDo\n- [ ] Recognize 404 pages which return 200 in SVN/CVS\n- [ ] Try to repeat each trick after previous trick was successful\n- [ ] Progress bars\n\n### Done\n- [x] Support for brute forcing pack names \n- [x] Intelligent guessing of packed refs\n- [x] Support for objects/info/packs from https://www.kernel.org/pub/software/scm/git/docs/gitrepository-layout.html\n- [x] Recognize 404 pages which return 200 \n- [x] Introduce ignore SSL/TLS verification in SVN/CVS\n- [x] Bzr support\n\n","funding_links":[],"categories":["Tools","Forensics","Miscellaneous","Web Exploitation","Awesome Penetration Testing (\"https://github.com/Muhammd/Awesome-Pentest\")","Perl","Web","Tools:"],"sub_categories":["Leaking","Git","Web-accessible source code ripping tools","Tools","Web Exploitation","Penetration Testing Report Templates","Web Accessible Source Code Ripping Tools"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkost%2Fdvcs-ripper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkost%2Fdvcs-ripper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkost%2Fdvcs-ripper/lists"}