{"id":19055149,"url":"https://github.com/sangupta/wayback-machine-download","last_synced_at":"2025-04-24T04:10:19.234Z","repository":{"id":17464590,"uuid":"40252029","full_name":"sangupta/wayback-machine-download","owner":"sangupta","description":"A CLI to work with Archive.org Wayback Machine including downloading an entire snapshot","archived":false,"fork":false,"pushed_at":"2022-09-01T22:45:36.000Z","size":25,"stargazers_count":3,"open_issues_count":5,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-18T12:18:30.519Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sangupta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-05T15:05:14.000Z","updated_at":"2022-04-04T22:16:23.000Z","dependencies_parsed_at":"2022-08-30T10:10:55.398Z","dependency_job_id":null,"html_url":"https://github.com/sangupta/wayback-machine-download","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fwayback-machine-download","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fwayback-machine-download/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fwayback-machine-download/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fwayback-machine-download/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sangupta","download_url":"https://codeload.github.com/sangupta/wayback-machine-download/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250560053,"owners_count":21450172,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T23:42:16.521Z","updated_at":"2025-04-24T04:10:19.217Z","avatar_url":"https://github.com/sangupta.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"Wayback Machine Download\n========================\n\nA simple command-line interface to interact with \u003ca href=\"http://archive.org\"\u003earchive.org\u003c/a\u003e\n**WayBack Machine** including tool to download a snapshot of an entire site. This is particularly\nuseful for making up for lost backups and disaster recovery.\n\nDependencies\n------------\n\nThe tool needs to the following to run:\n\n* Oracle JDK 7 (should work with OpenJDK, but haven't tested)\n* Internet Connection\n\nUsage\n-----\n\n```\nusage: wmd \u003ccommand\u003e [\u003cargs\u003e]\n\nThe most commonly used wmd commands are:\n    configure   Generate a configuration for the site\n    download    Download the entire site\n    help        Display help information\n\nSee 'wmd help \u003ccommand\u003e' for more information on a specific command.\n```\n\nTo begin the download process, first generate a configuration answering the command prompts:\n\n```\n$ java -jar wayback-machine-download.jar configure\n\nConfiguration filename: somesite\nWayback URL to start with: http://web.archive.org/web/20140929053608/http://www.somesite.com/\nFolder path where to dump the site to: ~/wayback\nMax crawling depth: 5\n```\n\nThis will generate a configuration file called `somesite.wmd` in the folder to be used.\n\nTo start the download process, execute:\n\n```\n$ java -jar wayback-machine-download.jar download somesite.wmd\n```\n\nThis will crawl and dump the entire site in `~/wayback` folder.\n\nWhy this tool?\n--------------\n\nI lost code to a static website that I maintained for a friend of mine, and the server it was hosted\non crashed beyond recovery. I was left with two options:\n\n* either to recreate the site manually, which would have been months if not days\n* or, to try and download everything from archive.org\n\nAs I found there were two tools that could help me retrieve dump from wayback machine:\n\n* warrick - https://code.google.com/p/warrick/\n* http://waybackdownloader.com/\n\nI tried using `warrick` but for some reason it did not work. My bad - am sure, I would not have \nconfigured it properly. Again am not a `Perl` guy so as to debug it. The second option requested \nme money, and thus was ruled out.\n\nThis led to the birth of this tool: to help me recover the site from `wayback machine`.\n\nAm sure someone would need it too, one day!\n\nLicense\n-------\n\t\n```\nwmdownload - Download an entire website using Wayback Machine\nCopyright (c) 2015-2016, Sandeep Gupta\n\nhttp://sangupta.com/projects/wayback-machine-download\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsangupta%2Fwayback-machine-download","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsangupta%2Fwayback-machine-download","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsangupta%2Fwayback-machine-download/lists"}