{"id":21675051,"url":"https://github.com/thaliaarchi/urlhero","last_synced_at":"2025-08-03T00:35:28.692Z","repository":{"id":57564830,"uuid":"318376217","full_name":"thaliaarchi/urlhero","owner":"thaliaarchi","description":"Link resolver for current and defunct URL shorteners","archived":false,"fork":false,"pushed_at":"2022-06-23T21:11:07.000Z","size":146,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-25T10:08:31.578Z","etag":null,"topics":["archiveteam","internet-archive","url-shortener","url-unshortener","urlteam"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thaliaarchi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-04T02:11:49.000Z","updated_at":"2024-02-27T17:37:37.000Z","dependencies_parsed_at":"2022-08-23T12:10:31.171Z","dependency_job_id":null,"html_url":"https://github.com/thaliaarchi/urlhero","commit_stats":null,"previous_names":["andrewarchi/urlhero","andrewarchi/urlteam"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thaliaarchi%2Furlhero","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thaliaarchi%2Furlhero/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thaliaarchi%2Furlhero/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thaliaarchi%2Furlhero/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thaliaarchi","download_url":"https://codeload.github.com/thaliaarchi/urlhero/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244583159,"owners_count":20476233,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archiveteam","internet-archive","url-shortener","url-unshortener","urlteam"],"created_at":"2024-11-25T13:50:48.061Z","updated_at":"2025-03-20T09:18:43.484Z","avatar_url":"https://github.com/thaliaarchi.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# URLHero\n\nURLHero is a link resolver for current and defunct URL shorteners. It\nuses link mappings from [URLTeam](https://wiki.archiveteam.org/index.php/URLTeam)\narchives, dumps provided by shortener operators, and links captured by\nthe Internet Archive.\n\n## Planned features\n\n### Downloader\n\n- Automatically download and process daily URLTeam releases.\n- Hopefully gain access to [301Works dumps](301works.md).\n- Switch to a torrent client that can scale to handle 1500 webseed\n  items. [anacrolix/torrent](https://github.com/anacrolix/torrent) has\n  [less mature webseed support](https://github.com/anacrolix/torrent/issues/465)\n  and is relatively slow. [Transmission](https://transmissionbt.com/)\n  was unable to handle all torrents, in simple tests.\n- Support Internet Archive API authentication. For example,\n  [URLTeamTorrentRelease2013July](https://archive.org/download/URLTeamTorrentRelease2013July)\n  can only be downloaded when signed in.\n\n### Link resolver\n\n- Create link resolving website and API.\n- Create Web Extension that redirects dead short links using URLHero.\n- Proxy unknown shortener requests and contribute back to URLTeam\n  dataset.\n- Possibly fork [unshort.link](https://github.com/simonfrey/unshort.link).\n\n### Parsing\n\n- Process URLTeam first-generation TinyBack releases.\n- Write custom CSV parser for qr-cx datasets to handle unescaped quotes.\n- Full BEACON format spec compliance.\n\n### Database\n\n- Find a relational or key-value database with efficient compression.\n\n## Contributing\n\nThere are many ways to contribute:\n\n- File an issue or PR to submit a feature or bug report.\n- Send link mappings for a URL shortener that you operate or have\n  archived.\n- Join URLTeam and help us archive at-risk shorteners by running the\n  terroroftinytown project [in Docker](https://wiki.archiveteam.org/index.php/Running_Archive_Team_Projects_with_Docker#Basic_usage)\n  or via the [Archive Team Warrior](https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior#Installing_and_running_with_Docker).\n\nIf you want to get in touch, join the\n[#urlteam](https://webirc.hackint.org/#irc://irc.hackint.org/#urlteam)\nchannel on hackint or email me.\n\n## License\n\nThis project is made available under the\n[Mozilla Public License, v. 2.0](https://www.mozilla.org/en-US/MPL/2.0/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthaliaarchi%2Furlhero","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthaliaarchi%2Furlhero","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthaliaarchi%2Furlhero/lists"}