{"id":21035496,"url":"https://github.com/archiveteam/tinyarchive","last_synced_at":"2026-03-07T16:03:32.537Z","repository":{"id":5827811,"uuid":"7043354","full_name":"ArchiveTeam/tinyarchive","owner":"ArchiveTeam","description":"Software behind tracker.tinyarchive.org - Warning: Very hacky code","archived":false,"fork":false,"pushed_at":"2013-09-30T12:58:16.000Z","size":760,"stargazers_count":5,"open_issues_count":1,"forks_count":4,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-07-13T11:28:51.653Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ArchiveTeam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-12-06T21:33:39.000Z","updated_at":"2022-09-19T03:46:00.000Z","dependencies_parsed_at":"2022-09-05T11:31:57.886Z","dependency_job_id":null,"html_url":"https://github.com/ArchiveTeam/tinyarchive","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ArchiveTeam/tinyarchive","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchiveTeam%2Ftinyarchive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchiveTeam%2Ftinyarchive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchiveTeam%2Ftinyarchive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchiveTeam%2Ftinyarchive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ArchiveTeam","download_url":"https://codeload.github.com/ArchiveTeam/tinyarchive/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ArchiveTeam%2Ftinyarchive/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266161904,"owners_count":23885928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T13:15:06.068Z","updated_at":"2026-03-07T16:03:32.495Z","avatar_url":"https://github.com/ArchiveTeam.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Introduction\nThe _tinyarchive_ repository is a loose collection of scripts to help with\nbacking up URL shorteners. Most scripts are written in Python.\n\n# Concepts\n## Tinyarchive database\nThe very core of the whole thing. It consists of multiple [Berkely\nDB](https://en.wikipedia.org/wiki/Berkeley_DB) B-Tree databases that contain\nmappings from short url codes to long URLs. For each shortener there is one\ndatabase. For example, the database bitly.db might contain the following\nmappings:\n\n* a -\u003e http://apple.com\n* b -\u003e http://bit.ly/2lkCBm\n* c -\u003e http://www.timwilson.us\n\n## Tracker\nThe tracker is a completely separate application that hands out tasks to\n[tinyback](https://github.com/soult/tinyback) instances.\n\n## trim-old\nWhen tr.im shut down, part of it's database [was preserved](http://urlte.am/).\nIn 2013 tr.im was relaunched by [Matthew Kelly](https://twitter.com/mattkel),\nbut all the old shortlinks were lost. With a little magic, it was possible to\nrefill the new tr.im database with links from the old tr.im database. One such\nmagic trick is trim-old.tinyarchive.org: Since tr.im had trouble with some URLs\n(for whatever reason), instead of directly linking to the URL, it was created\nto redirect to trim-old.tinyarchive.org/$UUID and then is redirected to the\nreal URL from there.\n\n# Scripts\n## Database scripts\n### create\\_release.py\nCreates a new release from the database. By specifying the location of a\nprevious release, the create\\_release.py script can check which files have not\nchanged and avoid recompressing them, which would waste time and possibly\nchange their hashsum. The code\\_to\\_file.json file is used to map from a\nshortener name and code to a specific output file.\n### create\\_trim-old\\_db.py\nCreates the sqlite3 database used by the trim-old website.\n### import.py\nImports finished tasks from the tracker into the database.\n### import\\_tnyim.py\nOne-off script to import CSV dumps from the URL shortener at\n[tny.im](http://tny.im/).\n### release\\_import.py\nOpposite of create\\_release.py: Takes a release and imports it into the\ndatabase, using the code\\_to\\_file.json file to map from input file to URL\nshortener name.\n### stats.py\nOutputs a JSON structure containing a mapping from URL shortener name to number\nof shorturls in the database.\n## Tracker scripts\n### cleanup.py\nCalls the tracker's cleanup admin function, which removes finished tasks and\nresets assignments for tasks assigned over 30 minutes ago.\n### fetch\\_finished.py\nFetches a list of finished tasks from the tracker, then for each task first\ndownloads the payload and then tells the tracker to mark the task as deleted.\nFor each task, a JSON file with the task metadata and a corresponding txt.gz\nwith the payload is stored in the output directory.\n### redo.py\nTakes a JSON file containing task metadata and registers a new task with the\nsame parameters at the tracker.\n### task\\_create.py\nFile with some helper functions to create new tasks at the tracker.\n### twitter\\_spritzer\\_import.py\nUntested and unfinished tool to import the unrolled URLS from the Twitter\nspritzer provided by swebb.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farchiveteam%2Ftinyarchive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farchiveteam%2Ftinyarchive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farchiveteam%2Ftinyarchive/lists"}