Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/lazza/bitlyescape

Easily back up your Bitly data and prepare it for seamless import into YOURLS
https://github.com/lazza/bitlyescape

backup bitly bitly-api yourls

Last synced: 6 months ago
JSON representation

Easily back up your Bitly data and prepare it for seamless import into YOURLS

Awesome Lists containing this project

README 

          
# Bitly Escape (for YOURLS migration)



This is a collection of Python scripts aimed at helping you escape from Bitly.



Starting from February 2025, Bitly began **inserting unwanted ads** on free

accounts, by [adding an interstitial page][bitly_ads] to links shared from free

accounts.



For those using a custom domain, this is especially bad because users may assume

that ads are run by the domain's owner rather than Bitly. Ads are generally a

negative experience for users, because they are annoying and they may come with

malware or other unwanted content.



[YOURLS][yourls] is a very popular self-hosted option for those wanting to run

their own free URL shortening service. Unfortunately, there is no direct way to

import Bitly data into YOURLS, however, its plugin-based architecture allows

users to add additional features, including data import.



This repository includes software mainly aimed at exporting data from Bitly,

while converting it in a format that can be imported via the

[YOURLS-Import-Export][import_export] plugin.



## How to use



First of all, make sure you are running an up-to-date version of YOURLS

configured to run on your custom domain name.



**Note:** in order to ensure uninterrupted service, do not migrate your domain's

DNS away from Bitly yet. Update the DNS record on your PC by editing the `hosts`

file for now.



### Get group ID and token



Log in to Bitly and look at the URL: you will see your "group ID". Go to the

settings page and [generate an access token][access_token].



### Set up the scripts



If you wish to, activate a _virtualenv_. Then install the requirements:



    pip install -r requirements.txt



Copy the `config.example.py` file to `config.py` and edit it to include your

Bitly access token and your group ID.



### Export links from Bitly



You are now ready to grab all the links from Bitly and store the data safely in

a SQLite database.



Run the following command:



    python bitly_escape.py



The script will download links in batches of 50 at a time. If you get rate

limited due to excessive API usage, it will sleep for 1 minute and then try

again until all the links have been collected.



It is very important that the computer remains connected to the Internet during

the process.



You will get a SQLite file called `bitly_links.db` as a result.



### Optional: export engagement data



If you wish to retain the overall number of clicks each link has received,

according to Bitly, you need to execute the second script.



Simply run this command:



    python populate_clicks.py



Wait until it's done and the SQLite file will be updated accordingly.



### Generate the XML import file



With all links safely backed up in the SQLite database, you can generate the

**XML file needed for importing the data into YOURLS.**



Run the following command:



    python generate_xml.py



The script will output the total number of links and generate a new file called

`bitly_links.xml`.



## Importing data into YOURLS



**Note:** before importing data, it is suggested to check if your Bitly links

use aliases compatible with your character settings. If needed, several YOURLS

plugins allow you to change the allowed character set.



The XML file generated by the last script can be imported using the

[YOURLS-Import-Export][import_export] plugin. Please refer to the documentation

of said plugin for additional information.



Very large collections of links may need to be broken down into several parts.

Please read the following remarks for more details.



Once you are done, you can point the DNS record of your domain to your new

YOURLS installation and install a valid TLS certificate. Links should work as

expected!



## Additional remarks



It is worth noting that the script gathers all links from your account, even if

they are _not_ using a custom domain name. You might want to filter the SQLite

database manually, depending on your needs.



For custom-assigned Bitly aliases (e.g. `MyFancyName`) Bitly creates a random

link alias as well. The script does not create an entry for this, only the

custom alias is stored.



If you have a large number of links, the XML import process may time out and the

web page could show a Server Error. In my testing, the process still continued

to import all data in the background, but this may depend on server settings.



This was tested with roughly 2200 links.



You can see the total number of links from the YOURLS interface. If you are in

doubt, consider splitting the SQLite database into different files and

generating multiple XML files for separate imports.



There will be a delay between the time you grab the number of clicks on links

and when you are done with the migration. If your links are very active, they

may get additional clicks recorded on Bitly.



You can export the engagement data again after your new YOURLS installation has

gone live, but this topic is beyond the scope of this guide.



Finally, keep in mind that YOURLS does not comply with the GDPR using standard

settings, as it keeps full logs of IP addresses, user-agent strings and

referers. You may either disable logging, or consider using anonymization

plugins that may redact some of the data.



Verify the database tables and ensure that the data is properly sanitized.



## Acknowledgments



The development of this project has been significantly accelerated with the

assistance of [Le Chat][le_chat], an AI tool used to draft large portions of the

code.



  [bitly_ads]: https://support.bitly.com/hc/en-us/articles/32874287800333-Why-are-there-ads-on-my-links

  [yourls]: https://yourls.org

  [import_export]: https://github.com/GautamGupta/YOURLS-Import-Export

  [access_token]: https://app.bitly.com/settings/api/

  [le_chat]: https://chat.mistral.ai/