https://github.com/joshwcomeau/script-search

Find code used on the world's top sites
https://github.com/joshwcomeau/script-search

Last synced: 17 days ago
JSON representation

Find code used on the world's top sites

• Host: GitHub
• URL: https://github.com/joshwcomeau/script-search
• Owner: joshwcomeau
• Created: 2017-01-01T04:03:40.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2017-01-01T04:04:53.000Z (about 8 years ago)
• Last Synced: 2024-12-09T14:14:43.400Z (24 days ago)
• Language: Python
• Size: 10.8 MB
• Stars: 3
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
### Site Fetch

TODO

### Getting Data

The current data source is a txt file of the top 1,000,000 sites, fetched from

https://statvoo.com/dl/top-1million-sites.csv.zip

This needs to be trimmed into a more reasonable number. Use the following command line script to trim it to a more reasonable number, as well as remove line prefixes:

    head -50000 data.txt | cut -d, -f2 | cut -d/ -f1 > topsites.txt