https://github.com/akurtz87/domain-scraper
https://github.com/akurtz87/domain-scraper
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/akurtz87/domain-scraper
- Owner: AKurtz87
- Created: 2023-03-10T22:16:02.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-03-10T22:49:51.000Z (over 2 years ago)
- Last Synced: 2025-02-13T19:38:59.594Z (4 months ago)
- Language: JavaScript
- Size: 4.88 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Domain Scraper 🔎
This is a JavaScript script that scrapes a web page for domain names, writes the domain names to a text file, and triggers a download of the file.
Here's a breakdown of what the script does:
1. The script creates an empty array called domains.
2. The script loops from i = 3 to i = 1000.
3. For each iteration of the loop, the script builds a query string that includes the current value of i.
4. The script uses document.querySelector to select the element using the current query string.
5. If no element was found, **to not break the loop i use console.log("#") but is possible ti use also continue.**
6. When an element was found, the script gets the text content of the element, splits it by spaces, and pushes the first part to the domains array.
7. Once the loop has completed (because it reached i = 1000), the domains array is converted to a string using the join method and a new Blob object is created from the string. I used 1000 beacuse i know that domains elements were less than 1000 in the webpage.
8. The script creates a new URL object from the Blob object and creates a new element with the appropriate href and download attributes to trigger the download.
9. The script appends the element to the DOM, clicks it to trigger the download, and removes it from the DOM.
**This script is specific to the web page being scraped (https://zonefiles.io/detailed-domain-lists/), and assumes that the domain names are located in a specific table on the page. You may need to modify the script if you are scraping a different web page or if the page layout changes.**
## How to use the script 📝
To use the JavaScript script provided, simply navigate to the webpage that you want to scrape for domain names in your web browser. Once on the webpage, open the browser console by pressing **Ctrl + Shift + J** on Windows or **Cmd + Option + J** on Mac. Then, copy and paste the script into the console and press Enter to execute it.
The script will scrape the webpage for domain names, write them to a text file, and trigger a download of the file.
The resulting text file can then be opened and used as needed.
