https://github.com/KR1GOS/Web-Scaping

This is a Python web scraper that extracts information about AI-related tools from "https://topai.tools/browse." The scraper fetches data from the website, including the tool name, URL, description, pricing, and associated tags. The data is then exported to an Excel (.xlsx) file for further analysis.
https://github.com/KR1GOS/Web-Scaping

Last synced: about 2 months ago
JSON representation

This is a Python web scraper that extracts information about AI-related tools from "https://topai.tools/browse." The scraper fetches data from the website, including the tool name, URL, description, pricing, and associated tags. The data is then exported to an Excel (.xlsx) file for further analysis.

• Host: GitHub
• URL: https://github.com/KR1GOS/Web-Scaping
• Owner: KR1GOS
• Created: 2023-08-01T07:36:19.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2023-08-01T07:39:27.000Z (11 months ago)
• Last Synced: 2024-02-02T09:04:59.260Z (5 months ago)
• Language: Jupyter Notebook
• Size: 318 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Lists

• project-awesome - KR1GOS/Web-Scaping - This is a Python web scraper that extracts information about AI-related tools from "https://topai.tools/browse." The scraper fetches data from the website, including the tool name, URL, description, p (Jupyter Notebook)

README

        
# Web Scaping

 

This is a Python web scraper that extracts information about AI-related tools from "https://topai.tools/browse." The scraper fetches data from the website, including the tool name, URL, description, pricing, and associated tags. The data is then exported to both an Excel (.xlsx) file and a CSV file for further analysis.

Requirements

-Python 3.x

-pandas

-requests

-BeautifulSoup

-lxml

-selenium (only if using the selenium approach for infinite scrolling)