https://github.com/surbhitt/qaahl
a crawler that can scrap and visualize the path qrawled
https://github.com/surbhitt/qaahl
beautifulsoup4 pygame webcrawler webscraper
Last synced: 6 months ago
JSON representation
a crawler that can scrap and visualize the path qrawled
- Host: GitHub
- URL: https://github.com/surbhitt/qaahl
- Owner: surbhitt
- Created: 2023-08-10T23:19:46.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-12-09T13:13:02.000Z (almost 2 years ago)
- Last Synced: 2025-02-16T09:29:27.294Z (8 months ago)
- Topics: beautifulsoup4, pygame, webcrawler, webscraper
- Language: Python
- Homepage:
- Size: 146 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Objective
To develop an application capable of scrapping data from webpages and to provide visuals to assist the process through a GUI. Implementing a light weight multithreaded interactive application.
# Introduction
Qaahl serves as a minimalist webcrawler providing the basic functionality of a scrapper. A lightweight graphical interface built using pygames.
# Libraries utilised
- Pygame
- Requests
- Beautiful Soup (Bs4)
# Run
```console
pip install -r requirements.txt
python3 main.py $URL $flags
```
| `$URL` | with the link as the base url |
| -------- | ----------------------------------------------- |
| `$flags` | with flags e.g. -d 3 performs depth=3 traversal |