https://github.com/hotaro26/wiki_web
wiki content scrapper built with python
https://github.com/hotaro26/wiki_web
python scrapper web
Last synced: 2 months ago
JSON representation
wiki content scrapper built with python
- Host: GitHub
- URL: https://github.com/hotaro26/wiki_web
- Owner: Hotaro26
- Created: 2026-04-14T02:28:35.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2026-04-14T02:39:02.000Z (3 months ago)
- Last Synced: 2026-04-14T04:21:21.487Z (3 months ago)
- Topics: python, scrapper, web
- Language: Python
- Homepage:
- Size: 14.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Wiki Scraper & Web App
A versatile Python project that offers multiple ways to fetch and save Wikipedia content, ranging from command-line scrapers to a full-featured web application.
## 🚀 Features
- **Standard Scraping:** Uses `BeautifulSoup4` and `requests` to parse Wikipedia pages directly.
- **API Integration:** Uses the official Wikipedia API for cleaner and faster data retrieval.
- **File Management:** Option to save fetched articles as `.txt` files for offline reading.
- **Visual Enhancements:** A dedicated CLI version with colored terminal output for better readability.
- **Web Interface:** A Flask-based web application to search and scrape Wikipedia from your browser.
## 📁 Project Structure
```text
wiki_web/
├── app.py # Flask web application
├── main.py # BeautifulSoup4-based scraper
├── wiki_api.py # Basic Wikipedia API scraper
├── wiki_api_with_save.py # API scraper with .txt save functionality
├── wiki_api_with_save-colours.py # API scraper with save functionality & colored UI
├── requirements.txt # Project dependencies
├── templates/ # HTML files for the web app
└── venv/ # Virtual environment
```
## 🛠️ Installation
1. **Clone the repository:**
```bash
git clone
cd wiki_web
```
2. **Create and activate a virtual environment (optional but recommended):**
```bash
python -m venv venv
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate
```
3. **Install dependencies:**
```bash
pip install -r requirements.txt
```
## 💻 Usage
### 1. Web Application
To run the web interface on `localhost`:
```bash
python app.py
```
Open your browser and navigate to `http://127.0.0.1:5000`.
### 2. Command Line Tools
You can run any of the specific scrapers depending on your needs:
- **Basic Scraper:** `python main.py`
- **API Scraper:** `python wiki_api.py`
- **Save to Text:** `python wiki_api_with_save.py`
- **Colored UI Scraper:** `python wiki_api_with_save-colours.py`
## 📦 Dependencies
This project uses several Python libraries:
- `requests`: To handle HTTP requests.
- `BeautifulSoup4`: For parsing HTML content.
- `Wikipedia-API`: For interacting with the official Wiki API.
- `Flask`: To power the web application.
- `colorama`: (Assuming) For the colored terminal output.
---
