https://github.com/magnus0969/wikipedia-web-scraping
A web scraping project that extracts text, tables, and links from Wikipedia using BeautifulSoup, Requests, and Pandas. The scraped data is structured and saved in text formats for analysis.
https://github.com/magnus0969/wikipedia-web-scraping
data-science python web-scraping wikipedia-api wikipedia-scraper
Last synced: 3 months ago
JSON representation
A web scraping project that extracts text, tables, and links from Wikipedia using BeautifulSoup, Requests, and Pandas. The scraped data is structured and saved in text formats for analysis.
- Host: GitHub
- URL: https://github.com/magnus0969/wikipedia-web-scraping
- Owner: Magnus0969
- License: mit
- Created: 2025-03-06T15:48:59.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-06T16:07:24.000Z (over 1 year ago)
- Last Synced: 2025-03-06T17:21:39.023Z (over 1 year ago)
- Topics: data-science, python, web-scraping, wikipedia-api, wikipedia-scraper
- Language: Jupyter Notebook
- Homepage:
- Size: 15.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 🌐 Wikipedia Web Scraping
## 📌 Overview
This project scrapes data from **Wikipedia** using **BeautifulSoup**, and **Requests**. The script extracts useful information from Wikipedia pages, processes the data, and stores it in structured formats for analysis.
## 🛠 Technologies Used
- **Python** 🐍
- **BeautifulSoup** 🌐 (for parsing HTML)
- **Requests** 📡 (for fetching Wikipedia pages)
## 📈 Key Features
- Scrapes Wikipedia pages for text, tables, and links.
- Cleans and structures the extracted data.
- Saves data in **Text** formats.
- Implements request handling and rate-limiting to avoid blocking.
## 🚀 Getting Started
### 1️⃣ Clone the Repository
```bash
git clone https://github.com/Magnus0969/Wikipedia-Web-Scraping.git
cd Wikipedia-Web-Scraping
```
### 2️⃣ Install Dependencies
```bash
pip install -r requirements.txt
```
### 3️⃣ Run the Scraper
To scrape a Wikipedia page:
```bash
python scrape_wikipedia.py "https://en.wikipedia.org/wiki/Web_scraping"
```
## 📂 Output
- Extracted text is saved in `Satoshi Nakamoto.txt`.
## 📝 Example Extracted Data
**Sample Output from Wikipedia:**
```
Satoshi Nakamoto is the name used by the presumed pseudonymous person or persons who developed bitcoin, authored the bitcoin white paper, and created and deployed bitcoin's original reference implementation. As part of the implementation, Nakamoto also devised the first blockchain database. Nakamoto was active in the development of bitcoin until December 2010.
There has been widespread speculation about Nakamoto's true identity, with various people posited as the person or persons behind the name. Though Nakamoto's name is Japanese, and inscribed as a man living in Japan, most of the speculation has involved software and cryptography experts in the United States or Europe.
```
## 📬 Contact
For any questions or collaborations, feel free to reach out on **[LinkedIn](https://www.linkedin.com/in/kmagadi/)**.
---
Made with ❤️ by Karthik B Magadi