https://github.com/iyanuashiri/wikipedia-reference-scraper

Wikipedia API wrapper for references
https://github.com/iyanuashiri/wikipedia-reference-scraper

api api-wrapper beautifulsoup python scraper wikipedia

Last synced: 17 days ago
JSON representation

Wikipedia API wrapper for references

• Host: GitHub
• URL: https://github.com/iyanuashiri/wikipedia-reference-scraper
• Owner: iyanuashiri
• Created: 2018-02-23T02:38:05.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2022-12-09T07:39:46.000Z (almost 2 years ago)
• Last Synced: 2024-10-12T07:28:24.725Z (about 1 month ago)
• Topics: api, api-wrapper, beautifulsoup, python, scraper, wikipedia
• Language: Python
• Size: 10.7 KB
• Stars: 33
• Watchers: 3
• Forks: 6
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# wikipedia-reference-scraper

Wikipedia API wrapper for references

# Motivation

I just graduated from Physiology department, University of Ibadan. I started typing my final year project some days before submission

deadline. I made use of Wikipedia for my literature review because each page is supported with enough references. The next task was

to copy and paste the references. This was a lot of work considering the fact that some pages has over 200 references and I wasn't

working with just one page. I decided to make use of Wikipedia API wrappers but all the ones I checked didn't do what I needed. So

I decided to write a simple script that scraped Wikipedia page.

# Usage

## Python Interactive Shell

```

>>> from wikipedia_reference_scraper import WikipediaReferenceScraper as wrs

>>> wrs().write_to_document('https://en.wikipedia.org/wiki/Blood_pressure#References', 'filename.docx')

```

## Through the Command Line (with python-fire)

```

$ python wikipedia_reference_scraper.py write_to_document https://en.wikipedia.org/wiki/Blood_pressure#References filename.docx

```

It pulls the references from a Wikipedia page and saves the references in a file.

# Tools I Used

Requests

BeautifulSoup

python-fire