https://github.com/akronix/scrap_wikia_page_views
Web scrapping of the Special:Insights/popularpages endpoint of every wiki in Wikia.
https://github.com/akronix/scrap_wikia_page_views
Last synced: 2 months ago
JSON representation
Web scrapping of the Special:Insights/popularpages endpoint of every wiki in Wikia.
- Host: GitHub
- URL: https://github.com/akronix/scrap_wikia_page_views
- Owner: Akronix
- License: gpl-3.0
- Created: 2018-11-08T11:05:42.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2023-02-10T22:44:51.000Z (over 3 years ago)
- Last Synced: 2025-01-19T06:40:41.232Z (over 1 year ago)
- Language: Python
- Size: 28.3 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# scrap_wikia_page_views
Web scrapping of the Special:Insights/popularpages endpoint of every wiki in Wikia.
## Install
Use the requirements.txt file: `pip install -r requirements.txt`
Important! This script is programmed in Python3 and tested in Python 3.5.2.
## Usage
There two ways to use the script:
One is providing the urls of the wikis to retrieve as arguments to the script:
`python -m extract_page_views http://gardening.wikia.com http://zh.halo.wikia.com`
Another one is providing the urls from a file which has one url per line:
`python -m extract_page_views --file wikia-test-index.txt`
The results will be stored in the file called `page_views-partk.csv`, if you wanted another output file, just change the variable `SUCCESS_FILENAME` in the source file.
You can ask for help to the script with `python extract_page_views.py --help`