Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/siznax/wptools
Wikipedia tools (for Humans): easily extract data from Wikipedia, Wikidata, and other MediaWikis
https://github.com/siznax/wptools
api-client commons data-science glam linked-open-data mediawiki mediawiki-api open-data python restbase wikidata wikimedia-commons wikipedia wikipedia-api
Last synced: 22 days ago
JSON representation
Wikipedia tools (for Humans): easily extract data from Wikipedia, Wikidata, and other MediaWikis
- Host: GitHub
- URL: https://github.com/siznax/wptools
- Owner: siznax
- License: mit
- Created: 2012-02-06T04:10:27.000Z (over 12 years ago)
- Default Branch: master
- Last Pushed: 2023-08-05T19:06:41.000Z (over 1 year ago)
- Last Synced: 2024-04-14T11:14:59.454Z (7 months ago)
- Topics: api-client, commons, data-science, glam, linked-open-data, mediawiki, mediawiki-api, open-data, python, restbase, wikidata, wikimedia-commons, wikipedia, wikipedia-api
- Language: Python
- Homepage:
- Size: 1.14 MB
- Stars: 561
- Watchers: 23
- Forks: 79
- Open Issues: 38
-
Metadata Files:
- Readme: README.rst
- Changelog: HISTORY.rst
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
Wikipedia tools (for Humans)
============================.. image:: https://img.shields.io/pypi/v/wptools.svg
:target: https://pypi.python.org/pypi/wptools/.. image:: https://travis-ci.org/siznax/wptools.svg?branch=master
:target: https://travis-ci.org/siznax/wptools.. image:: https://coveralls.io/repos/github/siznax/wptools/badge.svg?branch=master
:target: https://coveralls.io/github/siznax/wptoolsPython and command-line MediaWiki access for Humans
- get page extracts, image, Infobox data, Wikidata, and more
- get a random page, category, or site
- get page statistics
- get category members
- get site info and stats
- get data in any languageThis package is intended to make it as easy as possible to get data
from MediaWiki instances, expose more Wikidata, and extend Wikimedia
APIs just for kicks. We say "(for Humans)" because that is a goal_.
Questions, feedback, and especially contributions_ are welcome!Install
-------.. code-block:: bash
$ pip install wptools
✨🦄✨Example
-------.. code-block:: python
>>> import wptools
Get a page object:
.. code-block:: python
>>> page = wptools.page('Gandhi')
Get `API:Query`_ data:
.. _`API:Query`: https://www.mediawiki.org/wiki/API:Query
.. code-block:: python
>>> page.get_query()
en.wikipedia.org (query) Gandhi
en.wikipedia.org (imageinfo) File:Portrait Gandhi.jpg
Mahatma Gandhi (en) data
{
aliases: M K Gandhi, Mohandas Gandhi, Bapu, Gandhi, M...
assessments: Pakistan, Alternative Views, South Afric...
description: pre-eminent leader of Indian nationalism ...
extext: Mahātmā **Mohandas Karamchand Gandhi** ( ; H...
extract:Mahātmā Mohandas Karamchand Gandhi {u'size': 2951123, 'kind': 'query-pageimage', u...
label: Mahatma Gandhi
length: 262,790
links: 10 Janpath, 14th Dalai Lama, 1915 Singapore M...
modified: page
pageid: 19379
random: Salt
redirected: {u'to': u'Mahatma Gandhi', u'from': u'Gandhi'}
redirects: {u'ns': 0, u'pageid': 55342, u'title': u'M...
requests: query, imageinfo
title: Mahatma Gandhi
url: https://en.wikipedia.org/wiki/Mahatma_Gandhi
url_raw: https://en.wikipedia.org/wiki/Mahatma_Gandhi?action=raw
watchers: 1,811
wikibase: Q1001
wikidata_url: https://www.wikidata.org/wiki/Q1001
}Get `API:Parse`_ data:
.. _`API:Parse`: https://www.mediawiki.org/wiki/API:Parse
.. code-block:: python
>>> page.get_parse()
en.wikipedia.org (parse) Gandhi
en.wikipedia.org (imageinfo) File:MKGandhi.jpg
Mahatma Gandhi (en) data
{
image: {u'size': 2951123, 'kind': 'parse-image', u'des...
infobox: known_for, other_names, image, signature, bi...
iwlinks: https://biblio.wiki/wiki/Mohandas_K._Gandhi,...
pageid: 19379
parsetree: Redirect...
requests: parse, imageinfo
title: Mahatma Gandhi
wikibase: Q1001
wikidata_url: https://www.wikidata.org/wiki/Q1001
wikitext: {{Redirect|Gandhi}}{{pp-move-indef}}{{pp...
}Get Wikidata_:
.. _Wikidata: https://www.wikidata.org/w/api.php
.. code-block:: python
>>> page = wptools.page(wikibase='Q1001')
>>> page.get_wikidata()
www.wikidata.org (wikidata) Q1001
www.wikidata.org (labels) Q1280678|P535|P434|Q1860|P3762|Q668|P12...
www.wikidata.org (labels) P119|Q1930187|P691|P18|P19|P1066|P509|P...
www.wikidata.org (labels) Q6512732|Q1568|P972|Q84|P1430|P31|Q2140...
www.wikidata.org (labels) P1576|Q4964182|P1368|P140|Q22336956|P12...
en.wikipedia.org (imageinfo) File:Portrait Gandhi.jpg
Mahatma Gandhi (en) data
{
aliases: M K Gandhi, Mohandas Gandhi, Bapu, Gandhi, M...
claims: P646, P535, P906, P434, P648, P3762, P1711, ...
description: pre-eminent leader of Indian nationalism ...
image: {u'size': 2951123, 'kind': 'wikidata-image', u'...
label: Mahatma Gandhi
labels: Q1280678, P535, Q131149, P434, Q1860, P3762,...
modified: wikidata
requests: wikidata, labels, labels, labels, labels, im...
title: Mahatma_Gandhi
what: human
wikibase: Q1001
wikidata: Geni.com profile ID (P2600), National Libr...
wikidata_pageid: 1330
wikidata_url: https://www.wikidata.org/wiki/Q1001
}Get RESTBase_ data:
.. _RESTBase: https://www.mediawiki.org/wiki/RESTBase
.. code-block:: python
>>> page.get_restbase('/page/summary/')
en.wikipedia.org (restbase) /page/summary/Gandhi
Mahatma Gandhi (en) data
{
description: pre-eminent leader of Indian nationalism ...
exhtml:Mahātmā Mohandas Karamchand Gandhi...
exrest: Mahātmā Mohandas Karamchand Gandhi (; Hindust...
image: {'kind': 'restbase-original', u'width': 2024, '...
pageid: 19379
requests: restbase
title: Mahatma_Gandhi
url: https://en.wikipedia.org/wiki/Gandhi
url_raw: https://en.wikipedia.org/wiki/Gandhi?action=raw
}Get all the things (at once):
.. code-block:: python
>>> page.get()
en.wikipedia.org (query) Gandhi
en.wikipedia.org (parse) 19379
www.wikidata.org (wikidata) Q1001
www.wikidata.org (labels) Q1280678|P535|P434|Q1860|P3762|Q668|P12...
www.wikidata.org (labels) P119|Q1930187|P691|P18|P19|P1066|P509|P...
www.wikidata.org (labels) Q6512732|Q1568|P972|Q84|P1430|P31|Q2140...
www.wikidata.org (labels) P1576|Q4964182|P1368|P140|Q22336956|P12...
en.wikipedia.org (restbase) /page/summary/Mahatma_Gandhi
en.wikipedia.org (imageinfo) File:MKGandhi.jpg|File:Portrait Gandhi.jpg
Mahatma Gandhi (en) data
{
aliases: M K Gandhi, Mohandas Gandhi, Bapu, Gandhi, M...
assessments: Pakistan, Alternative Views, South Afric...
claims: P646, P535, P906, P434, P648, P3762, P1711, ...
description: pre-eminent leader of Indian nationalism ...
exhtml:Mahātmā Mohandas Karamchand Gandhi...
exrest: Mahātmā Mohandas Karamchand Gandhi (; Hindust...
extext: Mahātmā **Mohandas Karamchand Gandhi** ( ; H...
extract:Mahātmā Mohandas Karamchand Gandhi {u'size': 2951123, 'kind': 'query-pageimage', u...
infobox: known_for, other_names, image, signature, bi...
iwlinks: https://biblio.wiki/wiki/Mohandas_K._Gandhi,...
label: Mahatma Gandhi
labels: Q1280678, P535, Q131149, P434, Q1860, P3762,...
length: 262,790
links: 10 Janpath, 14th Dalai Lama, 1915 Singapore M...
modified: wikidata, page
pageid: 19379
parsetree: Redirect...
random: Salt
redirected: {u'to': u'Mahatma Gandhi', u'from': u'Gandhi'}
redirects: {u'ns': 0, u'pageid': 55342, u'title': u'M...
requests: query, parse, wikidata, labels, labels, labe...
title: Mahatma_Gandhi
url: https://en.wikipedia.org/wiki/Mahatma_Gandhi
url_raw: https://en.wikipedia.org/wiki/Mahatma_Gandhi?action=raw
watchers: 1,811
what: human
wikibase: Q1001
wikidata: Geni.com profile ID (P2600), National Libr...
wikidata_pageid: 1330
wikidata_url: https://www.wikidata.org/wiki/Q1001
wikitext: {{Redirect|Gandhi}}{{pp-move-indef}}{{pp...
}Get more (expensive) data:
.. code-block:: python
>>> page.get_more()
en.wikipedia.org (querymore) Gandhi
Mahatma Gandhi (en) data
{
categories: Category:1869 births, Category:1948 death...
contributors: 2,606
files: File:Aum Om red.svg, File:Commons-logo.svg, Fi...
languages: {u'lang': u'af', u'title': u'Mahatma Gand...
pageid: 19379
redirected: {u'to': u'Mahatma Gandhi', u'from': u'Gandhi'}
requests: querymore
title: Mahatma Gandhi
views: 19,242
}Get data in `another language`_:
.. _`another language`: https://github.com/siznax/wptools/wiki/Language-Codes
.. code-block:: python
>>> page = wptools.page(lang='zh')
zh.wikipedia.org (random) 🍰
哈莉特·塔布曼 (zh) data
{
pageid: 211070
title: 哈莉特·塔布曼
}Get data from `another wiki`_:
.. _`another wiki`: https://meta.wikimedia.org/wiki/List_of_Wikipedias
.. code-block:: python
>>> page = wptools.page(wiki='en.wikiquote.org')
en.wikiquote.org (random) 🍪
Malala_Yousafzai (en)
{
pageid: 146817
title: Malala_Yousafzai
}Documentation
-------------See our wiki_.
Please enjoy!
@siznax 👹
.. _contributions: https://github.com/siznax/wptools/blob/master/CONTRIBUTING.md
.. _goal: http://docs.python-requests.org/en/master/user/intro/
.. _wiki: https://github.com/siznax/wptools/wiki