https://github.com/peter17/pywdcollections

A bot framework based on pyWikiBot to harvest Wikipedia templates to Wikidata
https://github.com/peter17/pywdcollections

pywikibot sparql-query-builder wikidata-bot wikipedia

Last synced: about 2 months ago
JSON representation

A bot framework based on pyWikiBot to harvest Wikipedia templates to Wikidata

• Host: GitHub
• URL: https://github.com/peter17/pywdcollections
• Owner: peter17
• Created: 2019-12-13T22:24:59.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2024-10-20T16:36:35.000Z (2 months ago)
• Last Synced: 2024-10-20T19:42:59.602Z (2 months ago)
• Topics: pywikibot, sparql-query-builder, wikidata-bot, wikipedia
• Language: Python
• Size: 143 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Security: SECURITY.md

Awesome Lists containing this project

README

        
A Python framework based on pyWikibot to harvest Wikipedia templates to Wikidata.

# Usage example

This harvests properties Commonscat (P373), image (P18) and administrative location (P131) from the English Wikipedia templates "Commonscat" and "Infobox cemetery".

Running this will create a local SQLite database, download all cemeteries items from Wikidata and for the items that miss at least one of those three properties, it will scan all English Wikipedia articles related to the items to search for those properties in those templates.

    #!/usr/bin/env python3

    # -*- coding: utf-8 -*-

    import os

    import pywdcollections as PYWDC

    path = os.path.dirname(os.path.realpath(__file__))

    class Cemeteries(PYWDC.Collection):

        def __init__(self, pywb):

            self.db = PYWDC.Database(path + '/cemeteries.db')

            self.name = 'cemeteries'

            self.commit_frequency = 10000

            self.main_type = 39614 # cemetery

            self.properties = [18, 131, 373]

            self.languages = ['en']

            self.templates = {

                'enwiki': {

                    'Commonscat': 373,

                    'Infobox cemetery': {

                        'image': 18,

                        'location': 131,

                    },

                },

        }

        super().__init__(pywb)

    if __name__ == '__main__':

        pywb = PYWDC.PYWB('', 'en')

        pywb.wikidata.login()

        collection = Cemeteries(pywb)

        collection.fetch()

        collection.copy_ciwiki_to_declaration()

        collection.update_outdated_items()

        collection.harvest_templates()

        collection.copy_harvested_properties([18, 131, 373])