{"id":13446395,"url":"https://github.com/bcicen/wikitables","last_synced_at":"2025-04-04T15:11:36.504Z","repository":{"id":62589104,"uuid":"60958475","full_name":"bcicen/wikitables","owner":"bcicen","description":"Import tables from any Wikipedia article as a dataset in Python","archived":false,"fork":false,"pushed_at":"2021-10-19T20:01:30.000Z","size":89,"stargazers_count":290,"open_issues_count":4,"forks_count":34,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-04-27T03:27:06.243Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bcicen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-06-12T09:22:38.000Z","updated_at":"2024-03-26T17:44:50.000Z","dependencies_parsed_at":"2022-11-03T17:56:35.289Z","dependency_job_id":null,"html_url":"https://github.com/bcicen/wikitables","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fwikitables","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fwikitables/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fwikitables/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcicen%2Fwikitables/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bcicen","download_url":"https://codeload.github.com/bcicen/wikitables/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247198469,"owners_count":20900081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T05:00:52.635Z","updated_at":"2025-04-04T15:11:36.479Z","avatar_url":"https://github.com/bcicen.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# wikitables\n\n[![Documentation Status](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](http://wikitables.readthedocs.org/en/latest) [![PyPI version](https://badge.fury.io/py/wikitables.svg)](https://badge.fury.io/py/wikitables) [![Build Status](https://travis-ci.com/bcicen/wikitables.svg?branch=master)](https://travis-ci.com/bcicen/wikitables)\n\nImport tables from any Wikipedia article as a dataset in Python\n\n## Installing\n\n```bash\npip install wikitables\n```\n\n## Usage\n\n### Importing\n\nImporting all tables from a given article:\n\n```python\nfrom wikitables import import_tables\ntables = import_tables('List of cities in Italy')  # returns a list of WikiTable objects\n```\n\nTo import an article from a different language, simply add the Wikipedia language code as an argument to `import_tables`.\nThis will also show country names in the given language.\n\n```python\ntables = import_tables('İtalya\\'daki_şehirler_listesi', 'tr')  # returns a list of WikiTable objects\n```\n\n### Accessing\n\nIterate over a table's rows:\n```python\nprint(tables[0].name)\nfor row in tables[0].rows:\n    print('{City}: {Area(km2)}'.format(**row))\n```\n\noutput:\n```\nList of cities in Italy[0]\nMilan: 4,450.11\nNaples: 3,116.52\nRome: 3,340.41\nTurin: 1,328.40\n...\n```\n\nOr return the table encoded as JSON:\n```python\ntables[0].json()\n```\n\noutput:\n```json\n[\n    {\n        \"City\": \"Milan\",\n        \"Population January 1, 2014\": \"6,623,798\",\n        \"Density(inh./km2)\": \"1,488\",\n        \"Area(km2)\": \"4,450.11\"\n    },\n    {\n        \"City\": \"Naples\",\n        \"Population January 1, 2014\": \"5,294,546\",\n        \"Density(inh./km2)\": \"1,699\",\n        \"Area(km2)\": \"3,116.52\"\n    },\n    ...\n```\n\n### Table Head\n\nAfter import, table column names may been modified by setting a new header:\n\n```python\ntable.head = [ 'newfield1', 'newfield2', 'newfield3' ]\n```\n\nThis change will be recursively reflected on all of a given tables rows.\n\n### Commandline\n\nWikitables also comes with a simple cli tool to fetch and output table json:\n\n```bash\n# from article name\nwikitables 'List of cities in Italy'\n\n# from URL\nwikitables https://en.wikipedia.org/wiki/Radio_spectrum#ITU\n```\n\n### Creating list of DataFrames\n\n```python\nfrom wikitables import import_tables\nimport pandas as pd\n\n\ndef get_df_from_table_object(table):\n    rows = [row for row in table.rows]\n    return pd.DataFrame(rows)\n\n\ndef get_list_of_df_of_wiki_article(wiki_title):\n    tables = import_tables(wiki_title)\n    return [get_df_from_table_object(table) for table in tables]\n\n\nprint(get_list_of_df_of_wiki_article(wiki_title='List of cities in Italy'))\n```\n\noutput:\n```bash\n[    Rank         City 2011    Census 2020    Estimate                Change    Region\n0      1         Rome        2617175          2856133     9.130379130168986     Lazio\n1      2        Milan        1242123          1378689     10.99456334034552  Lombardy\n2      3       Naples         962003           959188   -0.2926186300874267  Campania\n3      4        Turin         872367           875698   0.38183470947434905  Piedmont\n4      5      Palermo         657651           663401    0.8743239195257102    Sicily\n..   ...          ...            ...              ...                   ...       ...\n139  140  Battipaglia          51133            51005  -0.25032757710284903  Campania\n140  141          Rho          50686            50904    0.4300990411553407  Lombardy\n141  142       Chieti          54305            50770    -6.509529509253287   Abruzzo\n142  143      Scafati          50794            50686   -0.2126235382131747  Campania\n143  144    Scandicci          50309            50645    0.6678725476554792   Tuscany\n\n[144 rows x 6 columns]]\n```\n\n\n## Roadmap\n\nSome planned and wishlist features:\n\n* Type guesing from MediaWiki template values\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcicen%2Fwikitables","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbcicen%2Fwikitables","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcicen%2Fwikitables/lists"}