{"id":15657560,"url":"https://github.com/msorkhpar/wiki-entity-summarization-toolkit","last_synced_at":"2025-04-14T06:22:00.100Z","repository":{"id":244167730,"uuid":"808406356","full_name":"msorkhpar/wiki-entity-summarization-toolkit","owner":"msorkhpar","description":"a user-friendly toolkit for the Wiki-Entity-Summarization (WikES) project.","archived":false,"fork":false,"pushed_at":"2024-11-28T19:23:56.000Z","size":893,"stargazers_count":23,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-12T01:11:41.655Z","etag":null,"topics":["entity-summarization","esbm","python-library","wikes","wikes-toolkit","wiki-entity-summarization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/msorkhpar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-05-31T02:24:56.000Z","updated_at":"2024-11-28T19:24:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"efb4add8-513c-4b9d-9bd7-4b0b67fc43e7","html_url":"https://github.com/msorkhpar/wiki-entity-summarization-toolkit","commit_stats":null,"previous_names":["msorkhpar/wiki-entity-summarization-toolkit"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msorkhpar%2Fwiki-entity-summarization-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msorkhpar%2Fwiki-entity-summarization-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msorkhpar%2Fwiki-entity-summarization-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msorkhpar%2Fwiki-entity-summarization-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/msorkhpar","download_url":"https://codeload.github.com/msorkhpar/wiki-entity-summarization-toolkit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248830904,"owners_count":21168368,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["entity-summarization","esbm","python-library","wikes","wikes-toolkit","wiki-entity-summarization"],"created_at":"2024-10-03T13:08:08.230Z","updated_at":"2025-04-14T06:22:00.075Z","avatar_url":"https://github.com/msorkhpar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wiki-entity-summarization-toolkit\n\n![PyPI - Status](https://img.shields.io/pypi/status/wikes-toolkit)![PyPI - Downloads](https://img.shields.io/pypi/dm/wikes-toolkit)![GitHub License](https://img.shields.io/github/license/msorkhpar/wiki-entity-summarization-toolkit)[![arXiv](https://img.shields.io/badge/arXiv-2406.08435-B31B1B.svg)](https://doi.org/10.48550/arXiv.2406.08435)\n\nA user-friendly toolkit for the Wiki-Entity-Summarization (WikES) and ESBM datasets.\n\n## Parent project\n\nThis toolkit is part of the\n[Wiki-Entity-Summarization project (WikES)](https://github.com/msorkhpar/wiki-entity-summarization).\n\n## Installation\n\n```bash\npip install wikes-toolkit\n```\n\n## Usage\n\n### WikES usage (object based)\n\nUsing this toolkit with Object-Oriented Programming (OOP) is straightforward. \u003c/br\u003e\n\nHere is an example of how to use it:\n\n```python\nfrom wikes_toolkit import WikESToolkit, WikESVersions, WikESGraph\n\ntoolkit = WikESToolkit(save_path=\"./data\")  # save_path is optional\nG = toolkit.load_graph(\n    WikESGraph,\n    WikESVersions.V1.WikiCinema.SMALL,\n    entity_formatter=lambda e: f\"Entity({e.wikidata_label})\",\n    predicate_formatter=lambda p: f\"Predicate({p.label})\",\n    triple_formatter=lambda\n        t: f\"({t.subject_entity.wikidata_label})-[{t.predicate.label}]-\u003e ({t.object_entity.wikidata_label})\"\n)\n\nroot_nodes = G.root_entities()\nnodes = G.entities()\nnx_graph = G._G\nedges = G.triples()\nlabels = G.predicates()\nnumber_of_nodes = G.total_entities()\nnumber_of_directed_edges = G.total_triples()\nnode = G.fetch_entity('Q303')\nnode_degree = G.degree('Q303')\nneighbors = G.ground_truths(node)\nground_truth_summaries = G.ground_truths(root_nodes[0])\nG.mark_triple_as_summary(edges[0].subject_entity, edges[0])\n\nfor root in G.root_entities():\n    print(f\"Neighbors of [{root}]:\")\n    for triple in G.neighbors(root):\n        print(triple)\n\n    for _ in range(5):\n        print(\"*\" * 40)\n\n    print(\"Ground truth summaries:\")\n    for summary in G.ground_truths(root):\n        print(summary)\n    G.mark_triples_as_summaries(root, G.neighbors(root))\n    break\n\nf1 = G.f1_score()\nf1_5 = G.f1_score(5)\nf1_10 = G.f1_score(10)\nmap = G.map_score()\nmap_5 = G.map_score(5)\nmap_10 = G.map_score(10)\n\"\"\" Output of the above code:\nNeighbors of [Entity(Elvis Presley)]:\n(Elvis Presley)-[military unit]-\u003e (32nd Cavalry Regiment)\n(Elvis Presley)-[genre]-\u003e (blues)\n...\n(Jim Morrison)-[influenced by]-\u003e (Elvis Presley)\n(Elvis Country – I'm 10,000 Years Old)-[performer]-\u003e (Elvis Presley)\n(The King)-[main subject]-\u003e (Elvis Presley)\n****************************************\nGround truth summaries:\n(Elvis Presley)-[genre]-\u003e (rockabilly)\n(Million Dollar Quartet)-[has part(s)]-\u003e (Elvis Presley)\n(Jailhouse Rock)-[cast member]-\u003e (Elvis Presley)\n...\n(Viva Las Vegas)-[cast member]-\u003e (Elvis Presley)\n(Elvis Presley)-[genre]-\u003e (rhythm and blues)\n(Elvis Presley)-[record label]-\u003e (Sun Records)\n(Elvis Presley)-[genre]-\u003e (pop music)\n\"\"\"\n```\n\nTo load all graphs, you can use the `load_all_graphs` method:\n\n```python\nfrom wikes_toolkit import WikESToolkit, WikESVersions, WikESGraph\n\nfor dataset_name, G in WikESToolkit(save_path=\"./data\").load_all_graphs(WikESGraph, WikESVersions.V1):\n    print(f\"Dataset [{dataset_name}:\")\n    print(G.total_entities())\n```\n\n### Pandas usage\n\nThere is another version of this toolkit that uses Pandas DataFrame to store the graph data. To use this version, you\ncan change the first parameter of the `load_graph` method to `PandasWikESGraph`:\n\n```python\nfrom wikes_toolkit import WikESToolkit, PandasWikESGraph, WikESVersions\n\ntoolkit = WikESToolkit()\nG = toolkit.load_graph(PandasWikESGraph, WikESVersions.V1.WikiLitArt.SMALL, entity_formatter=lambda e: e.identifier)\n\nroot_nodes = G.root_entities()\nfirst_root_node = G.root_entity_ids()[0]\nnodes = G.entities()\nedges = G.triples()\nfirst_edge = G.fetch_triple(edges.iloc[0])\nlabels = G.predicates()\nnumber_of_nodes = G.total_entities()\nnumber_of_directed_edges = G.total_triples()\nnode = G.fetch_entity('Q303')\nnode_degree = G.degree('Q303')\nground_truths = G.ground_truths(node)\nneighbors = G.neighbors(node)\nground_truth_summaries = G.ground_truths(first_root_node)\nG.mark_triple_as_summary(\n    ground_truths.iloc[0].name,\n    (ground_truths.iloc[0]['subject'], ground_truths.iloc[0]['predicate'], ground_truths.iloc[0]['object'])\n)\nG.mark_triples_as_summaries(node, neighbors.iloc[0:2])\nf1 = G.f1_score()\nf1_5 = G.f1_score(5)\nf1_10 = G.f1_score(10)\nmap = G.map_score()\nmap_norel = G.map_score(no_rel=True)\nmap_5 = G.map_score(5)\nmap_10 = G.map_score(10)\nG.clear_summaries()\n\n```\n\n### Export WikESGraphs as CSV files\n\nYou can export the graphs as CSV files using the `export_as_csv` method. This method exports the graph as three CSV\nfiles:\n\n```python\nfrom wikes_toolkit import WikESToolkit, WikESVersions, PandasWikESGraph\n\ntoolkit = WikESToolkit()\n\nfor dataset, G in toolkit.load_all_graphs(PandasWikESGraph, WikESVersions.V1):\n    G.export_as_csv(\"./data\")\n```\n\n**Note: This method has been implemented for WikESPandasGraph, others yet to be implemented.**\n\n## ESBM\n\nWe also cover ESBM datasets with almost the same functionalities and syntax as before. If you want to check how we have\nconverted ESBM datasets to NetworkX format, or if you want to use GraphML formats of this dataset,\ncheck [ESBM-to-nx-format](https://github.com/msorkhpar/ESBM-to-nx-format) repository.\u003c/br\u003e\n\nHere is an example of loading ESBM and ESM_Plus datasets with this toolkit:\n\n```python\nfrom wikes_toolkit import WikESToolkit, ESBMGraph, ESBMVersions\n\ntoolkit = WikESToolkit()\nG = toolkit.load_graph(\n    ESBMGraph,\n    ESBMVersions.Plus.DBPEDIA_FULL,  # or ESBMVersions.Plus.DBPEDIA_FULL, ESBMVersions.V1Dot2.LMDB_TRAIN_1, etc.\n    entity_formatter=lambda e: e.identifier\n)\n\nroot_nodes = G.root_entities()\nfirst_root_node = G.root_entity_ids()[0]\nnodes = G.entities()\nedges = G.triples()\nlabels = G.predicates()\nnumber_of_nodes = G.total_entities()\nnumber_of_directed_edges = G.total_triples()\nnode = G.fetch_entity('http://dbpedia.org/resource/Adrian_Griffin')\nnode_degree = G.degree('http://dbpedia.org/resource/Adrian_Griffin')\ngold_top5_0 = G.gold_top_5(node, 0)\ngold_top10_0 = G.gold_top_10(node, 0)\nneighbors = G.neighbors(node)  # or G.neighbors('http://dbpedia.org/resource/Adrian_Griffin')\n\nG.mark_triples_as_summaries(\n    \"http://dbpedia.org/resource/3WAY_FM\",\n    [\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://purl.org/dc/terms/subject',\n            'http://dbpedia.org/resource/Category:Radio_stations_in_Victoria'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://xmlns.com/foaf/0.1/homepage',\n            'http://3wayfm.org.au'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://dbpedia.org/ontology/broadcastArea',\n            'http://dbpedia.org/resource/Victoria_(Australia)'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://dbpedia.org/ontology/callsignMeaning',\n            '3 - Victoria'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://xmlns.com/foaf/0.1/name',\n            '3WAY FM@en'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://www.w3.org/2000/01/rdf-schema#label',\n            '3WAY FM@en'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://dbpedia.org/ontology/callsignMeaning',\n            'Warrnambool And You'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://dbpedia.org/ontology/broadcastArea',\n            'http://dbpedia.org/resource/Warrnambool'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',\n            'http://schema.org/RadioStation'\n        ),\n        (\n            'http://dbpedia.org/resource/3WAY_FM',\n            'http://purl.org/dc/terms/subject',\n            'http://dbpedia.org/resource/Category:Radio_stations_established_in_1990'\n        )]\n)\n\nG.clear_summaries()\n\nG.mark_triples_as_summaries(\n    \"http://dbpedia.org/resource/3WAY_FM\",\n    [(\n        'http://dbpedia.org/resource/3WAY_FM',\n        'http://purl.org/dc/terms/subject',\n        'http://dbpedia.org/resource/Category:Radio_stations_in_Victoria'\n    )]\n)\n\nG.mark_triples_as_summaries(\n    \"http://dbpedia.org/resource/Adrian_Griffin\",\n    [\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://dbpedia.org/ontology/activeYearsEndYear',\n            '2008'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://purl.org/dc/terms/subject',\n            'http://dbpedia.org/resource/Category:Orlando_Magic_assistant_coaches'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://dbpedia.org/ontology/birthDate',\n            '1974-07-04'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://dbpedia.org/ontology/height',\n            '1.9558'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://dbpedia.org/ontology/weight',\n            '98431.2'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://dbpedia.org/ontology/team',\n            'http://dbpedia.org/resource/Orlando_Magic'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://purl.org/dc/terms/subject',\n            'http://dbpedia.org/resource/Category:Basketball_players_from_Kansas'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',\n            'http://dbpedia.org/class/yago/BasketballPlayersFromKansas'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',\n            'http://dbpedia.org/class/yago/BasketballCoach109841955'\n        ),\n        (\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',\n            'http://dbpedia.org/class/yago/BasketballPlayer109842047'\n        ),\n    ]\n)\n\nf1_5 = G.f1_score(5)\nf1_10 = G.f1_score(10)\nmap_5 = G.map_score(5)\nmap_10 = G.map_score(10)\n```\n\n#### Using Ranked NT File as Summary (Legacy Support)\n\nAlthough this toolkit can process triples marked with `.nt` ranked files, we don't recommend using this feature as it\ndoesn't align with our toolkit's core approach.\n\n\n```python\nfrom wikes_toolkit import WikESToolkit, PandasESBMGraph, ESBMGraph, ESBMVersions\n\ntoolkit = WikESToolkit()\nG = toolkit.load_graph(\n    ESBMGraph, # or PandasESBMGraph\n    ESBMVersions.V1Dot2.DBPEDIA_FULL,\n    entity_formatter=lambda e: e.identifier,\n)\n\n# Mark summaries for '3WAY_FM'\nG.mark_nt_file_as_summary(\n    1,  # '3WAY_FM' eid\n    './result/dbpedia/1/1_rank.nt'  # .nt file path\n)\n\n# Mark summaries for 'Adrian_Griffin'\nG.mark_nt_file_as_summary(\n    'http://dbpedia.org/resource/Adrian_Griffin',  # ESBM entity name\n    './result/dbpedia/2/2_rank.nt'  # .nt file path\n)\n\nf1_5 = G.f1_score(5)\nf1_10 = G.f1_score(10)\n\n```\n\n### ESBM Pandas usage\n\nFor Pandas version of ESBM datasets, you can use `PandasESBMGraph` instead of `ESBMGraph`:\n\n```python\nfrom wikes_toolkit import WikESToolkit, PandasESBMGraph, ESBMVersions\nimport pandas as pd\n\ntoolkit = WikESToolkit()\nG = toolkit.load_graph(PandasESBMGraph, ESBMVersions.V1Dot2.DBPEDIA_FULL, entity_formatter=lambda e: e.identifier)\n\nfirst_root_node_id = G.root_entity_ids()[0]\nroot_nodes = G.root_entities()\nfirst_root_node = G.root_entities().iloc[0]\nnodes = G.entities()\nedges = G.triples()\nlabels = G.predicates()\nnumber_of_nodes = G.total_entities()\nnumber_of_directed_edges = G.total_triples()\nnode = G.fetch_entity('http://dbpedia.org/resource/Adrian_Griffin')\nnode_degree = G.degree('http://dbpedia.org/resource/Adrian_Griffin')\ngold_top5_0 = G.gold_top_5(node, 0)\ngold_top10_0 = G.gold_top_10(node, 0)\nneighbors = G.neighbors(node)\nall_golds = G.all_gold_top_k(5)\ngolds = G.gold_top_5(G.root_entities().iloc[0], 4)\n\nG.mark_triples_as_summaries(\n    \"http://dbpedia.org/resource/Adrian_Griffin\",\n    pd.Series(\n        [\n            'http://dbpedia.org/resource/Adrian_Griffin',\n            'http://dbpedia.org/ontology/activeYearsEndYear',\n            '2008'\n        ],\n        index=['subject', 'predicate', 'object']\n    )\n)\nf1_5 = G.f1_score(5)\n\n```\n\n## Citation\n\nIf you use this project in your research, please cite the following paper:\n\n```bibtex\n@misc{javadi2024wiki,\n    title = {Wiki Entity Summarization Benchmark},\n    author = {Saeedeh Javadi and Atefeh Moradan and Mohammad Sorkhpar and Klim Zaporojets and Davide Mottin and Ira Assent},\n    year = {2024},\n    eprint = {2406.08435},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.IR}\n}\n```\n\nIn case you use ESBM datasets, please cite their paper as well ours:\n\n```bibtex\n@inproceedings{esbm,\n    author = {Qingxia Liu and\n Gong Cheng and\n Kalpa Gunaratna and\n Yuzhong Qu},\n    title = {ESBM: An Entity Summarization Benchmark},\n    booktitle = {ESWC},\n    year = {2020}\n}\n``` \n\n## License\n\nThis project is licensed under the CC-BY-4.0 License. See the [LICENSE](LICENSE) file for details.\nAlso, you can check ESBM license [here](https://github.com/nju-websoft/ESBM/blob/master/LICENSE.txt)\nor [here](https://opendatacommons.org/licenses/by/1-0/index.html).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsorkhpar%2Fwiki-entity-summarization-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmsorkhpar%2Fwiki-entity-summarization-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsorkhpar%2Fwiki-entity-summarization-toolkit/lists"}