https://github.com/steadbytes/wiki2neo

Parse Wikipedia XML dumps into Neo4j CSV imports
https://github.com/steadbytes/wiki2neo

Last synced: 2 months ago
JSON representation

Parse Wikipedia XML dumps into Neo4j CSV imports

• Host: GitHub
• URL: https://github.com/steadbytes/wiki2neo
• Owner: SteadBytes
• Created: 2019-02-19T19:52:15.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2022-12-08T01:40:35.000Z (about 2 years ago)
• Last Synced: 2024-10-11T20:19:54.822Z (2 months ago)
• Language: Python
• Size: 16.6 KB
• Stars: 3
• Watchers: 3
• Forks: 2
• Open Issues: 5
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# wiki2neo

[![PyPI version shields.io](https://img.shields.io/pypi/v/wiki2neo.svg)](https://pypi.python.org/pypi/wiki2neo/)

Produce [Neo4j](https://neo4j.com/) import CSVs from [Wikipedia database dumps](https://en.wikipedia.org/wiki/Wikipedia:Database_download#English-language_Wikipedia)

to build a graph of links between Wikipedia pages.

## Installation

```bash

$ pip install wiki2neo

```

## Usage

```

Usage: wiki2neo [OPTIONS] [WIKI_XML_INFILE]

  Parse Wikipedia pages-articles-multistream.xml[.bz2] dump into two Neo4j import

  CSV files:

      Node (Page) import, headers=["title:ID", "id"]

      Relationships (Links) import, headers=[":START_ID", ":END_ID"]

  Reads from stdin by default, pass [WIKI_XML_INFILE] to read from file.

Options:

  -p, --pages-outfile FILENAME  Node (Pages) CSV output file  [default:pages.csv]

  -l, --links-outfile FILENAME  Relationships (Links) CSV output file [default: links.csv]

  --help                        Show this message and exit.

Import resulting CSVs into Neo4j:

$ neo4j-admin import --nodes:Page pages.csv \

        --relationships:LINKS_TO links.csv \

        --ignore-duplicate-nodes --ignore-missing-nodes --multiline-fields

```

Downloads from Wikipedia are in compressed `xml.bz2` format. `wiki2neo` supports

parsing either the compressed `bz2` file directly or an uncompressed `xml` file:

```bash

# compressed

$ wiki2neo pages-articles-multistream.xml.bz2

# uncompressed

$ wiki2neo pages-articles-multistream.xml

```