https://github.com/domargan/parallel-packed-csr

A parallel packed CSR data structure for large-scale dynamic graphs
https://github.com/domargan/parallel-packed-csr

csr data-structures dynamic-graphs evolving-graphs graph graph-data graph-databases graph-datastructures graph-processing graphs packed-memory-array pcsr stream-graphs streaming-graph-data streaming-graph-processing

Last synced: 30 days ago
JSON representation

A parallel packed CSR data structure for large-scale dynamic graphs

• Host: GitHub
• URL: https://github.com/domargan/parallel-packed-csr
• Owner: domargan
• License: mit
• Created: 2020-06-30T16:17:26.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2022-10-24T00:46:45.000Z (about 2 years ago)
• Last Synced: 2024-08-03T18:21:05.136Z (4 months ago)
• Topics: csr, data-structures, dynamic-graphs, evolving-graphs, graph, graph-data, graph-databases, graph-datastructures, graph-processing, graphs, packed-memory-array, pcsr, stream-graphs, streaming-graph-data, streaming-graph-processing
• Language: C++
• Homepage:
• Size: 17.4 MB
• Stars: 13
• Watchers: 3
• Forks: 4
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-dynamic-graphs - PPCSR

README

        
# Parallel Packed CSR

A parallel implementation of the packed CSR data structure, based on its initial single threaded design[[1]](#1)(https://github.com/wheatman/Packed-Compressed-Sparse-Row/) and further parallel extension [[2]](#2).

Accepts a core graph, which it loads first and an update file, which it uses to update the core graph (insert or delete edges).

Performs edge insertions and deletions in parallel and uses NUMA-aware placement of data and worker threads.

# Prerequisites

* CMAKE 3.8 or newer required.

# Build

Create a build directory and run cmake & make there:

```

$ mkdir build && cd build

$ cmake ..

$ make

```

# Running

Run the `parallel-packed-csr` binary from your build directory.

## Command line options

* `-threads=`: specifies number of threads to use for updates, default=8

* `-size=`: specifies number of edges that will be read from the update file, default=1000000

* `-lock_free`: runs the data structure lock-free version of binary search, locks during binary search by default

* `-partitions_per_domain=`: specifies the number of graph partitions per NUMA domain

* `-insert`: inserts the edges from the update file to the core graph

* `-delete`: deletes the edges from the update file from the core graph

* `-core_graph=`: specifies the filename of the core graph

* `-update_file=`: specifies the filename of the update file

* Available partitioning strategies (if multiple strategies are given, the last one is used):

  * `-ppcsr`: No partitioning

  * `-pppcsr`: Partitioning (1 partition per NUMA domain)

  * `-pppcsrnuma`: Partitioning with explicit NUMA optimizations (default)

# Authors

* Eleni Alevra

* Christian Menges 

* Dom Margan 

# References

[1]

Wheatman, B., & Xu, H. (2018).

Packed Compressed Sparse Row: A Dynamic Graph

Representation. 

2018 IEEE High Performance Extreme Computing Conference, HPEC 2018.

[2]

Alevra, E., & Pietzuch, P. (2020).

A Parallel Data Structure for Streaming Graphs. 

Master’s thesis, Imperial College London, 2020.