https://github.com/blacklanternsecurity/spiderfoot-neo4j

Import, visualize, and analyze SpiderFoot scans in Neo4j, a graph database
https://github.com/blacklanternsecurity/spiderfoot-neo4j

Last synced: 2 months ago
JSON representation

Import, visualize, and analyze SpiderFoot scans in Neo4j, a graph database

• Host: GitHub
• URL: https://github.com/blacklanternsecurity/spiderfoot-neo4j
• Owner: blacklanternsecurity
• License: gpl-3.0
• Created: 2021-08-12T20:07:51.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2023-01-16T18:11:22.000Z (about 2 years ago)
• Last Synced: 2024-11-10T05:13:43.642Z (3 months ago)
• Language: Python
• Size: 47.9 KB
• Stars: 66
• Watchers: 6
• Forks: 7
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• project-awesome - blacklanternsecurity/spiderfoot-neo4j - Import, visualize, and analyze SpiderFoot scans in Neo4j, a graph database (Python)

README

        
# SpiderFoot Neo4j Tools

Import, visualize, and analyze SpiderFoot OSINT data in Neo4j, a graph database

![A big graph](https://user-images.githubusercontent.com/20261699/129272391-ba0299a1-83da-4037-a55d-d607b620dc05.png)

### Step 1: Installation

NOTE: This installs the `sfgraph` command-line utility

~~~

$ pip install spiderfoot-neo4j

~~~

### Step 2: Start Neo4j

NOTE: [Docker must first be installed](https://docs.docker.com/get-docker/)

~~~

$ docker run --rm --name sfgraph -v "$(pwd)/neo4j_database:/data" -e 'NEO4J_AUTH=neo4j/CHANGETHISIFYOURENOTZUCK' -e 'NEO4JLABS_PLUGINS=["apoc", "graph-data-science"]' -e 'NEO4J_dbms_security_procedures_unrestricted=apoc.*,gds.*' -p "7474:7474" -p "7687:7687" neo4j

~~~

### Step 3: Import Scans

![Spiderfoot scan ID in web browser](https://user-images.githubusercontent.com/20261699/129256011-ff751637-afdd-4632-8335-24ffae2ff65e.png)

~~~

$ sfgraph path_to/spiderfoot.db -s   ...

~~~

### Step 4: Browse Spiderfoot Data in Neo4j

Visit http://127.0.0.1:7474 and log in with `neo4j/CHANGETHISIFYOURENOTZUCK`

![Spiderfoot data in Neo4j](https://user-images.githubusercontent.com/20261699/129265805-056ef29a-aea9-4be0-90fc-383ae15510c5.png)

### Step 5 (Optional): Use cool algorithms to find new targets

The `--suggest` option will rank nodes based on their connectedness in the graph. This is perfect for finding closely-related affiliates (child companies, etc.) to scan and add to the graph. By default, [Harmonic Centrality](https://neo4j.com/docs/graph-data-science/current/algorithms/harmonic-centrality/) is used, but others such as [PageRank](https://neo4j.com/docs/graph-data-science/current/algorithms/page-rank/) can be specified with `--closeness-algorithm`

~~~

$ sfgraph --suggest DOMAIN_NAME

~~~

![Closeness scores](https://user-images.githubusercontent.com/20261699/129263951-977d1092-8fdd-4ea1-bccb-d1ab6e4a6612.png)

## Example CYPHER Queries

~~~

# match all INTERNET_NAMEs

MATCH (n:INTERNET_NAME) RETURN n

# match multiple event types

MATCH (n) WHERE n:INTERNET_NAME OR n:DOMAIN_NAME OR n:EMAILADDR RETURN n

# match by attribute

MATCH (n {data: "evilcorp.com"}) RETURN n

# match by spiderfoot module (relationship)

MATCH p=()-[r:WHOIS]->() RETURN p

# shortest path to all INTERNET_NAMEs from seed domain

MATCH p=shortestPath((d:DOMAIN_NAME {data:"evilcorp.com"})-[*]-(n:INTERNET_NAME)) RETURN p

# match only primary targets (non-affiliates)

MATCH (n {scanned: true}) return n

# match only affiliates

MATCH (n {affiliate: true}) return n

~~~

## CLI Help

~~~

sfgraph [-h] [-db SQLITEDB] [-s SCANS [SCANS ...]] [--uri URI] [-u USERNAME] [-p PASSWORD] [--clear] [--suggest SUGGEST]

               [--closeness-algorithm {pageRank,articleRank,closenessCentrality,harmonicCentrality,betweennessCentrality,eigenvectorCentrality}] [-v]

optional arguments:

  -h, --help            show this help message and exit

  -db SQLITEDB, --sqlitedb SQLITEDB

                        Spiderfoot sqlite database

  -s SCANS [SCANS ...], --scans SCANS [SCANS ...]

                        scan IDs to import

  --uri URI             Neo4j database URI (default: bolt://127.0.0.1:7687)

  -u USERNAME, --username USERNAME

                        Neo4j username (default: neo4j)

  -p PASSWORD, --password PASSWORD

                        Neo4j password

  --clear               Wipe the Neo4j database

  --suggest SUGGEST     Suggest targets of this type (e.g. DOMAIN_NAME) based on their connectedness in the graph

  --closeness-algorithm {pageRank,articleRank,closenessCentrality,harmonicCentrality,betweennessCentrality,eigenvectorCentrality}

                        Algorithm to use when suggesting targets

  -v, -d, --debug       Verbose / debug

~~~