Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/aaronlelevier/neo4j-github-followers
Example project combining Neo4j, Python and Github API
https://github.com/aaronlelevier/neo4j-github-followers
github neo4j python
Last synced: about 1 month ago
JSON representation
Example project combining Neo4j, Python and Github API
- Host: GitHub
- URL: https://github.com/aaronlelevier/neo4j-github-followers
- Owner: aaronlelevier
- License: other
- Created: 2018-04-25T15:31:03.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2019-06-04T16:13:25.000Z (over 5 years ago)
- Last Synced: 2024-08-07T23:18:21.768Z (5 months ago)
- Topics: github, neo4j, python
- Language: Python
- Size: 8.79 KB
- Stars: 20
- Watchers: 7
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# README
## Summary
This is an example project combining Neo4j, Python and Github API
This project demostrates:
1. Call Github API using Python multithreading
2. Collect data from threads and output to a `followers.csv`
3. Use `followers.csv` which contains `User` data and `Follower` relationships to output CSVs distict for each:
- `users.csv` - distinct Users
- `follows.csv` - distinct mappings from User-to-Follows
4. Load CSVs into Neo4j
5. Create Neo4j relationships
6. Query Neo4j using the Python `neo4j-driver` client
7. Write queried data to write `json`
8. Use `json` to generate a `D3Js` web graph
## Quick Example
The current `web/d3-results.json` is already populated with my data, so just run:
```
cd web
npm install http-server
http-server
```
## To build and run the project with your data
Create a Github API token - this will be needed to make requests if making over 60 requests an hour, the unauthenticated rate limit
`pip install -r requirements.txt`
Populate CSVs for the Github User whose followers is being populated
```
python code/followers.py --username
python code/follows.py --username
python code/users.py --username
```
Data can be loaded into Neo4j using the below queries
After data is loaded, run this command to populate `web/d3-results.json`.
Environment varialbe `NEO4J_PASSWORD` will need to be set for the Neo4j database connection in `query.py`
```
python query.py
```
Start Node server and see data with D3Js
```
cd web
npm install http-server
http-server
```
## Neo4j Queries
### Load CSV
CSV must first be placed in the Neo4j `/import/` dir:
```
/Users/aaron/Library/Application\ Support/Neo4j\ Desktop/Application/neo4jDatabases/database-0bf7614b-cb8d-405e-baef-e1b94485e662/installation-3.3.5/import/
```
Load `Users` table using CSV:
```
LOAD CSV WITH HEADERS FROM "file:///users.csv" AS row
CREATE (u:User)
SET u = row
```
Load `Follows` table using CSV:
```
LOAD CSV WITH HEADERS FROM "file:///follows.csv" AS row
CREATE (u:Follows)
SET u = row
```
### Create Relationships
Create `FOLLOWS` relationships using previous 2 tables:
```
MATCH (u:User),(f:Follows), (u2:User)
WHERE u.login = f.login
AND f.follows = u2.login
CREATE (u)-[:FOLLOWS]->(u2)
```
### If you have to delete data and start over
First delete relationships:
```
MATCH ()-[r:FOLLOWS]-()
DELETE r
```
Then delete tables:
```
MATCH (r:User)
DELETE r
```
```
MATCH (r:Follows)
DELETE r
```
## Libraries Used
### Database
[Neo4j](https://neo4j.com/)
### Python
[requests](https://github.com/requests/requests)
[neo4j-driver](https://github.com/neo4j/neo4j-python-driver)
### Javascript
[d3js](https://github.com/d3/d3)
[queue.js](https://github.com/d3/d3-queue)
## LICENSE
BSD-2
## CONTRIBUTING
Contact me if you found this interesting
## FUTURE PLANS
Add more end-to-end Neo4j Graph Database examples
## Example Graph output images from this repo
### 1st and 2nd level relationships (D3Js)
### 1st level relationships (Neo4j)
### 1st and 2nd level relationships (Neo4j)
