{"id":13559212,"url":"https://github.com/entrepreneur-interet-general/graph-explorer","last_synced_at":"2025-06-27T22:34:43.903Z","repository":{"id":92023057,"uuid":"129085284","full_name":"entrepreneur-interet-general/graph-explorer","owner":"entrepreneur-interet-general","description":"Graph Explorer","archived":false,"fork":false,"pushed_at":"2020-01-03T18:54:18.000Z","size":30040,"stargazers_count":28,"open_issues_count":2,"forks_count":10,"subscribers_count":19,"default_branch":"master","last_synced_at":"2024-11-04T10:43:54.216Z","etag":null,"topics":["d3js","entrepreneur-interet-general","flask","fraud-detection","graph","gremlin","janusgraph","network-visualization","python","vuejs"],"latest_commit_sha":null,"homepage":"https://graph-explorer.fr","language":"Vue","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/entrepreneur-interet-general.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-04-11T11:47:20.000Z","updated_at":"2024-10-22T19:02:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"7cee0fc8-0e40-41ff-8fc5-39302c26acc6","html_url":"https://github.com/entrepreneur-interet-general/graph-explorer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2Fgraph-explorer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2Fgraph-explorer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2Fgraph-explorer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/entrepreneur-interet-general%2Fgraph-explorer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/entrepreneur-interet-general","download_url":"https://codeload.github.com/entrepreneur-interet-general/graph-explorer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224194980,"owners_count":17271571,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["d3js","entrepreneur-interet-general","flask","fraud-detection","graph","gremlin","janusgraph","network-visualization","python","vuejs"],"created_at":"2024-08-01T12:05:25.942Z","updated_at":"2024-11-12T00:26:22.465Z","avatar_url":"https://github.com/entrepreneur-interet-general.png","language":"Vue","funding_links":[],"categories":["Vue","flask"],"sub_categories":[],"readme":"# Graph Explorer\r\nVisualize and explore a large graph of money transactions.\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n\u003cimg src=\"docs/img/home.png\" alt=\"architecture\"\u003e\r\n\u003c/div\u003e\r\n\r\n## Table of Contents\r\n\r\n* **[Architecture](#architecture)**\r\n* **[Demo](#demo)**\r\n* **[Installation](#installation)**\r\n  * [Prerequisites](#prerequisites)\r\n  * [Steps](#steps)\r\n* **[Development](#development)**\r\n  * [Flask Server](#flask-server)\r\n  * [Frontend](#frontend)\r\n* **[Credits](#credits)**\r\n* **[License](#license)**\r\n\r\n## Architecture\r\n\r\n* JanusGraph 0.2.1\r\n* ScyllaDB 2.2.0 (storage backend for JanusGraph)\r\n* Elasticsearch 6.0.1 (index backend for JanusGraph)\r\n* Python Flask server\r\n* Vue.js\r\n* d3.js v4 force layout\r\n\r\n\u003c/br\u003e\r\n\u003cdiv align=\"center\"\u003e\r\n\u003cimg src=\"docs/img/architecture.png\" alt=\"architecture\" width=\"400px\"\u003e\r\n\u003c/div\u003e\r\n\r\n\r\n## Demo\r\nStart exploring the graph from any node (e.g. with *Amanda Walker*)\r\n[https://graph-explorer.fr](https://graph-explorer.fr)\r\n\r\n\r\n## Installation\r\n\r\n### Prerequisites\r\n\r\n* Docker CE \u003e= 1.13.0\r\n* docker-compose \u003e= 1.10.0\r\n\r\nBelow is a step by step guide explaining how to install the app with sample data.\r\nIf you want to run it with your own data, just replace the following files with your owns:\r\n* `janus/data/nodes.csv`\r\n* `janus/data/links.csv`\r\n*  `logstash/data/transactions.csv`\r\n\r\nData directories for Elasticsearch and ScyllaDB will be mounted in the `PROJECT_HOME/data`.\r\n\r\nThe different services in the `docker-compose` setup need to be started in a specific order. [`wait-for-it.sh`](https://github.com/vishnubob/wait-for-it) mechanisms (or alike) could be implemented in the future to allow running all the services in a one line command like `docker-compose --build up`.\r\n\r\n#### Steps\r\n1. Clone the repository\r\n```\r\n\u003e git clone  git@github.com:entrepreneur-interet-general/graph-explorer.git\r\n\u003e cd graph-explorer\r\n```\r\n2. Download and build Docker images\r\n```\r\n\u003e docker-compose build\r\n```\r\n3. Create data directories\r\n```\r\nmkdir -p data/elasticsearch data/scylla\r\nchown -R 1000:1000 data/elasticsearch\r\n```\r\n4. Start Elasticsearch\r\n```\r\n\u003e docker-compose up -d elasticsearch\r\n```\r\n5. Wait for Elasticsearch to be available on port `9200`\r\n```\r\n\u003e curl localhost:9200/_cat/health\r\ndocker-cluster yellow 1 1 6 6 0 0 6 0 - 50.0%\r\n```\r\n6. Start ScyllaDB\r\n```\r\n\u003e docker-compose up -d scylladb\r\n```\r\n7. Wait for ScyllaDB to be available\r\n```\r\n\u003e docker-compose exec scylladb nodetool status\r\n--  Address     Load       Tokens       Owns    Host ID                               Rack\r\nUN  172.22.0.3  1.07 MB    256          ?       c961595a-ee52-4f94-baf3-74cdc5058af6  rack1\r\n```\r\n8. Start JanusGraph\r\n```\r\n\u003e docker-compose up -d janus\r\n```\r\nIf a previous janus container has not been shut down correctly, you might get the following error:\r\n```\r\nA JanusGraph graph with the same instance id [*] is already open. Might required forced shutdown.\r\n```\r\nIn this case you can run a cleanup script and then restart the `janus` container.\r\n```\r\n\u003e docker-compose run --no-deps janus bin/gremlin.sh -e scripts/clean.groovy\r\n```\r\n9.  Wait for JanusGraph to be available on port `8182`\r\n```\r\n\u003e curl -XPOST -d '{\"gremlin\" : \"1+1\" }' localhost:8182\r\n{\"result\":{\"data\":[2],\"meta\":{}}}\r\n```\r\n10. Create the graph schema and load nodes and edges into JanusGraph (do it only the first time or after deleting the data directory)\r\n```\r\n\u003e docker-compose exec janus bin/gremlin.sh -e scripts/create_schema.groovy\r\n\u003e docker-compose exec janus bin/gremlin.sh -e scripts/load_data.groovy\r\n```\r\n11. Checks that nodes and edges have been loaded\r\n```\r\n\u003e curl -XPOST -d '{\"gremlin\" : \"g.V().count()\" }' localhost:8182\r\n{\"result\":{\"data\":[1606],\"meta\":{}}}\r\n\r\n\u003e curl -XPOST -d '{\"gremlin\" : \"g.E().count()\" }' localhost:8182\r\n{\"result\":{\"data\":[2156],\"meta\":{}}}\r\n```\r\n12. Load raw transactions into Elasticsearch (do it only the first time or after deleting the data directoy)\r\n```\r\n\u003e docker-compose up -d logstash\r\n```\r\n13. Check that transactions have been loaded in the `transactions` index\r\n```\r\n\u003e curl localhost:9200/transactions/doc/_count\r\n{\"count\":2156,\"_shards\":{\"total\":5,\"successful\":5,\"skipped\":0,\"failed\":0}}\r\n```\r\n14. Start the Flask server with Gunicorn\r\n```\r\ndocker-compose up -d app\r\n```\r\n* Visit [http://localhost:5000](http://localhost:5000)\r\n\r\n## Development\r\n\r\n### Run the tests\r\n\r\n* Unit tests\r\n```\r\nmake test_unit\r\n```\r\n\r\n* Integration tests requiring ScyllaDB, Elasticsearch and Janus to be up and running with data\r\nloaded from steps 10 and 12\r\n```\r\nmake test_integration\r\n```\r\n\r\n### Flask server\r\n* Start `Elasticsearch`, `ScyllaDB` and `JanusGraph` with `docker-compose ` as described in the installation steps.\r\n* Create a vitualenv for this project with `Python` version 3.6.0 or higher.\r\n* Install the dependencies\r\n```\r\npip install -r requirements.txt\r\n```\r\n* Run `Flask` in development mode. This will reload the server on code changes.\r\n```\r\nFLASK_APP=api.app.py FLASK_ENV='development' CONFIG='api.config.Development' python -m flask run\r\n```\r\n\r\n### Frontend\r\n* Make sure to use a version of `nodejs` higher than 8.0.0. We recomand using [`nvm`](https://github.com/creationix/nvm).\r\n* Install the dependencies\r\n```\r\nnpm install\r\n```\r\n* The project is built using webpack into the file `local/build.js`.\r\n* Build the source once with:\r\n```\r\nnpm run build\r\n```\r\n* Or watch for file changes with:\r\n```\r\nnpm run watch\r\n```\r\n\r\nVisit [http://localhost:5000](http://localhost:5000) and start coding!\r\n\r\n## Credits\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n  \u003ca href=\"https://entrepreneur-interet-general.etalab.gouv.fr/\"\u003e\r\n    \u003cimg src=\"docs/img/logo-eig.png\" width=\"500px\"\u003e\r\n  \u003c/a\u003e\r\n\u003c/div\u003e\r\n\r\nPlease visit the [Hopkins mission](https://entrepreneur-interet-general.etalab.gouv.fr/defis/2018/hopkins.html) page for more information.\r\n\r\n## License\r\n\r\nMIT License\r\n\r\nCopyright (c) 2018 Ministère de l'Action et des Comptes Publics, Benoît Guigal, Paul Boosz\r\n\r\nPermission is hereby granted, free of charge, to any person obtaining a copy\r\nof this software and associated documentation files (the \"Software\"), to deal\r\nin the Software without restriction, including without limitation the rights\r\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\r\ncopies of the Software, and to permit persons to whom the Software is\r\nfurnished to do so, subject to the following conditions:\r\n\r\nThe above copyright notice and this permission notice shall be included in all\r\ncopies or substantial portions of the Software.\r\n\r\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\r\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\r\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\r\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\r\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\r\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r\nSOFTWARE.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fentrepreneur-interet-general%2Fgraph-explorer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fentrepreneur-interet-general%2Fgraph-explorer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fentrepreneur-interet-general%2Fgraph-explorer/lists"}