Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/lynxkite/lynxkite
The complete graph data science platform
https://github.com/lynxkite/lynxkite
complex-networks data-science graph-algorithms graph-visualization hacktoberfest machine-learning
Last synced: about 2 months ago
JSON representation
The complete graph data science platform
- Host: GitHub
- URL: https://github.com/lynxkite/lynxkite
- Owner: lynxkite
- License: agpl-3.0
- Created: 2020-03-30T10:56:47.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-04-12T15:47:38.000Z (5 months ago)
- Last Synced: 2024-04-12T21:15:24.073Z (5 months ago)
- Topics: complex-networks, data-science, graph-algorithms, graph-visualization, hacktoberfest, machine-learning
- Language: Scala
- Homepage: https://lynxkite.com/
- Size: 37.3 MB
- Stars: 134
- Watchers: 8
- Forks: 13
- Open Issues: 104
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
- Awesome - LynxKite - LynxKite is a complete graph data science platform for very large graphs and other datasets. It seamlessly combines the benefits of a friendly graphical interface and a powerful Python API. (Multimedia / Image and pictures)
README
# LynxKite
[LynxKite](https://lynxkite.com/) is a complete graph data science platform for very large graphs
and other datasets.
It seamlessly combines the benefits of a friendly graphical interface and a powerful Python API.
- Hundreds of scalable **graph operations**, including graph metrics like PageRank, embeddedness,
and centrality, machine learning methods including
[GCNs](https://tkipf.github.io/graph-convolutional-networks/), graph segmentations like modular
clustering, and various transformation tools like aggregations on neighborhoods.
- The two main data types are **graphs and relational tables**. Switch back and forth between the
two as needed to describe complex logical flows. Run SQL on both.
- A **friendly web UI** for building powerful pipelines of operation boxes. Define your own custom
boxes to structure your logic.
- Tight integration with **Python** lets you implement custom transformations or create whole
workflows through a simple API.
- Integrates with the **Hadoop ecosystem**. Import and export from CSV, JSON, Parquet, ORC, JDBC,
Hive, or Neo4j.
- [Fully documented.](https://lynxkite.com/docs/latest)
- Proven in production on large clusters and real datasets.
- Fully configurable **graph visualizations** and **statistical plots**. Experimental 3D and
ray-traced graph renderings.
LynxKite is under active development. Check out our [Roadmap](https://lynxkite.com/roadmap) to see
what we have planned for future releases.
## Getting started
Quick try:
```
docker run --rm -p2200:2200 lynxkite/lynxkite
```
Setup with persistent data:
```
docker run \
-p 2200:2200 \
-v ~/lynxkite/meta:/metadata -v ~/lynxkite/data:/data \
-e KITE_MASTER_MEMORY_MB=1024 \
--name lynxkite lynxkite/lynxkite
```
## Contributing
If you find any bugs, have any questions, feature requests or comments, please
[file an issue](https://github.com/lynxkite/lynxkite/issues/new)
or email us at [email protected].
### Containerized build
If you build LynxKite with Earthly, you don't have to install anything on your
system except Earthly and get really reliable builds.
1. Install [**Earthly**](https://earthly.dev/get-earthly).
2. Run `earthly +run` to build and run LynxKite. See the `Earthfile` for other targets.
### Native build
You can install LynxKite's dependencies (Scala, Node.js, Go) locally with
[Conda](https://docs.conda.io/en/latest/miniconda.html).
Before the first build:
tools/git/setup.sh # Sets up pre-commit hooks.
conda env create --name lk --file conda-env.yml
conda activate lk
cp conf/kiterc_template ~/.kiterc
We use `make` for building the whole project.
make
LynxKite can be run as a fat jar started with `spark-submit`. See `run.sh` for an example of this.
## Tests
We have test suites for the different parts of the system:
- **Backend tests** are unit tests for the Scala code. They can also be executed with Sphynx as the
backend. If you run `make backend-test` it will do both. Or you can start `sbt` and run
`testOnly *SomethingTest` to run just one test. Run `./test_backend.sh -si` to start `sbt` with
Sphynx as the backend.
- **Frontend tests** use [Playwright](https://playwright.dev/) to simulate a user's actions
on the UI. `make frontend-test` will build everything, start a temporary LynxKite instance and run
the tests against that. If you already have a running run `npm test` in the `web`
directory. You can start up a dev server that proxies backend requests to LynxKite with
`npm start`.
- **Python API tests** are started with `make remote_api-test`. If you already have a running
LynxKite that is okay to test on, run `python/remote_api/test.sh`. This script can also run a
subset of the test suite: `python/remote_api/test.sh -p *something*`
## License
- [GNU Affero General Public License v3.0](https://github.com/lynxkite/lynxkite/blob/master/LICENSE)