Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/opsdis/nodegraph-provider
The nodegraph-provider works with the Grafana datasource "Node Graph API Datasource Plugin"
https://github.com/opsdis/nodegraph-provider
grafana node-graph
Last synced: about 19 hours ago
JSON representation
The nodegraph-provider works with the Grafana datasource "Node Graph API Datasource Plugin"
- Host: GitHub
- URL: https://github.com/opsdis/nodegraph-provider
- Owner: opsdis
- License: gpl-3.0
- Created: 2022-01-30T10:44:53.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2024-01-27T11:30:07.000Z (10 months ago)
- Last Synced: 2024-06-21T03:18:33.118Z (5 months ago)
- Topics: grafana, node-graph
- Language: Go
- Homepage:
- Size: 501 KB
- Stars: 5
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: COPYING
Awesome Lists containing this project
README
nodegraph-provider
----------------
# Overview
The nodegraph-provider works with the Grafana datasource "Node Graph API Datasource Plugin",
https://github.com/exaco/nodegraph-api-plugin.
The nodegraph-provider use a RedisGraph, https://github.com/RedisGraph/RedisGraph, to keep a graph model
of what to expose to the Node Graph API datasource.
To create a model the nodegraph-provider expose api endpoints to create and update nodes and edges.
The attributes that can be used is limited to the attributes of the Grafana Node Graph panel plugin,
https://grafana.com/docs/grafana/latest/visualizations/node-graph/.
# Configuration
For available configuration check out the `config.yaml`. All configuration can be overridden by environment variables
except the `graph_schemas`.
Prefix all configuration varaibles with `NODEGRAPH_PROVIDER_` and replace `.` with `_`, e.g. set the redis server:
export NODEGRAPH_PROVIDER_REDIS_HOST=my_redis_host
# Graph schema model
The graph model must be compatible to the data model for nodes and edges in the Node Graph panel.
The attributes allowed is specified at https://grafana.com/docs/grafana/latest/visualizations/node-graph/#data-api.
In the nodegraph-provider each instance schema of a model is described in the configuration file, e.g.
```yaml
graph_schemas:
micro:
node_fields:
- field_name: "id"
type: "string"
- field_name: "title"
type: "string"
- field_name: "subTitle"
type: "string"
- field_name: "mainStat"
type: "string"
displayName: "CPU"
- field_name: "secondaryStat"
type: "number"
- field_name: "arc__failed"
type: "number"
color: "red"
- field_name: "arc__passed"
type: "number"
color: "green"
- field_name: "detail__role"
type: "string"
displayName: "Role"
edge_fields:
- field_name: "id"
type: "string"
- field_name: "source"
type: "string"
- field_name: "target"
type: "string"
- field_name: "mainStat"
type: "number"
- field_name: "secondaryStat"
type: "number"
- field_name: "detail__traffic"
type: "string"
```
The `graph_schemas` can have multiple named schema definitions. Above is a schema called `micro`.
The name is used as a key in all API accesses to support multiple instances. E.g. when setting up the URL in the
Node Graph API Datasource Plugin we define it as part of the path, `http://localhost:9393/micro`.
This enables nodegraph-provider to support multiple data sources in a single instance.
The schema defines the different fields and the type, string or number, that are allowed for the specific schema.
# API endpoints
All endpoint expects the header "Content-Type" set to "application/json"
## Data source API
GET /{graph_schema}/api/graph/data?query=
The above query parameter is optional, If empty or not set all nodes and edges are returned.
If set, it must be a valid Cypher statement, https://neo4j.com/docs/cypher-manual/current/clauses/where/,
but with some limitation documented in https://neo4j.com/docs/cypher-manual/current/clauses/where/.
For example the Redis Cypher do not support regular expression.
> The `WHERE` should not be part of the query string, and the string must be urlencoded.
The WHERE clause can only operate where source nodes, edges and target nodes are returned, e.g.
the statement `edge.mainStat > 0` will be in encoded url:
query=edge.mainStat%20%3E%200
The source nodes are referred to as `source`, target nodes are referred to as `target` and edges are
referred to as `edge`
GET /{graph_schema}/api/graph/fields
GET /{graph_schema}/api/health
## Mange nodes and edges
POST /api/nodes/{graph_schema}
GET /api/nodes/{graph_schema}/{id}
PUT /api/nodes/{graph_schema}/{id}
DELETE /api/nodes/{graph_schema}/{id}
POST /api/edges/{graph_schema}
GET /api/edges/{graph_schema}/{source_id}/{target_id}
PUT /api/edges/{graph_schema}/{source_id}/{target_id}
DELETE /api/edges/{graph_schema}/{source_id}/{target_id}
POST operations expect a json body. The content should only include the field names in the graph schema.
Node example:
```json
{
"id": "lb-01",
"title": "lb01",
"subTitle": "instance:#01",
"arc__failed": 0.0,
"arc__passed": 0.0,
"detail__role": "load",
"mainStat": 0.0,
"secondaryStat": 0.0
}
```
PUT takes query parameters with the fields to be updated and id as a path parameter.
DELETE and GET do not have any query parameters.
```bash
curl -s -i -H "Content-Type: application/json" -X PUT "localhost:9393/api/nodes/micro/lb-01?arc__failed=0.1?arc__passed=0.9"
```
## Manage a complete graph
The api endpoints will operate on a complete graph. The POST will first
delete before create. For a client that have the full "picture" of the graph
model, this is the most effective endpoint to use.
POST /api/graphs/{graph_schema}
DELETE /api/graphs/{graph_schema}
> The graph endpoints in singulars are deprecated:
>
> POST /api/graphs/{graph_schema}
>
> DELETE /api/graphs/{graph_schema}
The POST endpoint require a body of a list of nodes and edges, e.g.
```json
{
"nodes": [
{
"id": "lb-1",
"title": "lb",
"subTitle": "instance:#01",
"detail__role": "load",
"arc__failed": 0,
"arc__passed": 1,
"mainStat": 0,
"secondaryStat": 0
},
....
],
"edges": [
{
"source": "lb-1",
"target": "cust-svc-1",
"mainStat": 0,
"secondaryStat": 0
},
....
]
}
```
Please see the `examples/graph.json` and `examples/setup_graph.sh` for a
complete example.
## Internal metrics
Internal metrics are exposed on endpoint `/metrics`. Go metrics and nodegraph-provider specific metrics are exposed
on the endpoint in Prometheus format.
## Deprecated API
The following api are deprecated:
POST /api/controller/{graph_schema}/delete-all
## Return status
- 200
- Successful - PUT, DELETE, GET
- 201
- Create successful - POST
- 400
- Not a valid json - POST
- Not a valid field name - POST, PUT
- 404
- Object do not exist - PUT, DELETE, GET
- 409
- Object already exists - POST
- 415
- Invalid value for header "Content-Type"
# Get started
Install Grafana and datasource https://github.com/exaco/nodegraph-api-plugin.
Start grafana to allow unsigned plugins
export GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=hamedkarbasi93-nodegraphapi-datasource ; ./bin/grafana-serve
Start redis with module RedisGraph. Simple way just use docker.
docker run -p 6379:6379 redislabs/redismod
Start nodegraph-provider, configuration file is default `config.yml`
go build -o build/nodegraph-provider *.go
./build/nodegraph-provider
To see all options
./build/nodegraph-provider -h
Create a data source in Grafana with the nodegraph-api-plugin and set url to `http://localhost:9393/micro`.
Name it to `Micro`.
Create a dashboard and select the "Node Graph" plugin. Select the data source `Micro`.
Load the simple graph model by create nodes and edges:
./examples/setup_test.sh
Or run the example to create a complete graph:
./examples/setup_graph.sh
In Grafana you should now see this graph.
Add a new node with id `cust-svc-2`
curl -s -i -H "Content-Type: application/json" -X POST localhost:9393/api/nodes/micro -d @examples/node_create.json
Create an edge between `lb-1` and `cust-svc-3`
curl -s -i -H "Content-Type: application/json" -X POST localhost:9393/api/edges/micro -d @examples/edge_create.json
Update metrics on `lb-1`
curl -s -i -H "Content-Type: application/json" -X PUT "localhost:9393/api/nodes/micro/book-svc-1?mainStat=$RANDOM&secondaryStat=$RANDOM&arc__failed=0.1&arc__passed=0.9"
Update metrics on edge between `lb-1` to `cust-svc-1
curl -s -i -H "Content-Type: application/json" -X PUT "localhost:9393/api/edges/micro/lb-1/cust-svc-1?mainStat=$RANDOM&secondaryStat=$RANDOM"
You should now see something like this.
Delete the whole graph
curl -s -i -H "Content-Type: application/json" -X DELETE "localhost:9393/api/graph/micro"
# Build docker
Use the Dockerfile in the root directory of the project
docker build --tag nodegraph-provider .
To run the image a config file must be mounted
docker run -p 9393:9393 -v $(pwd)/config_tempo.yaml:/app/config.yaml nodegraph-provider
Environment variables can be set to override defaults and config file
docker run -p 9393:9393 -v $(pwd)/config_tempo.yaml:/app/config.yaml -e NODEGRAPH_PROVIDER_PORT=9393 -e NODEGRAPH_PROVIDER_REDIS_HOST=localhost -e NODEGRAPH_PROVIDER_REDIS_PORT=6379 nodegraph-provider
# Demo examples
Checkout the [tempo_trace_aggregation](https://github.com/opsdis/tempo_trace_aggregation) project
where nodegraph-provider is used to create a "dynamic service map" based on
aggregated traces stored in [Grafana Tempo](https://github.com/grafana/tempo).