Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/claranet/aws-inventory-graph
Explore your AWS platform with, Dgraph, a graph database.
https://github.com/claranet/aws-inventory-graph
aws dgraph golang graphdb inventory
Last synced: 5 days ago
JSON representation
Explore your AWS platform with, Dgraph, a graph database.
- Host: GitHub
- URL: https://github.com/claranet/aws-inventory-graph
- Owner: claranet
- License: apache-2.0
- Created: 2019-12-02T19:42:00.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2020-01-02T09:36:16.000Z (almost 5 years ago)
- Last Synced: 2024-06-20T05:03:54.974Z (5 months ago)
- Topics: aws, dgraph, golang, graphdb, inventory
- Language: Go
- Homepage:
- Size: 281 KB
- Stars: 63
- Watchers: 8
- Forks: 10
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: securitygroups.go
Awesome Lists containing this project
README
# Description
   
Explore your AWS platform with, [`Dgraph`](https://dgraph.io/), a graph database.
Thanks to Go and its goroutines, we can insert thousand of ressources in few seconds.
## Prerequisites
- [docker](https://docs.docker.com/)
- [docker-compose](https://docs.docker.com/compose/install/)
- [Go](https://golang.org/) (for build only)
## Install
Download and move to `/usr/local/bin/` a binary from [release page](https://github.com/claranet/aws-inventory-graph/releases)
## Build
This project uses *go.mod*, so after cloning this repo, simply run :
```bash
go build && chmod +x ./aws-inventory-graph
```
or
```bash
GOBIN=/usr/local/bin/ go install && chmod +x /usr/local/bin/aws-inventory-graph
```
## Usage
### Start Dgraph server
```bash
make
```
or
```bash
make up
```
Access to WebUI (**dgraph-ratel**) : [http://localhost:8000](http://localhost:8000)
### Import ressources
Authentication is based on your `.aws/config` file.
```bash
Usage of aws-inventory-graph:
-dgraph string
Dgraph server (ip:port) (default "127.0.0.1:9080")
-drop
Drop all nodes and the schema
-list
List available ressource types
-no-schema
Disable the refresh schema at each run
-profile string
Profile from ~/.aws/config (default "default")
-region string
AWS Region (default "eu-west-1")
-type string
Get the schema for a type (only after importing some data)
```
Example :
```bash
aws-inventory-graph -region us-west-2 -profile xxxx
2019/11/29 17:35:58 Drop all previous data
2019/11/29 17:35:58 Add schema
2019/11/29 17:36:04 List ...
...
2019/11/29 17:36:05 Add ... Nodes
...
2019/11/29 17:36:08 Add ... Edges
...
```
### Get schema for a type
You can get all schemas for *types* and *predicates* in **dgraph-ratel** WebUI:
or with binary, in JSON format :
```bash
aws-inventory-graph -type Address | jq
{
"types": [
{
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "Service",
"type": "string"
},
{
"name": "Region",
"type": "string"
},
{
"name": "OwnerId",
"type": "string"
},
{
"name": "PrivateIpAddress",
"type": "string"
},
{
"name": "PublicIp",
"type": "string"
},
{
"name": "Domain",
"type": "string"
},
{
"name": "AllocationId",
"type": "string"
},
{
"name": "_Instance",
"type": "Instance"
}
],
"name": "Address"
}
]
}
```
**Predicates which are prefixed with a `_` are Edges, and they all have a reverse.**
## Stop and/or Remove Dgraph
Stop :
```bash
make stop
```
Remove :
```bash
make rm
```
## Available Ressources
Here the list of currently supported ressources :
- Address
- AutoScalingGroup
- AvailabilityZone
- CacheCluster
- CacheSubnetGroup
- Cidr
- DbCluster
- DbClusterParameterGroup
- DbInstance
- DbParameterGroup
- DbSubnetGroup
- Image
- Instance
- InstanceProfile
- KeyPair
- LaunchConfiguration
- LaunchTemplate
- LoadBalancer
- NatGateway
- OptionGroup
- SecurityGroup
- Snapshot
- Subnet
- TargetGroup
- Volume
- Vpc
- VpcPeeringConnection
**All Edges between the Nodes have reversed.**
:warning: AWS API is often messy, names are not consistent from one endpoint to another, we try to fix that and keep a global coherance. This is why some Predicates don't match exact names returned by API.
## Query examples
See [here](https://docs.dgraph.io/query-language/) to get more info about **Dgraph’s GraphQL+**.
### Get Elastic IPs + Instances + NatGateways
```bash
{
Address(func: type(Address)) @filter(has(_Instance) or has(_NatGateway)){
name dgraph.type PublicIp
Instance:_Instance {name dgraph.type InstanceId}
NatGateway: _NatGateway{name dgraph.type NatGatewayID}
}
}
```
### Get Classic LoadBalancers + AutoScalingGroups + Instances
```bash
{
LoadBalancer(func: type(LoadBalancer))@filter(eq(LoadBalancerType, classic)) @cascade{
name dgraph.type
AutoScaling:~_LoadBalancer {
name dgraph.type
Instance:_Instance{
name dgraph.type InstanceId
}
}
}
}
```
### Get Application LoadBalancers + TargetGroups + AutoScalingGroups + Instances
```bash
{
LoadBalancerV2(func: type(LoadBalancer))@filter(eq(LoadBalancerType, application))@cascade{
name dgraph.type
TargetGroup:~_LoadBalancer @filter(type(TargetGroup)){
name dgraph.type
AutoScalingGroup:~_TargetGroup @filter(type(AutoScalingGroup)){
name dgraph.type Instance:_Instance{
name dgraph.type InstanceId
}
}
}
}
}
```
### Get VpcPeeringConnections + Vpcs
```bash
{
VpcPeeringConnection(func: type(VpcPeeringConnection)){
name dgraph.type VpcPeeringConnectionId
AccepterVpc:_AccepterVpc {name dgraph.type VpcId}
RequesterVpc:_RequesterVpc {name dgraph.type VpcId}
}
}
```
### Get which Instances have access to which DbInstances
```bash
{
DbInstances(func: type(DbInstance))@cascade{
name dgraph.type
SecurityGroup:_SecurityGroup {
name dgraph.type GroupId
IngressSecurityGroup:_SecurityGroup @facets {
name dgraph.type
Instance:~_SecurityGroup @filter(type(Instance)){
name dgraph.type InstanceId
}
}
}
}
}
```
### Get the Cidr which are allowed for access to DbInstances
```bash
{
Rds(func: type(DbInstance))@cascade{
name dgraph.type
SecurityGroup:_SecurityGroup {
name dgraph.type GroupId
IngressCidr:_Cidr @facets {
name dgraph.type
}
}
}
}
```
### Get Instance opened worldwide + associated ports
```bash
{
OpenWorldCidr(func: eq(name, "0.0.0.0/0"))@cascade{
name dgraph.type
SecurityGroup:~_Cidr @filter(type(SecurityGroup)) @facets {
name dgraph.type GroupId
Instance:~_SecurityGroup @filter(type(Instance)){
name dgraph.type InstanceId
}
}
}
}
```
### Get which KeyPairs give access to which Instances
```bash
{
KeyPair(func: type(KeyPair))@cascade{
name dgraph.type
Instance:~_KeyName{
name dgraph.type InstanceId
}
}
}
```
### Get CacheClusters (Elasticache) with Instances which have access to them
```bash
{
Memcached(func: type(CacheCluster))@filter(eq(Engine, memcached))@cascade{
name dgraph.type
SecurityGroup:_SecurityGroup @facets {
name dgraph.type
IngressSecurityGroup:_SecurityGroup @facets {
name dgraph.type
Instance:~_SecurityGroup @filter(type(Instance)){
name dgraph.type InstanceId
}
}
}
}
Redis(func: type(CacheCluster))@filter(eq(Engine, redis))@cascade{
name dgraph.type
SecurityGroup:_SecurityGroup @facets {
name dgraph.type
IngressSecurityGroup:_SecurityGroup @facets {
name dgraph.type
Instance:~_SecurityGroup @filter(type(Instance)){
name dgraph.type InstanceId
}
}
}
}
}
```
### Get InstanceProfiles which are not used by Instances
```bash
{
InstanceProfile(func: type(InstanceProfile)) @filter(not has(~_InstanceProfile)) {
name dgraph.type
}
}
```
### Get Instances without backup (no linked Snapshot)
```bash
{
Instance(func: type(Instance)) @filter(eq(OwnerName, univadis-prod)) @cascade{
name dgraph.type
Volume:~_Instance @filter(not has(~_Volume) and type(Volume)){}
}
}
```
### Get the sum of Volumes by type
```bash
{
var(func: type(Volume)) @filter(eq(VolumeType, gp2)){
A as Size
}
var(func: type(Volume)) @filter(eq(VolumeType, standard)){
B as Size
}
Volume(){
gp2:sum(val(A))
standard:sum(val(B))
}
}
```
## Author
Thomas Labarussias ([@Issif](https://github.com/Issif))