{"id":37100512,"url":"https://github.com/automixer/gobi","last_synced_at":"2026-01-14T12:14:31.356Z","repository":{"id":144424322,"uuid":"490864717","full_name":"automixer/gobi","owner":"automixer","description":"A simple network flow monitoring application","archived":false,"fork":false,"pushed_at":"2025-01-08T17:55:46.000Z","size":331,"stargazers_count":18,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-08T18:49:13.976Z","etag":null,"topics":["docker-compose","go","grafana","ipfix","kubernetes","netflow","network-monitoring","nms","prometheus"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/automixer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-05-10T21:24:09.000Z","updated_at":"2025-01-08T17:55:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"c4bda2f0-79b1-478a-95ce-d2e391a353f3","html_url":"https://github.com/automixer/gobi","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/automixer/gobi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automixer%2Fgobi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automixer%2Fgobi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automixer%2Fgobi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automixer%2Fgobi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/automixer","download_url":"https://codeload.github.com/automixer/gobi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automixer%2Fgobi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28419748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","go","grafana","ipfix","kubernetes","netflow","network-monitoring","nms","prometheus"],"created_at":"2026-01-14T12:14:30.793Z","updated_at":"2026-01-14T12:14:31.326Z","avatar_url":"https://github.com/automixer.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gobi\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/automixer/gobi.svg)](https://pkg.go.dev/github.com/automixer/gobi)\n[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)\n[![Go Report Card](https://goreportcard.com/badge/github.com/automixer/gobi)](https://goreportcard.com/report/github.com/automixer/gobi)\n\nThis application allows you to visualize the highly complex flow Information data exported by network devices, in an easy and understandable graphical way. \n\nIt takes a decoded flow data stream from [GoFlow2](https://github.com/netsampler/goflow2) as input and exports it to [Prometheus Server](https://github.com/prometheus/prometheus) to be stored. Then, data can be queried and graphically represented using tools such as [Grafana.](https://github.com/grafana/grafana)\n\n## Project Goals\n\nGobi aims to be a fast, lightweight and easy to use tool for network flows analysis. It favours speed over absolute accuracy. With Gobi, flows are pre-processed and aggregated before getting stored to the DB. The subsequent data visualization operations are therefore greatly improved, allowing short response times even over large analysis intervals.\n\n## Get Started\n\nThe [examples](examples) folder contains two ready to deploy Gobi setups, one for docker-compose and the other for kubernetes environment. Pick one and send to it a NetFlow/IPFIX stream to get a sample of what Gobi can do. [Here](examples/README.md) you can find detailed installation instructions.\n\n![Gobi Example DashBoard](png/screenshot.png)\n\n## Setup a basic flow pipeline\n\nTo set up a basic flow pipeline, download Gobi and GoFlow2 to your local machine and create a `gobi.yml` config file like this template:\n\n\u003e ```\n\u003e global:\n\u003e   metricsaddr: :9310\n\u003e \n\u003e producer:\n\u003e   input: stdin\n\u003e \n\u003e promexporters:\n\u003e   - metricsname: example\n\u003e     labelset: [\"SamplerAddress\"]\n\u003e ```\n\nThen start the pipeline with the following command:\n\n```\n$ ./goflow2 -transport.file.sep= -format=pb -format.protobuf.fixedlen=true | ./gobi -f gobi.yml\n```\n\n The pipeline is now ready to be scraped by Prometheus server at the IP address of your local machine. A full config keys list supported by `gobi.yml` is available [here](./doc/CONFIG-GUIDE.md). The provided [examples](./examples/README.md) are a good starting point for Gobi deployments.\n\n## Application design\n\nGobi is modular. The `producer` receives input flow data, converts it in go structures, enriches it with extended infos and outputs to a set of registered `exporter` instances. Each `exporter` can then shape its data as needed and output it in the the desired format. The current Gobi implementation features a GoFlow2 `producer` and a Prometheus `exporter`.\n\n### The GoFlow2 producer\n\nIt takes its input from stdin or a named pipe. The expected format is GPB, as defined by GoFlow2 proto [file](https://github.com/netsampler/goflow2/blob/v1.1.0/pb/flow.proto) *(not all fields are currently exported)*. Each flow is then enriched with data coming from these popular DBs:\n\n* MaxMind [GeoLite2 Country](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en).\n* MaxMind [GeoLite2 ASN](https://dev.maxmind.com/geoip/docs/databases/asn?lang=en).\n* [TheDiveO](https://github.com/thediveo/netdb) netdb.\n\nFinally, a copy of this enriched flow is sent to each registered exporter for further processing.\n\n### The Prometheus exporter\n\nFlows coming from `producer` are initially stored in a memory buffer, waiting for a Prometheus server scrape event. When scraping happens, `promexporter` checks them out and process flows it as follows:\n\n* Aggregates over a user defined field (labels) set.\n* Checks if those aggregates are compliant with a minimum user defined data rate over the scrape interval. If not, they are removed from the table and accumulated in separate \"untracked\" counters.\n* Aggregates are then checked for aging. The expired ones are evicted from the table.\n* Finally, the remaining aggregates and the untracked counters are sent to the Prometheus server.\n\nThere is a `MaxScrapeInterval` user configurable timer that flushes the buffer if no scrapes occurs.\n\nA key aspect of this architecture is that Prometheus finally gives the clock to the resampling process, with its scrape interval. This allows data accuracy to be kept to acceptable levels while enjoying the simplicity and performance of PromQL.\n\n## License\n\nLicensed under BSD 3-Clause license. See [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautomixer%2Fgobi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautomixer%2Fgobi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautomixer%2Fgobi/lists"}