{"id":31853590,"url":"https://github.com/o/gaia","last_synced_at":"2025-10-12T13:31:51.090Z","repository":{"id":23092389,"uuid":"26446354","full_name":"o/gaia","owner":"o","description":"Successor of simmetrica - written in Dropwizard framework","archived":false,"fork":false,"pushed_at":"2014-11-13T08:20:32.000Z","size":224,"stargazers_count":20,"open_issues_count":1,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-06T00:01:56.891Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/o.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-11-10T17:06:43.000Z","updated_at":"2019-04-07T21:12:10.000Z","dependencies_parsed_at":"2022-08-21T10:40:08.960Z","dependency_job_id":null,"html_url":"https://github.com/o/gaia","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/o/gaia","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/o%2Fgaia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/o%2Fgaia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/o%2Fgaia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/o%2Fgaia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/o","download_url":"https://codeload.github.com/o/gaia/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/o%2Fgaia/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279011443,"owners_count":26084947,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-12T13:30:31.662Z","updated_at":"2025-10-12T13:31:51.082Z","avatar_url":"https://github.com/o.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"#Gaia\n\nGaia is an open-source RESTful application for collecting and aggregating event metrics in timeseries manner. It shares same philosophy with [Simmetrica](https://github.com/o/simmetrica), but built for concurrency and scalability. Uses [Dropwizard](http://dropwizard.io) and [Redis](http://redis.io) under the hood.\n\n###Prerequisites\n\n* Java JDK 1.7\n* Redis 2.6 or higher\n* Maven (For building from source)\n\n###Running\n\nGet the latest source code from Github.\n\n```\n$ git clone https://github.com/o/gaia.git\n```\n\nFor building a runnable \"fat\" JAR file, run package goal of maven in the root directory of project.\n\n```\n$ mvn package\n```\n\nAfter build process, JAR file will be created at `target` directory. You can run Gaia with the following command:\n\n```\n$ java -jar target/gaia-0.4.0.jar server config.yml\n```\n\nIf you press `^C`, the application will shut down gracefully.\n\n**You need to run `redis-server` before pushing events and querying stored data.**\n\n###Configuration\n\nGaia uses Redis for storing data. If you need to configure default server and port address of Redis instance, you can set it from `config.yml` which located in the root directory.\n\n###Talking to Gaia\n\nAfter running Gaia, it exposes RESTful API over HTTP with JSON output for pushing and querying data. It can be accessible from `http://127.0.0.1:8000`.\n\n####How to push events\n\nFor pushing a new event, we need to make an HTTP POST request with a JSON payload including the default parameters.\n\n```\nPOST /events\n{\n  \"name\": \"\u003cString\u003e\",\n  \"increment\": \"Optional\u003cLong\u003e\",\n  \"timestamp\": \"Optional\u003cLong\u003e\"\n}\n```\n\n**Required parameters**\n\n`name`, which is canonical name of your input data. You will also use this for querying event data. Event name should consist of alphanumeric characters and dash. (Ex: `view-userprofile-890`)\n\n**Optional parameters**\n\n`increment`, this argument is useful for overriding event count for submitting multiple events in single operation. (Defaults to 1)\n\n`timestamp`, if you need irregular updates, this argument lets you specify when event occurs. (Defaults to current Unix timestamp)\n\nSimply, if you want to send only one event related to current time, specifying `name` parameter is good enough. If everything went smoothly `201 Created` response with empty body is returned. If something went wrong (Ex: connection interruption with Redis instance) `500 Internal Server Error` response is returned.\n\nExample:\n\n```\n$ curl -i -X POST -H 'Content-Type: application/json' -d '{\"name\": \"foo\"}' http://localhost:8080/events\n\nHTTP/1.1 201 Created\nDate: Tue, 11 Nov 2014 14:21:12 GMT\nContent-Type: application/json\nContent-Length: 0\n```\n\nIf there was a validation error with JSON payload an error response with `400 Bad Request` for `increment` and `now` parameters, `422 Unprocessable Entity` for `event` parameter will be returned with error messages. If you forget to add `Content-Type: application/json` header in request `415 Unsupported Media Type` will be returned.\n\n\n####How to query events\n\nFor querying events from Gaia, we simply make an HTTP GET request and specify the parameters about event name, data granularity and time range.\n\n```\nGET /events/\u003cString\u003e?start=\u003cLong\u003e\u0026end=\u003cLong\u003e\u0026resolution=\u003cString\u003e\n```\n\n**All parameters are mandatory**\n\n`event`, as you guessed, we already used this value for feeding our data.\n\n`start` and `end` parameters take Unix timestamp for specifying interval of time-series.\n\n`resolution` is used for defining the resolution / granularity of data. Possible values are `min`, `5min`, `15min`, `hour`, `day`, `week`, `month` and `year`.\n\nExample:\n\n```\n$ curl -sS -X GET 'http://localhost:8080/events/foo?start=1415697030\u0026end=1415718630\u0026resolution=hour'\n\n[\n    {\n        \"count\": 31356,\n        \"timestamp\": 1415696400\n    },\n    {\n        \"count\": 28763,\n        \"timestamp\": 1415700000\n    },\n    {\n        \"count\": 38561,\n        \"timestamp\": 1415703600\n    },\n    {\n        \"count\": 44677,\n        \"timestamp\": 1415707200\n    },\n    {\n        \"count\": 54810,\n        \"timestamp\": 1415710800\n    },\n    {\n        \"count\": 52345,\n        \"timestamp\": 1415714400\n    },\n    {\n        \"count\": 49780,\n        \"timestamp\": 1415718000\n    }\n]\n\n```\n\n###Events naming design\n\nIt's completely up to your **querying strategy**, keep it simple, short and meaningful. Don't hesitate to push multiple events for different querying requirements.\n\nHere are some examples:\n\nScenario: How many times a product page displayed?\n\nSchema: `\u003cgroup\u003e-\u003cobject\u003e-\u003cidentifier\u003e`\n\nExample: `view-product-324569`\n\n---\n\nScenario: How many times a product page displayed from desktop **or** mobile?\n\nSchema: `\u003cgroup\u003e-\u003cobject\u003e-\u003cidentifier\u003e-\u003ctag\u003e`\n\nExample: `view-product-324569-desktop`, `view-product-324569-mobile`\n\n---\n\nScenario: How many times a product page displayed from desktop **and / or** mobile?\n\nSchema: `\u003cgroup\u003e-\u003cobject\u003e-\u003cidentifier\u003e-\u003ctag\u003e`\n\nExample: `view-product-324569-desktop`, `view-product-324569-mobile`, `view-product-324569-total`\n\n---\n\nScenario: How many times a product listed in search results?\n\nSchema: `\u003cgroup\u003e-\u003cobject\u003e-\u003cidentifier\u003e`\n\nExample: `listingview-product-324569`\n\n###Internals\n\nGaia keeps event data in hashes. Hashes are very [memory efficient](http://instagram-engineering.tumblr.com/post/12202313862/storing-hundreds-of-millions-of-simple-key-value-pairs) and [plays well with CPU](http://redis.io/topics/memory-optimization). Also event key lookups and querying data is faster.\n\nAll keys about events keeping under `gaia:` keyspace in redis. When you push a new event to Gaia, it sends following commands to Redis.\n\n```\n\"HINCRBY\" \"gaia:foo:min\" \"1415809680\" \"1\"\n\"HINCRBY\" \"gaia:foo:15min\" \"1415808900\" \"1\"\n\"HINCRBY\" \"gaia:foo:month\" \"1415232000\" \"1\"\n\"HINCRBY\" \"gaia:foo:5min\" \"1415809500\" \"1\"\n\"HINCRBY\" \"gaia:foo:year\" \"1387584000\" \"1\"\n\"HINCRBY\" \"gaia:foo:day\" \"1415750400\" \"1\"\n\"HINCRBY\" \"gaia:foo:hour\" \"1415808000\" \"1\"\n\"HINCRBY\" \"gaia:foo:week\" \"1415232000\" \"1\"\n```\n\nThis operation is enclosed in a pipeline for sending multiple commands in a single step. Using pipelining hugely improves performance of push operation.\n\nGaia supports different resolutions from minute to year and keeps all values without expiring old values. So, you can query values of last 2 years in a minute precision.\n\nFor the simplicity, Redis doesn't allow to expire keys of hashes. Manual deletion of hash keys for clearing old values is very expensive operation if you have millions of events, but it's in the road map.\n\nGaia does not support data source types like `GAUGE` and `COUNTER` . Naturally it does not use fixed sized databases like [RRDTool](http://oss.oetiker.ch/rrdtool/) or [Whisper](http://graphite.readthedocs.org/en/latest/whisper.html).\n\n###Deployment and operational tips\n\n####Health checks\n\nThanks to Dropwizard, Gaia comes with built-in health check for verifying thread deadlocks and connectivity to Redis instance.\n\n```\n$ curl 'http://localhost:8081/healthcheck'\n\n{\"deadlocks\":{\"healthy\":true},\"jedis-pool\":{\"healthy\":true}}\n```\n\nIf all health checks report success, a `200 OK` is returned. If any of them fails, a `500 Internal Server Error` is returned with the error messages.\n\n####Running behind Nginx and Haproxy\n\nHere is the example Nginx configuration:\n\n```\nserver {\n    listen 80;\n\n    server_name metrics.domain.com;\n\n    location / {\n        proxy_pass http://localhost:8080;\n    }\n\n    location /ping {\n        proxy_pass http://localhost:8081;\n        root /ping;\n    }\n\n}\n```\n\nTo deny access to 8080 and 8081 (used for administrative purposes) ports run following rules:\n\n```\niptables -A INPUT -p tcp -i eth0 --dport 8080 -j REJECT --reject-with tcp-reset\niptables -A INPUT -p tcp -i eth0 --dport 8081 -j REJECT --reject-with tcp-reset\n```\n\nIf you're running several Gaia instances behind Haproxy, you can enable health checks with the configuration like this:\n\n```\nfrontend http\n    bind :80\n\n    ...\n    default_backend gaia_cluster\n\nbackend gaia_cluster\n    option httpchk GET /ping\n    http-check expect string pong\n\n    ...\n    server metrics04 metrics04.domain.com:80 check\n```\n\n####Running with supervisord\n\n[Supervisor](http://supervisord.org/) allows users to monitor and control a number of process on UNIX-like operating systems. Installation instructions can be found in [here](http://supervisord.org/installing.html#installing-a-distribution-package).\n\nTo run Gaia as a daemon with supervisord, add the following lines to supervisord configuration.\n\n```\n[program:gaia]\ncommand=/usr/bin/java -jar /path/to/gaia.jar server /path/to/config.yml\n```\n\nThen run following commands:\n\n```\nsupervisorctl reread\nsupervisorctl update\n```\n\nFor checking status of Gaia daemon, run `supervisorctl status`.\n\n```\ngaia                             RUNNING    pid 10636, uptime 39 days, 6:15:15\n```\n\n###Tests\n\nYou can run tests with `mvn test`.\n\n###Contributing\n\nDon't hesitate to make performance optimization proposals, adding new tests or helpful tricks to documentation. If you have any problems with running Gaia please open an issue from Issues page.\n\n###License\n\nCopyright (c) 2014 Osman Ungur\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is furnished\nto do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fo%2Fgaia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fo%2Fgaia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fo%2Fgaia/lists"}