{"id":13581702,"url":"https://github.com/Clymene-project/Clymene","last_synced_at":"2025-04-06T10:32:52.924Z","repository":{"id":41850742,"uuid":"361615522","full_name":"Clymene-project/Clymene","owner":"Clymene-project","description":"the Clymene is time-series data and Logs collection platform for distributed systems.","archived":false,"fork":false,"pushed_at":"2024-11-04T23:24:23.000Z","size":11341,"stargazers_count":133,"open_issues_count":5,"forks_count":21,"subscribers_count":5,"default_branch":"develop","last_synced_at":"2024-11-05T21:45:04.945Z","etag":null,"topics":["distributed-system","elasticsearch","exporter","grpc","metrics","monitoring","opentsdb","platform","prometheus","prometheus-agent","timeseries","tsdb"],"latest_commit_sha":null,"homepage":"https://clymene-project.github.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Clymene-project.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":["bourbonkk"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2021-04-26T04:20:44.000Z","updated_at":"2024-10-23T01:48:54.000Z","dependencies_parsed_at":"2024-01-15T01:09:12.886Z","dependency_job_id":null,"html_url":"https://github.com/Clymene-project/Clymene","commit_stats":{"total_commits":261,"total_committers":8,"mean_commits":32.625,"dds":0.5938697318007663,"last_synced_commit":"4e6c243fb18d7019a8a234b32afc9fb14535c9ae"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Clymene-project%2FClymene","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Clymene-project%2FClymene/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Clymene-project%2FClymene/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Clymene-project%2FClymene/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Clymene-project","download_url":"https://codeload.github.com/Clymene-project/Clymene/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247470480,"owners_count":20944146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-system","elasticsearch","exporter","grpc","metrics","monitoring","opentsdb","platform","prometheus","prometheus-agent","timeseries","tsdb"],"created_at":"2024-08-01T15:02:11.490Z","updated_at":"2025-04-06T10:32:47.890Z","avatar_url":"https://github.com/Clymene-project.png","language":"Go","funding_links":["https://github.com/sponsors/bourbonkk"],"categories":["Go"],"sub_categories":[],"readme":"# Clymene  \n\n\n\n[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/5491/badge)](https://bestpractices.coreinfrastructure.org/projects/5491) ![CodeQL](https://github.com/clymene-project/clymene/workflows/CodeQL/badge.svg) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) ![GitHub release (latest by date)](https://img.shields.io/github/v/release/clymene-project/clymene) [![Go Reference](https://pkg.go.dev/badge/github.com/Clymene-project/Clymene.svg)](https://pkg.go.dev/github.com/Clymene-project/Clymene)  \n![Go](https://img.shields.io/badge/go-%2300ADD8.svg?style=for-the-badge\u0026logo=go\u0026logoColor=white) ![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white) ![Kubernetes](https://img.shields.io/badge/kubernetes-%23326ce5.svg?style=for-the-badge\u0026logo=kubernetes\u0026logoColor=white) ![ElasticSearch](https://img.shields.io/badge/-ElasticSearch-005571?style=for-the-badge\u0026logo=elasticsearch) \u003cimg src=\"https://img.shields.io/badge/influxdb-%2322ADF6.svg?\u0026style=for-the-badge\u0026logo=influxdb\u0026logoColor=white\"/\u003e  \u003cimg src=\"https://img.shields.io/badge/prometheus-%23E6522C.svg?\u0026style=for-the-badge\u0026logo=prometheus\u0026logoColor=white\" /\u003e   \u003cimg src=\"https://img.shields.io/badge/OpenTSDB-green?style=for-the-badge\"\u003e \u003cimg src=\"https://img.shields.io/badge/cortex-blue?style=for-the-badge\"\u003e  \u003cimg src=\"https://img.shields.io/badge/tdengine-gray?style=for-the-badge\"\u003e \u003cimg src=\"https://img.shields.io/badge/Loki-yellow?style=for-the-badge\"\u003e \u003cimg src=\"https://img.shields.io/badge/druid-darkgray?style=for-the-badge\"\u003e\n\n\n\u003cimg align=\"right\" width=\"40%\" height=\"40%\" src=\"https://user-images.githubusercontent.com/25188468/148681479-3ddf237c-6e5d-49a1-a517-8b3bfa92f54e.png\" alt=\"clymene_logo\"\u003e\n\nThe Clymene is a time-series data and logs collection platform for distributed systems inspired\nby [Prometheus](https://prometheus.io)\nand [Jaeger](https://www.jaegertracing.io). Time-series data and logs from various environments can be collected and stored in\ndifferent types of databases. It can be configured in a variety of architectures. Choose the storage that users can use easily and build a monitoring system with dashboards that users can use easily. Also, Clymene Agent's time series data collection uses fewer resources than Prometheus' remote_write. \nVarious time series data analysis functions will be added.\n\n\n# Architecture  \n\nThe Clymene project is a platform for collecting time series data and log data. So there are two pipelines: timeseries and logs. All agents can collect data and store it directly in the database,\nOr, depending on the situation, you can use the Clymene components to configure a variety of architectures. It can be configured with an architecture that can be used with a lot of traffic. \nThe Clymene Project provides various storage types, so choose a storage type you are familiar with. And build great monitoring systems using familiar dashboards.  \n  \n\u003cimg src=\"https://user-images.githubusercontent.com/25188468/219957948-8d5a76aa-2701-4f2b-87a9-a098214f2820.png\" width=\"90%\" height=\"90%\" alt=\"architecture_v2.1.0\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/25188468/218608485-78ce6dca-f3bb-41d2-a708-dfee4ced57e8.png\" width=\"90%\" height=\"90%\" alt=\"architecture_v2.2.2\"\u003e  \n\n# Components\n\n## Pipeline 1. TimeSeries data collection   \n\n### Clymene Agent([Getting Started](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-agent/README.md))\n\n```dockerhub : bourbonkk/clymene-agent:v2.x.x```  \n```redhatQuay: quay.io/clymene/clymene-agent:v2.x.x```  \n![Docker Pulls](https://img.shields.io/docker/pulls/bourbonkk/clymene-agent.svg?maxAge=86400) [![Docker Repository on Quay](https://quay.io/repository/clymene/clymene-agent/status \"Docker Repository on Quay\")](https://quay.io/repository/clymene/clymene-agent)   \nThe Clymene-agent is service that collects time series data(does not use disks)\n\n1. Service Discovery\n    - [Prometheus's Service Discovery](https://docs.sysdig.com/en/docs/sysdig-monitor/integrations-for-sysdig-monitor/collect-prometheus-metrics/enable-prometheus-native-service-discovery/)\n      feature finds Metric collection endpoints.\n2. scrape time series data\n3. Time-series data transfer to gateway(gRPC) (Optional)\n4. Time-series data transfer to kafka (Optional)\n5. Time-series data insert to Database([supported DB](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-agent/README.md#Option-description-by-storage-type)) (Optional)\n\n### Clymene Ingester(Optional) ([Getting Started](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-ingester/README.md))\n\n```dockerhub : bourbonkk/clymene-ingester:v2.x.x```  \n```redhatQuay: quay.io/clymene/clymene-ingester:v2.x.x```  \n![Docker Pulls](https://img.shields.io/docker/pulls/bourbonkk/clymene-ingester.svg?maxAge=86400) [![Docker Repository on Quay](https://quay.io/repository/clymene/clymene-ingester/status \"Docker Repository on Quay\")](https://quay.io/repository/clymene/clymene-ingester)  \nThe Clymene ingester is an optional service responsible for insert time series data loaded on kafka into the database.\n\n1. Kafka message consume\n2. Time-series data insert to Database([supported DB](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-ingester/README.md#Option-description-by-storage-type)) (Optional)\n\n### Clymene Gateway(Optional) ([Getting Started](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-gateway/README.md))\n\n```dockerhub : bourbonkk/clymene-gateway:v2.x.x```  \n```redhatQuay: quay.io/clymene/clymene-gateway:v2.x.x```  \n![Docker Pulls](https://img.shields.io/docker/pulls/bourbonkk/clymene-gateway.svg?maxAge=86400) [![Docker Repository on Quay](https://quay.io/repository/clymene/clymene-gateway/status \"Docker Repository on Quay\")](https://quay.io/repository/clymene/clymene-gateway)  \nThe Clymene Gateway is an optional service that can receive metric data from the another component through gRPC or HTTP\ncommunication.\n\n1. gRPC, HTTP Service\n2. Time-series data insert to Database([supported DB](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-gateway/README.md#Option-description-by-storage-type)) (Optional)\n\n\n## Pipeline 2. Logs collection  \n\n### Clymene Promtail([Getting Started](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-promtail/README.md))\n\n```dockerhub : bourbonkk/clymene-promtail:v2.x.x```  \n```redhatQuay: quay.io/clymene/clymene-promtail:v2.x.x```  \n![Docker Pulls](https://img.shields.io/docker/pulls/bourbonkk/clymene-promtail.svg?maxAge=86400) [![Docker Repository on Quay](https://quay.io/repository/clymene/clymene-promtail/status \"Docker Repository on Quay\")](https://quay.io/repository/clymene/clymene-agent)   \nThe Clymene-promtail customized loki's log collection agent for the Clymene project.\n\n1. [Service Discovery](https://clymene-project.github.io/docs/service-discovery/promtail-config/)\n2. log collection\n3. log data transfer to gateway(gRPC or HTTP)\n4. log data transfer to kafka\n5. log data insert to Database([supported DB](https://github.com/Clymene-project/Clymene/blob/main/docs/clymene-promtail/README.md#Option-description-by-storage-type)) (Optional)\n\n### Promtail Ingester(Optional) ([Getting Started](https://github.com/Clymene-project/Clymene/blob/main/docs/promtail-ingester/README.md))\n\n```dockerhub : bourbonkk/promtail-ingester:v2.x.x```  \n```redhatQuay: quay.io/clymene/promtail-ingester:v2.x.x```  \n![Docker Pulls](https://img.shields.io/docker/pulls/bourbonkk/promtail-ingester.svg?maxAge=86400) [![Docker Repository on Quay](https://quay.io/repository/clymene/promtail-ingester/status \"Docker Repository on Quay\")](https://quay.io/repository/clymene/promtail-ingester)  \nPromtail ingester is an optional service responsible for insert log data loaded on kafka into the database.\n\n1. Kafka message consume\n2. Time-series data insert to Database([supported DB](https://github.com/Clymene-project/Clymene/blob/main/docs/promtail-ingester/README.md#Option-description-by-storage-type)) (Optional)\n\n### Promtail Gateway(Optional) ([Getting Started](https://github.com/Clymene-project/Clymene/blob/main/docs/promtail-gateway/README.md))\n\n```dockerhub : bourbonkk/promtail-gateway:v2.x.x```  \n```redhatQuay: quay.io/clymene/promtail-gateway:v2.x.x```  \n![Docker Pulls](https://img.shields.io/docker/pulls/bourbonkk/promtail-gateway.svg?maxAge=86400) [![Docker Repository on Quay](https://quay.io/repository/clymene/promtail-gateway/status \"Docker Repository on Quay\")](https://quay.io/repository/clymene/promtail-gateway)  \nThe Promtail Gateway is an optional service that can receive log data from the another component through gRPC or HTTP\ncommunication.\n\n1. gRPC, HTTP Service\n2. Time-series data insert to Database([supported DB](https://github.com/Clymene-project/Clymene/blob/main/docs/promtail-gateway/README.md#Option-description-by-storage-type)) (Optional)\n\n\n## How to work it on k8s\n\nFor example, see\nthe [k8s example](https://clymene-project.github.io/docs/tutorials/k8s-example/)\n\n## How to work it on docker-compose\n\nFor example, see\nthe [docker-compose example](https://clymene-project.github.io/docs/tutorials/docker-compose/)\n\n## Roadmap\n\n1. AI/ML platform for clymene(https://github.com/Clymene-project/clymene-analyzer)\n2. push-type agent(node-exporter, cadvisor, process-exporter, etc)\n3. Add new pipeline(trace or Something - [discussions](https://github.com/Clymene-project/Clymene/discussions))\n4. eBPF trace agent(for v3.0.0 release)\n\n## Get in Touch\n\nHave questions, suggestions, bug reports? Reach the project community via these channels:\n\n* GitHub [issues](https://github.com/clymene-project/clymene/issues)\n  and [discussions](https://github.com/Clymene-project/Clymene/discussions)\n\n\n## Further reading\nTo learn more about Clymene, consult the following talks and articles.\n\n### Recent talks and articles\n- Jan 2022 blog post \"[Efficient time series data collection and management plan in a distributed environment](https://medium.com/@clymeneallen/efficient-time-series-data-collection-and-management-plan-in-a-distributed-environment-5c4a62d45e36)\"\n- Aug 2022 blog post \"[Best Practices, Monitoring System for Multi-K8s Cluster Environments Using Open Source](https://medium.com/@clymeneallen/best-practices-monitoring-system-for-multi-k8s-cluster-environments-using-open-source-d85544052f37)\"\n- Aug 2022 blog post \"[Best practice, k8s Node/POD resource usage and log monitoring system for multi-k8s cluster environment using Open source](https://medium.com/@clymeneallen/best-practice-k8s-node-pod-resource-usage-and-log-monitoring-system-for-multi-k8s-cluster-3f2e5c2c401a)\"\n- Sep 2022 YouTube \"[[CLYMENE-PROJECT] DEMO VIDEO](https://www.youtube.com/watch?v=TqFXVlbBOGQ)\"\n- Jan 2023 blog post \"[Best Practices, K8s Monitoring System Using Open Source Clymene and Druid](https://medium.com/@clymeneallen/best-practices-k8s-monitoring-system-using-open-source-clymene-and-druid-e47b2d09955e)\"\n\n## License\n\nApache License 2.0, see [LICENSE](https://github.com/Clymene-project/Clymene/blob/main/LICENSE).\n\n# Code Style\n\n#### 1. https://github.com/golang/go/wiki/CodeReviewComments\n\n#### 2. https://peter.bourgon.org/go-in-production/#formatting-and-style    \n\n# supported\nThe Clymene project is being developed with the support of Open Source development license of [Jetbrains'](https://www.jetbrains.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FClymene-project%2FClymene","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FClymene-project%2FClymene","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FClymene-project%2FClymene/lists"}