{"id":13764327,"url":"https://github.com/kodless/leek","last_synced_at":"2025-05-10T19:31:04.623Z","repository":{"id":38242217,"uuid":"295154233","full_name":"kodless/leek","owner":"kodless","description":"Celery Tasks Monitoring Tool","archived":false,"fork":false,"pushed_at":"2025-05-04T16:18:57.000Z","size":12965,"stargazers_count":168,"open_issues_count":14,"forks_count":17,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-04T16:35:01.630Z","etag":null,"topics":["celery","distributed","flower","monitor","monitoring-tool","python","queue","rabbitmq","redis","task","worker"],"latest_commit_sha":null,"homepage":"https://tryleek.com","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kodless.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-09-13T13:12:30.000Z","updated_at":"2025-05-04T16:19:00.000Z","dependencies_parsed_at":"2023-12-13T14:34:26.040Z","dependency_job_id":"3385bda5-d1b6-4b70-b570-8bfd315f5687","html_url":"https://github.com/kodless/leek","commit_stats":null,"previous_names":[],"tags_count":70,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kodless%2Fleek","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kodless%2Fleek/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kodless%2Fleek/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kodless%2Fleek/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kodless","download_url":"https://codeload.github.com/kodless/leek/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253470698,"owners_count":21913717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["celery","distributed","flower","monitor","monitoring-tool","python","queue","rabbitmq","redis","task","worker"],"created_at":"2024-08-03T16:00:19.029Z","updated_at":"2025-05-10T19:31:03.403Z","avatar_url":"https://github.com/kodless.png","language":"TypeScript","funding_links":["https://www.buymeacoffee.com/fennec"],"categories":["Administration and Monitoring"],"sub_categories":[],"readme":"[\u003cimg align=\"right\" width=\"200px\" height=\"45px\" src=\"https://cdn.buymeacoffee.com/buttons/default-yellow.png\"\u003e](https://www.buymeacoffee.com/fennec)\n\n# Leek\n\n\u003ch3 align=\"center\"\u003e\n    \u003cbr\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/kodless/leek/master/doc/static/img/logo.png\" alt=\"Leek Celery Monitoring Tool\" height=\"200\" width=\"200\"\u003e\u003c/a\u003e\n    \u003cbr\u003e\n    \u003cspan\u003eCelery Tasks Monitoring Tool\u003c/span\u003e\n    \u003cbr\u003e\n    \u003cspan\u003eDocumentation: https://tryleek.com\u003c/span\u003e\n    \u003cbr\u003e\n  \n[![Docs Build Status][docs-build-badge]][docs-build]\n[![App Build Status][app-build-badge]][app-build]\n[![Python 3.8][version-badge]][package]\n[![pulls][pulls-badge]][dockerhub]\n\n[![License][license-badge]][license]\n[![PRs Welcome][prs-badge]][prs]\n[![Code of Conduct][coc-badge]][coc]\n[![All Contributors](https://img.shields.io/badge/all_contributors-1-orange.svg?style=flat-square)](#contributors)\n\n[![Watch on GitHub][github-watch-badge]][github-watch]\n[![Star on GitHub][github-star-badge]][github-star]\n[![Tweet][twitter-badge]][twitter]\n\u003c/h3\u003e\n\n### What is Leek?\n\nLeek is a celery tasks monitoring tool, the main difference between Leek and other monitoring tools is that Leek can \nconnect to and monitor many brokers with a single container whereas other tools can monitor only a single broker at a \ntime.\n\nAlso leek supports environments branching, multiple applications, Google SSO, charts, issues monitoring, advanced \nfiltering and search, indexation and persistence, slack notifications and provides an awesome UI for a better user \nexperience.\n\nLeek came to remediate the issues found with other celery monitoring tools and provide a reliable results and cool \nfeatures to ease the process of monitoring your celery cluster, finding and respond to issues quickly.\n\n### What Leek is not?\n\nLeek is not a celery tasks/workers control tool and you cannot use leek to revoke/terminate/start tasks, restart your \nworkers fleet, or manage your brokers. however control features could be supported with future releases.\n\nLeek is not a package that can be installed/imported but a full stack application published as a docker image.\n\n### Features\n\nAs opposed to many other alternatives, leek came to fix the issues existing in other tools and offer many other cool \nfeatures that does not exist in other tools:\n\n- `Google SSO` - you can connect to leek using GSuite accounts for organizations and standard GMail accounts for\nindividuals.\n\n- `Multi brokers support` - other monitoring tools can connect to only one broker at a time, which enforces you to \ndeploy many instances to monitor them all. however, Leek with its Agent, it can monitor tasks from multiple brokers with \nonly a single instance of leek.\n\n- `Multi ENVs support` - when connecting Leek agent to brokers, you can specify an environment tag for that broker and \neach event sent from that broker will be tagged with that environment name, allowing you to split celery events into \nqa, stg, prod subsets so later you can filter task by environment name.\n\n- `Enhanced storage` - unlike other alternatives that stores the events in volatile RAM, celery events are indexed to \nelasticsearch to offer persistence and fast retrieval/search.\n\n- `Beatiful UI` - unlike other alternatives which are either a command line tool or have an ugly UI, Leek offers a \ngreat user experience thanks to its beautiful well designed UI.\n\n- `Notification` - you can define notification rules that will trigger a slack notification to inform you about critical\nevents, the notification triggers rules can match against task state, task name exclusion/exclusions, environment name, \nand runtime upper bound.\n\n- `Monitor Issues` - Leek can also monitor issues by aggregating the failed tasks by exception name, and for each \nexception it will calculate occurrences, recovered, pending, failed and critical exceptions.\n\n- `Charts` - Leek generate multiple charts giving you an idea about the application state, these chart includes: \ntasks states distribution, tasks queues distribution, top 5 executed tasks, top 5 slow tasks, tasks execution over time,\ntasks queue over time, tasks failure over time ...\n\n- `Filter by anything` - unlike other alternatives that doesn't provide a good support for filters, leek provides a wide\nrange of filters.\n\n- `Tasks control` - for now leek can retry tasks only, more tasks/workers control features may be introduced in the \n  future.\n\n\n### Running a local demo\n\nTo experiment with leek, you can run one of these demo docker-compose files:\n- [RabbitMQ Demo](https://github.com/kodless/leek/blob/master/demo/docker-compose-rmq-no-auth.yml)\n\n```bash\ncurl -sSL https://raw.githubusercontent.com/kodless/leek/master/demo/docker-compose-rmq-no-auth.yml \u003e docker-compose.yml\ndocker-compose up\n```\n\n- [Redis Demo](https://github.com/kodless/leek/blob/master/demo/docker-compose-redis-no-auth.yml)\n\n```bash\ncurl -sSL https://raw.githubusercontent.com/kodless/leek/master/demo/docker-compose-redis-no-auth.yml \u003e docker-compose.yml\ndocker-compose up\n```\n\nThis is an example of a demo, that includes 4 services:\n\n- Leek main application\n- A RabbitMQ or Redis broker\n- An elasticsearch node\n- Demo celery client (publisher)\n- Demo celery workers (consumer)\n\n- After running the services with `docker-compose up`, wait for the services to start and navigate to \nhttp://0.0.0.0:8000.\n\n- Create an application with the same name as in `LEEK_AGENT_SUBSCRIPTIONS`, which is `leek`.\n\n- Enjoy the demo\n\n```yaml\nversion: \"2.4\"\nservices:\n  # Main app\n  app:\n    image: kodhive/leek\n    environment:\n      # General\n      - LEEK_API_LOG_LEVEL=WARNING\n      - LEEK_AGENT_LOG_LEVEL=INFO\n      # Components\n      - LEEK_ENABLE_API=true\n      - LEEK_ENABLE_AGENT=true\n      - LEEK_ENABLE_WEB=true\n      # URLs\n      - LEEK_API_URL=http://0.0.0.0:5000\n      - LEEK_WEB_URL=http://0.0.0.0:8000\n      - LEEK_ES_URL=http://es01:9200\n      # Authentication\n      - LEEK_API_ENABLE_AUTH=false\n      # Subscriptions\n      - |\n        LEEK_AGENT_SUBSCRIPTIONS=\n        [\n          {\n            \"broker\": \"amqp://admin:admin@mq//\",\n            \"broker_management_url\": \"http://mq:15672\",\n            \"backend\": null,\n            \"exchange\": \"celeryev\",\n            \"queue\": \"leek.fanout\",\n            \"routing_key\": \"#\",\n            \"org_name\": \"mono\",\n            \"app_name\": \"leek\",\n            \"app_env\": \"prod\",\n            \"prefetch_count\": 1000,\n            \"concurrency_pool_size\": 2,\n            \"batch_max_size_in_mb\": 1,\n            \"batch_max_number_of_messages\": 1000,\n            \"batch_max_window_in_seconds\": 5\n          }\n        ]\n      - LEEK_AGENT_API_SECRET=not-secret\n    ports:\n      - 5000:5000\n      - 8000:8000\n    depends_on:\n      mq:\n        condition: service_healthy\n\n  # Just for local demo!! (Test worker)\n  worker:\n    image: kodhive/leek-demo\n    environment:\n      - BROKER_URL=pyamqp://admin:admin@mq:5672\n    depends_on:\n      mq:\n        condition: service_healthy\n\n  # Just for local demo!! (Test client)\n  publisher:\n    image: kodhive/leek-demo\n    environment:\n      - BROKER_URL=pyamqp://admin:admin@mq:5672\n    command: \u003e\n      bash -c \"python3 publisher.py\"\n    depends_on:\n      mq:\n        condition: service_healthy\n\n  # Just for local demo!! (Test broker)\n  mq:\n    image: rabbitmq:3.8.9-management-alpine\n    environment:\n      - RABBITMQ_DEFAULT_USER=admin\n      - RABBITMQ_DEFAULT_PASS=admin\n      - \"RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=-rabbit log [{console,[{level,error}]}]\"\n    ports:\n      - 15672:15672\n      - 5672:5672\n    healthcheck:\n      test: [ \"CMD\", \"nc\", \"-z\", \"localhost\", \"5672\" ]\n      interval: 2s\n      timeout: 4s\n      retries: 20\n\n  # Just for local development!! (Test index db)\n  es01:\n    image: elasticsearch:7.10.1\n    container_name: es01\n    environment:\n      - node.name=es01\n      - cluster.name=es-docker-cluster\n      - cluster.initial_master_nodes=es01\n      - bootstrap.memory_lock=true\n      - \"ES_JAVA_OPTS=-Xms512m -Xmx512m\"\n    command: [\"elasticsearch\", \"-Elogger.level=ERROR\"]\n    healthcheck:\n      test: [\"CMD-SHELL\", \"curl --silent --fail localhost:9200/_cluster/health || exit 1\"]\n      interval: 30s\n      timeout: 30s\n      retries: 3\n    ulimits:\n      memlock:\n        soft: -1\n        hard: -1\n      nofile:\n        soft: 65535\n        hard: 65535\n    ports:\n      - 9200:9200\n```\n\n[docs-build-badge]: https://api.netlify.com/api/v1/badges/33977c7c-42dc-44cb-8a4f-ac68c9877b6c/deploy-status\n[docs-build]: https://app.netlify.com/sites/leek/deploys\n[app-build-badge]: https://github.com/kodless/leek/workflows/Publish%20Leek%20Docker%20image/badge.svg\n[app-build]: https://app.netlify.com/sites/leek/deploys\n[version-badge]: https://img.shields.io/badge/python-3.8-blue.svg?style=flat-square\n[package]: https://pypi.org/project/xxxxxxx/\n[pulls-badge]: https://img.shields.io/docker/pulls/kodhive/leek.svg?style=flat-square\n[dockerhub]: https://hub.docker.com/r/kodhive/leek\n\n[license-badge]: https://img.shields.io/badge/License-Apache%202.0-blue.svg\n[license]: https://github.com/kodless/leek/blob/master/LICENSE\n[prs-badge]: https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square\n[prs]: https://github.com/kodless/leek/pulls/\n[coc-badge]: https://img.shields.io/badge/code%20of-conduct-ff69b4.svg?style=flat-square\n[coc]: https://github.com/kodless/leek/blob/master/CODE_OF_CONDUCT.md\n\n[github-watch-badge]: https://img.shields.io/github/watchers/kodless/leek.svg?style=social\n[github-watch]: https://github.com/kodless/leek/watchers\n[github-star-badge]: https://img.shields.io/github/stars/kodless/leek.svg?style=social\n[github-star]: https://github.com/kodless/leek/stargazers\n[twitter]: https://twitter.com/intent/tweet?text=Check%20out%20leek%20by%20%40HamzaAdami%20https%3A%2F%2Fgithub.com%2Fkodless%2Fleek%20%F0%9F%91%8D\n[twitter-badge]: https://img.shields.io/twitter/url/https/github.com/kodless/leek.svg?style=social\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkodless%2Fleek","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkodless%2Fleek","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkodless%2Fleek/lists"}