{"id":19652123,"url":"https://github.com/research-camp/minions","last_synced_at":"2025-07-18T08:05:56.602Z","repository":{"id":43027003,"uuid":"486866748","full_name":"research-camp/minions","owner":"research-camp","description":"MinIO distributed cache system.","archived":false,"fork":false,"pushed_at":"2024-01-12T18:03:27.000Z","size":165,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-23T04:41:32.013Z","etag":null,"topics":["cache","cache-storage","go","golang","minio","minio-cluster"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/research-camp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-29T06:47:37.000Z","updated_at":"2023-12-13T21:24:50.000Z","dependencies_parsed_at":"2024-01-12T23:27:11.470Z","dependency_job_id":"e1ad17c5-d8d6-48a5-b9e8-da0baed18f48","html_url":"https://github.com/research-camp/minions","commit_stats":null,"previous_names":["amirhnajafiz/minions","research-camp/minions"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/research-camp/minions","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/research-camp%2Fminions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/research-camp%2Fminions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/research-camp%2Fminions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/research-camp%2Fminions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/research-camp","download_url":"https://codeload.github.com/research-camp/minions/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/research-camp%2Fminions/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265724942,"owners_count":23817918,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","cache-storage","go","golang","minio","minio-cluster"],"created_at":"2024-11-11T15:09:23.270Z","updated_at":"2025-07-18T08:05:56.583Z","avatar_url":"https://github.com/research-camp.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Minions\n\nA distributed caching system for __MinIO__ cluster. Build with __Golang__. This system uses a main router\nwhich monitors cache systems and balances the input load between them. It also uses an __ETCD__ storage in order\nto keep the hit ratio as high as possible.\nBy using this system, we can speed our object storage cluster operations up to 20% faster as the average.\n\n## Collaborators\n\n- Amirhossein Najafizadeh (Amirkabir University of Technology)\n- Dr. Niloofar Charmchi (Université de Rennes I)\n\n## How it works?\n\nWe group our clients based on regions. After that we create a Minion for our regions. Some regions may have more than\none Minion based on the traffic that we receive from there. For every input request, we pass them to the router and it\nwill proxy the request to one of the Minions. Through this proxy, it gets a factor number from each Minion and it balances\nthe load between them by selecting the right Minion each time.\n\nEach Minion updates its factor in an interval and sends it to the router. The router keeps this metadata in an __ETCD__ storage\nto use them for future requests.\n\n![](.github/asserts/diagram.svg)\n\n## How to run?\n\nFirst, you need to setup a __MinIO__ cluster. You can do this by docker:\n\n```\ndocker run \\\n  -p 9000:9000 \\\n  -p 9001:9001 \\\n  -e \"MINIO_ROOT_USER=AKIAIOSFODNN7EXAMPLE\" \\\n  -e \"MINIO_ROOT_PASSWORD=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\" \\\n  quay.io/minio/minio server /data --console-address \":9001\"\n```\n\nAfter that you need to install two components, the minions and the router.\n\n### Router\n\nRouter is the brain behind all the system. It monitors Minions, controls the input load by\ndefining a load factor for each Minion, and tracks the cluster objects by storing them in an ETCD database.\n\nYou can start the router by using the following command:\n\n```\ngo run main.go router\n```\n\n### Minion\n\nEach Minion is a file manager which uses an interface in order to make calls to MinIO cluster.\nFor each Minion, a load factor is defined by $count(files) * sum(files.size)$. This factor allows the router to\nhandle the input traffic and keep the hitting ratio high.\n\nYou can start a single Minion by using the following command:\n\n```\ngo run main.go minion\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresearch-camp%2Fminions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fresearch-camp%2Fminions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresearch-camp%2Fminions/lists"}