{"id":13714394,"url":"https://github.com/CaptainCodeman/gce-cache-cluster","last_synced_at":"2025-05-07T01:33:41.072Z","repository":{"id":66357877,"uuid":"87826415","full_name":"CaptainCodeman/gce-cache-cluster","owner":"CaptainCodeman","description":"Easy groupcache clustering on GCE","archived":false,"fork":false,"pushed_at":"2019-01-17T01:12:57.000Z","size":23,"stargazers_count":35,"open_issues_count":0,"forks_count":5,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-11T00:38:53.614Z","etag":null,"topics":["autoscaling","cluster","gce","gcp","go","golang","gossip","groupcache","swim"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CaptainCodeman.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-10T15:21:51.000Z","updated_at":"2024-05-25T02:51:23.000Z","dependencies_parsed_at":"2023-02-23T04:16:01.021Z","dependency_job_id":null,"html_url":"https://github.com/CaptainCodeman/gce-cache-cluster","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainCodeman%2Fgce-cache-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainCodeman%2Fgce-cache-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainCodeman%2Fgce-cache-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainCodeman%2Fgce-cache-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CaptainCodeman","download_url":"https://codeload.github.com/CaptainCodeman/gce-cache-cluster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224551245,"owners_count":17330111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autoscaling","cluster","gce","gcp","go","golang","gossip","groupcache","swim"],"created_at":"2024-08-02T23:01:58.640Z","updated_at":"2024-11-14T01:31:07.511Z","avatar_url":"https://github.com/CaptainCodeman.png","language":"Go","readme":"# GCE Cache Cluster\n\nEasy groupcache clustering on GCE\n\n## Overview\n\n[Groupcache](https://github.com/golang/groupcache) is amazing, it's like\nmemcache on steroids but offering some new features:\n\n* Can run on your existing servers rather than as a separate service.\n* De-dupes cache requests to avoid \"stempeding herd\" cache filling.\n* Improves response time as hot items can kept in-memory (no network request).\n\nFun fact - Groupcache was written by the same author as memcache (so yeah, he\n_probably_ knows a thing or two about caching). It's also used within Google\nso you can be confident it works at scale.\n\nThere is a lot to love about groupcache so, if you're using Go, it's\nreally a no-brainer. There's just one issue ...\n\n## Peer Discovery\n\nIn order for groupcache to work, the servers need to know about each other.\nThat is, each server needs to keep an updated list of it's peers so it knows\nwhich node is responsible for loading and caching any item.\n\nThis package is designed to make that easier if you're using Google Compute\nEngine (GCE) especially if you have any kind of auto-scaling. It will handle\nthe job of maintaining the list of peers and can be configured to filter the\nnodes that should be considered part of any cluster (you may have more than\none \"service\" within a project which should each have their own independent\ngroupcache).\n\n## Usage\n\nImport the package:\n\n```go\nimport (\n    \"github.com/captaincodeman/gce-cache-cluster\"\n)\n```\n\nCreate a configuration or use the `LoadConfig` function to load from an `.ini`\nfile (defaults to `cachecluster.ini`):\n\n```go\nconfig, _ := cachecluster.LoadConfig(\"\")\n```\n\nExample configuration file:\n\n```ini\n[cache]\nport = 9080               # port that groupcache will run on\n\n[cluster]\nport = 9999               # port for cluster traffic\nheartbeat = 250           # heartbeat in milliseconds\n\n[match]\nproject = my-project      # project name (see note)\nzone = us-central1-b      # zone name (see note)\nnetwork_interface = nic0  # network interface to use\ntags = http-server        # tags used to match instances\n\n[meta]\nservice = web             # metadata used to match instances\n```\n\nNOTE: The project and zone are populated automatically from the GCE metadata.\n\nCreate a new cache instance with the config:\n\n```go\ncache, _ := cachecluster.New(config)\n```\n\nStart the groupcache webserver:\n\n```go\ngo http.ListenAndServe(cache.ListenOn(), cache)\n```\n\nRun your own web server as normal using groupcache to fill data from the\ncache when needed.\n\nThe groupcache peers will be kept updated automatically whenever any GCE\ninstances are added or removed. The project, zone, tags and meta settings\nare used to limit which instances should be considered part of the cluster.\n\n## How it works\n\nWhen a machine starts up the GCE metadata service is used to retrieve a list\nof running instances within the project \u0026 zone and the configuration settings\nused to filter these to those that should be considered part of the cluster.\n\nA [clustering package](https://github.com/clockworksoul/smudge) based on\n[SWIM](https://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf)\n(Scalable Weakly-consistent Infection-style Membership) is then used to keep\ntrack of which nodes are alive. As new instances start the existing nodes are\nnotified and dead nodes are removed.\n\nThe end result is that each node in the cluster maintains a complete list of\npeers that is kept uptodate very quickly.\n\nAll this is done in a separate goroutine so startup of groupcache and the web\nservice is not delayed. The groupcache will initially be operating in 'local'\nmode so the app will work fine while any cluster discovery is being performed.\n\n## Networking\n\nMost examples for groupcache seem to use port 9080 and the clustering package\nuses 9999 so I've continued to use those defaults. Whatever ports you use, it\nis important that the nodes be able to communicate with each other over them!\n\nExactly how you do this will depend on your setup but some things to consider\nare:\n\n* IPTable configuration\n* Docker configuration (exposed ports)\n* Networking firewall for GCP project\n\n## Example\n\nSee the example project for a working example.\n","funding_links":[],"categories":["Repositories"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCaptainCodeman%2Fgce-cache-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCaptainCodeman%2Fgce-cache-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCaptainCodeman%2Fgce-cache-cluster/lists"}