{"id":13504417,"url":"https://github.com/myntra/golimit","last_synced_at":"2025-04-04T07:04:58.457Z","repository":{"id":47448446,"uuid":"131137876","full_name":"myntra/golimit","owner":"myntra","description":"Golimit is Uber ringpop based distributed and decentralized rate limiter","archived":false,"fork":false,"pushed_at":"2023-07-31T18:22:59.000Z","size":9392,"stargazers_count":614,"open_issues_count":2,"forks_count":27,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-03-28T06:03:22.499Z","etag":null,"topics":["distributed-systems","golang","rate-configs","rate-limiter","ringpop"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/myntra.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-04-26T10:13:33.000Z","updated_at":"2025-02-17T03:06:31.000Z","dependencies_parsed_at":"2024-01-06T01:03:36.464Z","dependency_job_id":"10bf6077-c1b7-4579-a987-2497241358d8","html_url":"https://github.com/myntra/golimit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myntra%2Fgolimit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myntra%2Fgolimit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myntra%2Fgolimit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/myntra%2Fgolimit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/myntra","download_url":"https://codeload.github.com/myntra/golimit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247135141,"owners_count":20889420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-systems","golang","rate-configs","rate-limiter","ringpop"],"created_at":"2024-08-01T00:00:37.276Z","updated_at":"2025-04-04T07:04:58.414Z","avatar_url":"https://github.com/myntra.png","language":"Go","funding_links":[],"categories":["Components","Go"],"sub_categories":["Vectors"],"readme":"## Golimit A Distributed Rate limiter\nGolimit is Uber [ringpop](https://github.com/uber/ringpop-go \"ringpop\") based distributed and decentralized rate \nlimiter. It is horizontally scalable and is based on shared nothing architecture. Every node in system is capable of \nhandling read and writes of counters.\nIt is designed to offer sub milliseconds latency to caller application. Recommended deployment topology is sidecar \nmodel.\nEvery golimit node keeps local and global counter for api counter and local value is synchronized with other nodes on \nconfigurable periodic interval or at defined threshold. \n\n### Architecture\n\u003cb\u003eHttp server\u003c/b\u003e\nprovides http interface to increment counter against any arbitrary \u003cb\u003eKey\u003c/b\u003e string. It also exposes admin api to \nmanage global configurations.\n\n\u003cb\u003eStore\u003c/b\u003e encapsulates the data structure and functions to store, manage and replicate counters.\nCounter synchronisation is done in asynchronous way so the caller application is never blocked for cluster sync.\nSynchronizer module keeps aggregating counters in memory and broadcast to other nodes on periodic intervals or when the \ncounter has crossed threshold. the interval and threshold are configurable.\n\n\u003cb\u003eStatsD Emitter\u003c/b\u003e pushes metrics to configured statsd server.\n \n![Block Diagram](https://github.com/myntra/golimit/blob/master/images/block.png?raw=true)\n\n#### Deployment\nSuggested deployment model is to have golimit installed as sidecar. This will ensure application latency to \nsub milliseconds level.\nFor Go applications golimit can be directly integrated as a module, the way of using golimit as module is explained \nlater in document. Using as module takes away the pain of deployment and maintenance.\n\n![Block Diagram](https://github.com/myntra/golimit/blob/master/images/deployment.png?raw=true)\n\n#### Installation\n\n1. Build\n\n    ```\n    $ GOOS=linux GOARCH=amd64 go build  //Linux\n    \n    $ GOOS=darwin GOARCH=amd64 go build //OSX \n    \n    $ GOOS=windows GOARCH=amd64 go build //Windows\n    ```\n     \n2. Configure\n    \n    Yml config\n    ```yaml\n        clustername:  MyGolimitCluster # Cluster Name\n        tchannelport: 2345 # Ringpop T Channel Port\n        seed: \"127.0.0.1:2345\" # Seed node of cluster\n        unsyncedctrlimit: 5  # Unsynced counter limit\n        unsyncedtimelimit: 60000 # unsynced timeout in ms\n        httpport: 8080  # Http server port\n        statsdenabled: true # Enable statsd \n        statsdhostport: \"metrics.xyz.com:80\"  # statsd host port\n        statsdsamplerate: 1 # Statsd sampling rate\n        apisecret: alpha # secret key to use admin apis\n        hostname: \"127.0.0.1\"\n        unixsocketenable: true #enables unix socket,\n                               #gives nearly 2x better performance in response time \n                               #when enabled tcp port config will be ignored and disabled \n        unixsocket: /tmp/golimit.sock #unix socket file location\n        \n    ```\n    Note: Ensure the seed node is always reachable.\n    \n3. Run\n    \n    ```\n    $ ./golimitV3 --config=./golimitconfig.yml\n    ```\n  \n4. Use from Http Apis\n\n    | Param | Description|\n    |:-------:|:-----------|\n    |K      | Key a string, against this the counters are calculated|\n    |C|Count in number, numbers to increment in one api call, defaults to 1|\n    |W|Window in seconds, time window for which provided threshold is applicable|\n    |T|Threshold in number|\n    |P|PeakAveraged 0 or 1, if P=1 the provided rate limit is transposed to per second limit and then applied|\n    \n    * INCR Request\n       \n       Application passes Key threshold and window and reply is block or not.\n       This rate limiting is application driven as application has to pass all rate configuration in every call \n       In following example second curl within 10 seconds gives back blocked =true\n        \n        ```\n        $  curl -X POST \"http://localhost:8080/incr?K=abc\u0026T=1\u0026W=10\" \n        \n        {\"Block\":false}\n        \n        $  curl -X POST \"http://localhost:8080/incr?K=abc\u0026T=1\u0026W=10\" \n            \n        {\"Block\":true}\n        ```\n    * Create/Update global rate configuration, this is for rate limiting which is golimit cluster driven\n        \n        ```\n        $ curl -X PUT  \"http://localhost:8080/rate\" -d '{\"Window\":60,\"Limit\":5,\"Key\":\"a\",\"PeakAveraged\":false}' -H \"apisecret: alpha\" \n           \n        {\"Success\":true}\n        \n        $ curl -X POST  \"http://localhost:8080/ratelimit?K=a\" \n    \n        {\"Block\":false}\n        \n        # after 5 times\n        # {\"Block\":true}\n        \n        ```\n    * Ratelimit Request\n    \n        ```\n         $ curl -X POST  \"http://localhost:8080/ratelimit?K=a\" \n         \n         {\"Block\":false}\n         \n        ```\n        \n    * Get All defined Rate Config\n    \n        ```\n        $ curl  \"http://localhost:8080/rateall\" \n        \n        {\"a\":{\"Window\":60,\"Limit\":5,\"PeakAveraged\":false}}\n        \n        ```\n    \n    * Get a specific Rate Config\n    \n        ```\n        $ curl  \"http://localhost:8080/rate?K=a\" \n    \n        {\"Window\":60,\"Limit\":5,\"PeakAveraged\":false}%\n        \n        ```\n    * Get Cluster Info\n        ```\n        $ curl  \"http://localhost:8080/clusterinfo\" \n        \n        {\"Whoami\":\"127.0.0.1:2345\",\"Ready\":true,\"Uptime\":9223372036854775807,\"Members\":[\"127.0.0.1:2345\"]}   \n\n        ```\n\n5. Use as Go module\n   \n    If application is in golang, golimit can be used as module directly instead of deploying as separate process.\n    \n    \n    To install library:\n    \n    ```go get github.com/myntra/golimit```\n   \n    ```go\n    package main\n    import (\"github.com/myntra/golimit/store\")\n    \n    func main() {\n    \n        //Instantiate Store object, Use single store instance in one application\n        store := store.NewStore()\n\n        blocked := store.Incr(\"key\", 1, 1000, 60, true) // Increment api\n\n        if (blocked) {\n            //Blocked\n        }\n\n        //Ensure Store is closed on program exit\n        store.Close()\n    }\n    ```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmyntra%2Fgolimit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmyntra%2Fgolimit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmyntra%2Fgolimit/lists"}