{"id":24617899,"url":"https://github.com/jacoblincool/dcard-internship-assignment-2024","last_synced_at":"2025-03-18T21:27:59.518Z","repository":{"id":222320422,"uuid":"756611584","full_name":"JacobLinCool/dcard-internship-assignment-2024","owner":"JacobLinCool","description":"Golang + MongoDB: 40K RPS API Server","archived":false,"fork":false,"pushed_at":"2024-02-13T21:04:38.000Z","size":36,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-24T23:41:30.819Z","etag":null,"topics":["dcard"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JacobLinCool.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-13T00:14:55.000Z","updated_at":"2024-12-25T04:35:13.000Z","dependencies_parsed_at":"2024-02-13T16:12:45.521Z","dependency_job_id":"d8d009dc-ee7a-49b7-9d2f-53e0390c075a","html_url":"https://github.com/JacobLinCool/dcard-internship-assignment-2024","commit_stats":null,"previous_names":["jacoblincool/dcard-internship-assignment-2024"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacobLinCool%2Fdcard-internship-assignment-2024","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacobLinCool%2Fdcard-internship-assignment-2024/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacobLinCool%2Fdcard-internship-assignment-2024/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JacobLinCool%2Fdcard-internship-assignment-2024/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JacobLinCool","download_url":"https://codeload.github.com/JacobLinCool/dcard-internship-assignment-2024/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244308973,"owners_count":20432294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dcard"],"created_at":"2025-01-24T23:41:35.475Z","updated_at":"2025-03-18T21:27:59.500Z","avatar_url":"https://github.com/JacobLinCool.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dcard Internship Assignment 2024\n\nGolang + MongoDB: 40K RPS API Server\u003csup\u003e*\u003c/sup\u003e\n\n\u003e \u003csup\u003e*\u003c/sup\u003e 40K RPS, 200% CPU usage in docker container on MBP M1 Pro 16GB, see [Third Iteration](#third-iteration)\n\n## Assignment\n\nThe assignment can be found here: \u003chttps://drive.google.com/file/d/1dnDiBDen7FrzOAJdKZMDJg479IC77_zT/view\u003e\n\nTL;DR: Create a simple advertisement HTTP API server using Golang. External libraries and tools are allowed.\n\n\u003e It seems a quite fun project for me to refresh my Golang and MongoDB knowledge.\n\n## Analysis\n\nNumbers go first.\n\n- ~3000 AD creations per day\n- **~10000 AD views per second**\n- ~1000 active ads at any given time\n\nThis assignment is quite simple. I don't want to over-engineer it with reverse proxies, load balancers, etc.\n\nI think Golang is enough for this one, but considering the ease of use and scalability for data management, I choose MongoDB as the database.\n\n\u003e SQL or NoSQL is not a big deal for this assignment. Replication and sharding can always solve the problem I'm facing.\n\nAn RPS of 10k will be the primary challenge.\n\nGolang is fast enough to handle this load, so the bottleneck will be the database.\n\nPreliminary tests show MongoDB with indexes can handle around 2k RPS on my MBP, still far from the target.\n\nSome cache mechanism is needed. Redis? No, only 1000 active ads, so I can cache them all in memory.\n\nThe tech stack becomes obvious:\n\n- Golang / GIN for the HTTP server\n- MongoDB for data storage\n- Docker / Kubernetes for deployment (later)\n\n## Design\n\nLow latency/high throughput for the READ operation is the primary goal; no one wants to wait for an ad to load.\nThe WRITE operation is not that frequent, so it's not a big deal. Also, a few seconds of ad publishing delay is acceptable.\n\nTo reduce the latency, I will cache the ads in memory and update the local memory cache every second, so the cache will be eventually consistent.\n\n```mermaid\ngraph TD\n    P[Publisher]\n    V[Viewer]\n    S[Server]\n    DB[Database]\n    M[In-Memory Cache]\n    P --\u003e|Write| S --\u003e|Write| DB --\u003e|Update| M\n    M --\u003e|Cached| S --\u003e|Query| V\n```\n\n## Result\n\nI used K6 for load testing.\n\nEnvironment:\n- MacBook Pro M1 Pro 16GB, DevContainer (5 cores)\n- Go 1.21.7\n- MongoDB 7.0.2\n- K6 0.49.0\n\n### First Iteration\n\nThe result is quite good for the first iteration in DevContainer.\n\n```bash\nscenarios: (100.00%) 1 scenario, 300 max VUs, 40s max duration (incl. graceful stop):\n        * default: 300 looping VUs for 10s (gracefulStop: 30s)\n\n\ndata_received..................: 193 MB 19 MB/s\ndata_sent......................: 10 MB  1.0 MB/s\nhttp_req_blocked...............: avg=173.72µs min=375ns    med=792ns   max=175.38ms p(90)=1.87µs  p(95)=2.7µs   \nhttp_req_connecting............: avg=164.91µs min=0s       med=0s      max=141.21ms p(90)=0s      p(95)=0s      \nhttp_req_duration..............: avg=39.91ms  min=126.2µs  med=34.89ms max=214.87ms p(90)=66.72ms p(95)=78.79ms \n  { expected_response:true }...: avg=39.91ms  min=126.2µs  med=34.89ms max=214.87ms p(90)=66.72ms p(95)=78.79ms \nhttp_req_failed................: 0.00%  ✓ 0           ✗ 71399\nhttp_req_receiving.............: avg=662.3µs  min=4.7µs    med=13.16µs max=157.87ms p(90)=70.83µs p(95)=163.75µs\nhttp_req_sending...............: avg=105.1µs  min=2µs      med=4.2µs   max=137.76ms p(90)=11.54µs p(95)=28.54µs \nhttp_req_tls_handshaking.......: avg=0s       min=0s       med=0s      max=0s       p(90)=0s      p(95)=0s      \nhttp_req_waiting...............: avg=39.14ms  min=104.66µs med=34.45ms max=183.84ms p(90)=65.37ms p(95)=76.49ms \nhttp_reqs......................: 71399  7135.616822/s\niteration_duration.............: avg=41.61ms  min=540.91µs med=35.99ms max=223.15ms p(90)=69.52ms p(95)=82.84ms \niterations.....................: 71399  7135.616822/s\nvus............................: 300    min=300       max=300\nvus_max........................: 300    min=300       max=300\n```\n\n7K RPS, with less than 100% CPU usage for the server (K6 consumes the rest, 300%+, totaling 500% for 5 cores)\nI believe the number can be doubled outside the DevContainer.\n\n### Second Iteration\n\nAfter some side work on k6 and disabling the debug log, the result is blazing fast compared to the first iteration.\n\n\u003e The k6 URL builder and logger IO caused the server cannot use all the CPU power.\n\n```\nscenarios: (100.00%) 1 scenario, 500 max VUs, 40s max duration (incl. graceful stop):\n        * default: 500 looping VUs for 10s (gracefulStop: 30s)\n\n\ndata_received..................: 869 MB 87 MB/s\ndata_sent......................: 46 MB  4.6 MB/s\nhttp_req_blocked...............: avg=15.99µs  min=250ns   med=417ns  max=41.18ms  p(90)=834ns   p(95)=1.16µs \nhttp_req_connecting............: avg=10.41µs  min=0s      med=0s     max=31.58ms  p(90)=0s      p(95)=0s     \nhttp_req_duration..............: avg=15.03ms  min=49.95µs med=6.4ms  max=154.56ms p(90)=42.5ms  p(95)=54.04ms\n  { expected_response:true }...: avg=15.03ms  min=49.95µs med=6.4ms  max=154.56ms p(90)=42.5ms  p(95)=54.04ms\nhttp_req_failed................: 0.00%  ✓ 0            ✗ 323240\nhttp_req_receiving.............: avg=844.43µs min=3.29µs  med=7.5µs  max=59.84ms  p(90)=58.7µs  p(95)=2.73ms \nhttp_req_sending...............: avg=36.36µs  min=1.37µs  med=2.29µs max=72.13ms  p(90)=4.29µs  p(95)=14.79µs\nhttp_req_tls_handshaking.......: avg=0s       min=0s      med=0s     max=0s       p(90)=0s      p(95)=0s     \nhttp_req_waiting...............: avg=14.15ms  min=43.12µs med=6.28ms max=140.83ms p(90)=39.69ms p(95)=48.96ms\nhttp_reqs......................: 323240 32266.303532/s\niteration_duration.............: avg=15.41ms  min=65.37µs med=6.73ms max=154.58ms p(90)=43.03ms p(95)=54.67ms\niterations.....................: 323240 32266.303532/s\nvus............................: 500    min=500        max=500 \nvus_max........................: 500    min=500        max=500\n```\n\n32K RPS, with 250% CPU usage for the server (K6 consumes the rest, 200%+, totaling 500% for 5 cores)\n\n\u003e Note: VUS has been increased from 300 to 500.\n\n### Third Iteration\n\nAfter implementing some indexes for in-memory cache, 40K RPS is achieved.\n\n```\nscenarios: (100.00%) 1 scenario, 500 max VUs, 40s max duration (incl. graceful stop):\n        * default: 500 looping VUs for 10s (gracefulStop: 30s)\n\n\ndata_received..................: 253 MB 25 MB/s\ndata_sent......................: 58 MB  5.8 MB/s\nhttp_req_blocked...............: avg=21.11µs  min=250ns   med=416ns  max=41.4ms   p(90)=792ns   p(95)=1.12µs  \nhttp_req_connecting............: avg=17.6µs   min=0s      med=0s     max=29.93ms  p(90)=0s      p(95)=0s      \nhttp_req_duration..............: avg=11.82ms  min=29.54µs med=5.89ms max=124.07ms p(90)=30.98ms p(95)=39.29ms \n  { expected_response:true }...: avg=11.82ms  min=29.54µs med=5.89ms max=124.07ms p(90)=30.98ms p(95)=39.29ms \nhttp_req_failed................: 0.00%  ✓ 0            ✗ 406790\nhttp_req_receiving.............: avg=529.52µs min=3.25µs  med=6.29µs max=48.1ms   p(90)=24.58µs p(95)=145.58µs\nhttp_req_sending...............: avg=25.22µs  min=1.37µs  med=2.29µs max=51ms     p(90)=3.91µs  p(95)=12.58µs \nhttp_req_tls_handshaking.......: avg=0s       min=0s      med=0s     max=0s       p(90)=0s      p(95)=0s      \nhttp_req_waiting...............: avg=11.27ms  min=23.33µs med=5.83ms max=117.43ms p(90)=29.39ms p(95)=35.63ms \nhttp_reqs......................: 406790 40651.191179/s\niteration_duration.............: avg=12.21ms  min=45.95µs med=6.31ms max=124.09ms p(90)=31.58ms p(95)=40.25ms \niterations.....................: 406790 40651.191179/s\nvus............................: 500    min=500        max=500 \nvus_max........................: 500    min=500        max=500\n```\n\n40K RPS, with ~200% CPU usage for the server (K6 consumes the rest ~250%, totaling 500% for 5 cores)\n\n### Other Observations\n\nIn addition to the previous results, I noticed that after restarting the Docker engine, the RPS increased to around 45K while maintaining the same CPU usage.\n\nWhen running the application outside the DevContainer on my MBP, the RPS reached approximately 60K with around 300% CPU usage. The memory usage remained low, around 25MB when idle and 50MB under load.\n\nFurthermore, I conducted another test by simulating 10,000 active ads, which is 10x the original number. This test was performed within the same DevContainer environment.\n\n- The memory usage increased to around 60MB when idle and 100MB under load.\n- The RPS achieved was approximately 17K with a CPU usage of around 300%.\n\n```\nscenarios: (100.00%) 1 scenario, 500 max VUs, 40s max duration (incl. graceful stop):\n        * default: 500 looping VUs for 10s (gracefulStop: 30s)\n\n\ndata_received..................: 627 MB 63 MB/s\ndata_sent......................: 24 MB  2.4 MB/s\nhttp_req_blocked...............: avg=16.26µs min=291ns  med=458ns  max=31.37ms  p(90)=792ns    p(95)=1.08µs  \nhttp_req_connecting............: avg=12.05µs min=0s     med=0s     max=13.32ms  p(90)=0s       p(95)=0s      \nhttp_req_duration..............: avg=28.94ms min=7.95µs med=5.87ms max=382.99ms p(90)=101.56ms p(95)=121.13ms\n  { expected_response:true }...: avg=28.94ms min=7.95µs med=5.87ms max=382.99ms p(90)=101.56ms p(95)=121.13ms\nhttp_req_failed................: 0.00%  ✓ 0            ✗ 172140\nhttp_req_receiving.............: avg=99.62µs min=3.41µs med=9.29µs max=47.12ms  p(90)=23.75µs  p(95)=59.62µs \nhttp_req_sending...............: avg=12.92µs min=1.37µs med=2.25µs max=36.39ms  p(90)=3.7µs    p(95)=7.62µs  \nhttp_req_tls_handshaking.......: avg=0s      min=0s     med=0s     max=0s       p(90)=0s       p(95)=0s      \nhttp_req_waiting...............: avg=28.83ms min=0s     med=5.82ms max=382.98ms p(90)=101.4ms  p(95)=120.94ms\nhttp_reqs......................: 172140 17183.435768/s\niteration_duration.............: avg=29.04ms min=51µs   med=5.92ms max=383.01ms p(90)=101.85ms p(95)=121.2ms \niterations.....................: 172140 17183.435768/s\nvus............................: 500    min=500        max=500 \nvus_max........................: 500    min=500        max=500\n```\n\n## Conclusion\n\n| Scenario          | RPS | CPU Usage | Memory Usage (idle / loaded) | Environment                            |\n| ----------------- | --- | --------- | ---------------------------- | -------------------------------------- |\n| 1,000 Active Ads  | 40K | ~200%     | 25MB / 50MB                  | DevContainer (5 cores), MBP M1 Pro 16G |\n| 1,000 Active Ads  | 60K | ~300%     | -                            | MBP M1 Pro 16G                         |\n| 10,000 Active Ads | 17K | ~300%     | 60MB / 100MB                 | DevContainer (5 cores), MBP M1 Pro 16G |\n\nThe architecture is designed to be simple, efficient, and easily scalable by adding additional server nodes. To handle a larger number of ADs, sharding by region (country) can be implemented.\n\n```mermaid\ngraph TD\n    DB[Database]\n    S1[\"Server (TW)\"]\n    S2[\"Server (JP)\"]\n    S3[\"Server (US)\"]\n    M1[\"Local Cache (TW)\"]\n    M2[\"Local Cache (JP)\"]\n    M3[\"Local Cache (US)\"]\n    S1 --\u003e|Write| DB --\u003e|Update| M1\n    S2 --\u003e|Write| DB --\u003e|Update| M2\n    S3 --\u003e|Write| DB --\u003e|Update| M3\n    M1 --\u003e|Cached| S1\n    M2 --\u003e|Cached| S2\n    M3 --\u003e|Cached| S3\n```\n\n## Other stuff\n\n- CI/CD has been set up with GitHub Actions, and the Docker image is pushed to GitHub Container Registry.\n- Deployment is available in [`deployment`](./deployment) directory, including Docker Compose and Kubernetes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjacoblincool%2Fdcard-internship-assignment-2024","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjacoblincool%2Fdcard-internship-assignment-2024","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjacoblincool%2Fdcard-internship-assignment-2024/lists"}