{"id":19308789,"url":"https://github.com/knands42/url-shortener","last_synced_at":"2026-05-08T15:06:35.149Z","repository":{"id":259314400,"uuid":"865614059","full_name":"knands42/URL-Shortener","owner":"knands42","description":"Just a simple go application to leverage my knowledge in Golang and System Design","archived":false,"fork":false,"pushed_at":"2024-12-12T00:36:00.000Z","size":161,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-25T05:32:25.183Z","etag":null,"topics":["docker","docker-compose","go","golang","makefile","url-shortener"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/knands42.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-30T20:27:16.000Z","updated_at":"2024-10-23T21:29:39.000Z","dependencies_parsed_at":"2024-10-24T10:04:44.638Z","dependency_job_id":"4ba0f90e-3024-41e1-822d-0704a0651c18","html_url":"https://github.com/knands42/URL-Shortener","commit_stats":null,"previous_names":["knands42/url-shortener"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knands42%2FURL-Shortener","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knands42%2FURL-Shortener/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knands42%2FURL-Shortener/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/knands42%2FURL-Shortener/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/knands42","download_url":"https://codeload.github.com/knands42/URL-Shortener/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240409849,"owners_count":19796797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","go","golang","makefile","url-shortener"],"created_at":"2024-11-10T00:16:30.456Z","updated_at":"2026-05-08T15:06:35.098Z","avatar_url":"https://github.com/knands42.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# URL-Shortener\n\nJust a simple go application to leverage my knowledge in Golang and System Design\n\n## Execute the application\n\n### Pre requisites\n\nBefore running the application, make sure you have the following golang tools installed with the following command:\n\n\u003e **Note**: Golang version 1.20 (or higher), Python3.10 (or higher) and docker is required to be already pre installed in your machine.\n\n```sh\nmake setup\n```\n\nThis will install swagger and migrate cli tools used by the application and blazemeter to run the performance tests.\n\n### Boot up the application\n\nStart the depencies with the following command:\n\n```sh\ndocker compose up\n```\n\nGenerate the swagger documentation:\n\n```sh\nmake gen-docs\n```\n\nMigrate the database:\n\n```sh\nmake migrate-up\n```\n\nThen run the application with the following command:\n\n```sh\nmake build-and-run\n```\n\n### Useful links\n\n- [Swagger Documentation](http://localhost:3333/swagger/index.html)\n- [Healthcheck](http://localhost:3333/health)\n\n### Run the tests\n\nFirst check if integration tests are passing:\n\n```sh\nmake integration-tests\n```\n\nThen execute the performance tests:\n\n```sh\nmake performance-tests\n```\n\n## Technical Decisions\n\nThis application was built thinking like any other application on an early stage would be built, but the real world solution for a highly available system is ilustrated in the diagram in the root of the project: `architecture.drawio`.\n\n### Storage\n\nThe choice between SQL and NoSQL databases was made based on the fact that the application needs to be highly available, consistent and query efficient, so the best choice was to use a SQL database even though NOSQL can scale horizontally and is more flexible.\n\nIn the future more metadata will be added like user information and how they are accessing and generating new short urls, so a SQL database will help with the relationships between the tables, but this the current implementation no need to extract the most of the **3rd normal form**.\n\n**RDS Aurora** would be one of the best choice for a production environment since it was build to scale horizontally, but also keeping in mind that **sharding** (another way to scale SQL databases) comes out of the box with a NoSQL database like **MongoDB**.\n\n### Caching\n\nThe application uses **Redis** to cache the short urls and the original urls, this is a good choice to avoid hitting the database every time a request is made.\n\nA good strategy is to rely on the **LRU eviction policy** to avoid memory leaks and to keep the cache size under control if the cache is in-memory (which for a first time with a few users it may be good enough).\n\n### Graceful Shutdown\n\nIn order to avoid memory leasks and to make sure that the application is not processing any request when it is shutting down or any connection remains open, the application uses the `os.Signal` to catch the `SIGTERM` signal and to gracefully shutdown the application, like closing the database, redis and jaeger connection.\n\n### Observability\n\nThe application uses **Jaeger** to trace the requests and to have a better understanding of the application performance and to debug any issues that may arise (like `where` it did happen, not only `when`).\n\nBesides that, the application uses **logs** (which is a must) middleware and custom logs for some failures that may happen in the application.\n\nA centralized collector and the power of OTEL make the appplication work with many vendors and delegate the responsibility of managing what to do with the data that is being collected, like metrics, traces and logs.\n\n\u003e **Note**: Other observability tools that will benefit the application are: Prometheus for metrics, APM to manage mostly the health of the app and Grafana for a centralized place to connect many datasources and have nice dashboards visualization for the project.\n\n### Security\n\nFor testing purposes the **cors** is enabled for all origins, but in a production environment it should be restricted to the domains that are allowed to access the application.\n\n### Background Jobs\n\nSince the application is highly available, every time some use tries to access informations regading the short url or just delete it, the application needs to record this actions somehow.\n\nInstead of doing synchronously and hang the TCP connection for each GET request just to process this actions, for a production environment the so called **outbox pattern** would be a good choice to avoid this kind of problem by relying on the existing database to store this actions (events) and then a background job would process this actions asynchronously (this will work because this is a long running process and not serverless) no **broker** is needed (unless we start to implement connections through microservices).\n\n\u003e **Note**: Another reason to use a SQL database is to exploit the **ACID** features that it provides (talking about consistency here), so background jobs can rely on it for this outbox pattern\n\n### Resilience\n\nThe application does not contain any resilience pattern manually implemented, but a **circuit breaker** or **rate limiter (per user)** would help to avoid cascading failures.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknands42%2Furl-shortener","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fknands42%2Furl-shortener","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fknands42%2Furl-shortener/lists"}