{"id":28665325,"url":"https://github.com/nothingmuch/repricer","last_synced_at":"2025-06-13T13:38:26.287Z","repository":{"id":45276682,"uuid":"234134205","full_name":"nothingmuch/repricer","owner":"nothingmuch","description":"interview tech exercise","archived":false,"fork":false,"pushed_at":"2020-01-24T18:34:13.000Z","size":100,"stargazers_count":2,"open_issues_count":18,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-06-20T22:34:48.366Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nothingmuch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-15T17:24:09.000Z","updated_at":"2020-10-05T16:57:13.000Z","dependencies_parsed_at":"2022-09-14T05:11:06.873Z","dependency_job_id":null,"html_url":"https://github.com/nothingmuch/repricer","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/nothingmuch/repricer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nothingmuch%2Frepricer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nothingmuch%2Frepricer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nothingmuch%2Frepricer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nothingmuch%2Frepricer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nothingmuch","download_url":"https://codeload.github.com/nothingmuch/repricer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nothingmuch%2Frepricer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259654297,"owners_count":22890989,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-13T13:38:23.426Z","updated_at":"2025-06-13T13:38:26.252Z","avatar_url":"https://github.com/nothingmuch.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Repricer Service\n\n## Design\n\nThe service is implemented as a single, distroless Go container, with no\nexternal dependencies, built on Azure Pipelines and deployed to an Azure\nKubernetes Service cluster.\n\nThe `endpoints` package has 3 `net/http.Handler`s for the separate endpoints, as\ntheir implementation did not overlap by much (only `product` and `query`\nresponses share some serialization logic for the response body).\n\nThe handlers' constructors need a model to store the applications state,\nspecified as a Go `interface`, and this is implemented in the `storage`\npackage.\n\nThe storage model is built in two parts, one component to linearize operations\nand fill in the `previousPrice` field with *consistent* values, and one\ncomponent to aggregate new entries into separate files as specified.\n\n## My Approach and Experience\n\nAs my \"showing off\" I decided to try Azure Devops and Kubernetes, neither of\nwhich I had used before. For my own benefit, I decided not to take into\nconsideration any time spent learning these technologies, and only limit my\ncoding time to several hours (which also ran over budget, due to a redesign).\n\nI also suspended disbelief about the task, pretending that I was a new hire to\nsome company (i.e. I have other team members and need to collaborate with them\nusing standard tools and practices), and the requirements are \"real\" business\nrequirements but that I don't understand the reasons.\n\nAs such, I made a deliberate effor to focus on quality/robustness (e.g. avoiding\nedge cases, defining clear consistency properties for the data model, etc), like\na real product would need to, mainly because this gave me a clear direction to\npursue for the devops related tasks.\n\nSince the app is stateful and keeps data in memory but also writes it to disk it\nmust be deployed as a single instance application and avoid rolling deployments,\nwhich added some interest and provoked me to reconsider my design (the first\niteration of the storage model loaded all data on startup, but due to the\n`previousPrice` field requirements - which I have to say are [kind of\nridiculous](https://github.com/nothingmuch/repricer/issues/22)).\n\nTo avoid these \"production\" issues, I redesigned the storage model (the initial\ndesign worked in 2 phases, writing records to a staging area first, and then\nmoving those to the `results` directory when all `previousPrice` fields have\nbeen populated, which to ensure consistency needed snapshot reads of last\nprices, contrary to the `product` endpoint's unspecified read consistency\nsemantics).\n\n## Caveats\n\nSince I took too long on this there are still some things that are important,\nand which I would not consider acceptable in a real production system: there's\nalmost no logging, monitoring or observability.\n\nI had some issues with Azure Monitor which in hindsight seemed to have just been\ndelays, but I did not revisit that or Azure Application Insights even though I\nintended to.\n\nMost critically, some errors are simply discarded, since there's nowhere in the\ncode to send them too for logging/reporting in the current implementation.\n\n## Deployed Version\n\nThe [repository](https://github.com/nothingmuch/repricer) is mirrored to the\n`repricer` repostory here, but\n[issues](https://github.com/nothingmuch/repricer/issues) and [pull\nrequests](https://github.com/nothingmuch/repricer/pulls?utf8=%E2%9C%93\u0026q=is%3Apr)\nare hosted on GitHub (in hindsight I should have used Azure Devops repos, but I\nrealized too late, and there's no easy solution to import that data).\n\nThe master branch version of the service is deployed to AKS using Pipelines at\n[http://51.137.6.251/api](http://51.137.6.251/api/). Note that this is just the\nenvironment set up by the build pipeline, not a separate release pipeline..\n\n## Building \u0026 Running\n\nTo test the app there are several options:\n\n- `go run main.go` - the app will bind port 8080, and write the `results`\n  subdirectory (and an additional `results_by_product` subdirectory next to it)\n- `docker build .` will produce a distroless image that runs the service in\n  `/tmp/repricer`.\n- A Kubernetes deployment based on the Pipelines provided examples is defined in\n  the `manifests` subdirectory, but it uses Azure Disks for storage so the\n  persistent volume claim only makes sense on AKS.\n\n## Notes for Reviewer\n\n### Throttling\n\nAlthough I have implemented limiting of the maximum number of concurrent\nrequests, I had difficulties actually confirming these limits are enforced\nwithout adding e.g. `time.Sleep(10 * time.Millisecond)` to the HTTP handlers.\n\nSecondly `GET` endpoints have rate limiting implemented as a simple\n`http.Handler` middleware (with separate token buckets for each endpoint), but\nthe `reprice` endpoint is rate limited by the data model's nonblocking API,\nwhich queues up to 50 unwritten entries.\n\n### Unit Tests\n\nI deviated from the requirement document specified to *only* test one component.\n\nInitially I had set aside the storage, but then it became apparent that it can't\nreally be considered a single component, even though it's a single Go package.\n\nAs a result of this the `storage` package is [not very comprehensively unit\ntested](https://github.com/nothingmuch/repricer/issues/25), and arguably\nmultiple components have been tested in spite of the clear instructions not to\ndo that.\n\nNormally I often use a test driven approach, and even if not I try to aim for\n100% coverage in unit tests where possible as well as a clear distinction\nbetween unit tests, regression unit tests, and functional/integration tests.\n\n### Github vs. Azure Devops mirror of repository\n\nI started with an Azure Pipelines tutorial, and did not realize Azure Devops\nsupports hosting repositories until after I got it working with a Github\nrepository as per the instructions. To review pull requests \u0026 issues, make sure\nto look in the Github copy.\n\nSecondly, commits generally all have long messages, which azure devops doesn't\nlike to make very obvious to document the reasoning for the change as well as\nany caveats\n\n### Testing in Kubernetes\n\nSince the docker container is distroless, there's no easy way to inspect\n`/tmp/repricer` in the cluster environment (`kubectl cp` requires `tar`).\n\nI think the correct approach would be to create a new pod for testing, without\nthe persistent volume simulating the production environment (e.g. using\n`emptyDirectory`, see also\n[#26](https://github.com/nothingmuch/repricer/issues/26)).\n\nOf course it's very easy to test the docker image locally using a volume mounted\nin `/tmp/repricer`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnothingmuch%2Frepricer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnothingmuch%2Frepricer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnothingmuch%2Frepricer/lists"}