{"id":19490673,"url":"https://github.com/nurislam03/postoffice","last_synced_at":"2026-04-29T14:38:17.942Z","repository":{"id":101139992,"uuid":"439277397","full_name":"nurislam03/postoffice","owner":"nurislam03","description":"An example go (golang) project to illustrate the uses of RabbitMQ, Cron job worker, PostgreSQL, etc. A decent example how we can handle high throughput in our system. ","archived":false,"fork":false,"pushed_at":"2021-12-26T10:09:00.000Z","size":5011,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-08T08:45:38.321Z","etag":null,"topics":["cronjob","go","golang","message-queue","postgresql","rabbitmq"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nurislam03.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-17T09:25:41.000Z","updated_at":"2022-08-04T23:21:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"f419c62b-e6d3-45d8-a8d7-8b9f9478f70a","html_url":"https://github.com/nurislam03/postoffice","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nurislam03%2Fpostoffice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nurislam03%2Fpostoffice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nurislam03%2Fpostoffice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nurislam03%2Fpostoffice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nurislam03","download_url":"https://codeload.github.com/nurislam03/postoffice/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240738070,"owners_count":19849545,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cronjob","go","golang","message-queue","postgresql","rabbitmq"],"created_at":"2024-11-10T21:13:56.383Z","updated_at":"2026-04-29T14:38:12.921Z","avatar_url":"https://github.com/nurislam03.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Postoffice\n\n\u003c!-- GETTING STARTED --\u003e\nTo get a local copy of this project up and running follow these simple steps.\n\n\n### Technology used\n- [X] Go or golang\n- [X] Docker\n- [X] RabbitMQ\n- [X] PostgreSQL\n\n### Uses\nThis Project has two main part\n- [x] Server\n    - [x] RabbitMQ Publisher\n    - [x] Cron Job Scheduler\n- [x] Worker\n    - [x] RabbitMQ Consumer\n    - [x] Stores Object on PostgreSQL\n\n### Environment Setup\n1. In the source code you should get a file named `.example.env`.\n2. Copy everything from that file and past it by creating a file called `.env`\n\nIf you need any changes change accordingly.\n\n### Run Docker\n1. In `docker-compose.yml` file, two services are `rabbitmq` and `postgres`\n2. To run them follow the command:\n   ```sh\n   $ docker-compose up -d\n   ```\n3. Wait a bit. Now you should have both `rabbitmq` and `postgres` is ready.\n\n### Dependency\n1. To trac dependency create a `go.mod` file using the command below:\n   ```sh\n   $ go mod init \n   ```\n2. If you need to modify vendored packages use the command below:\n   ```sh\n   $ go mod vendor \n   ```\n3. Use the command below to cleans up unused dependencies or adds missing dependencies:\n   ```sh\n   $ go mod tidy \n   ```\n### Available Commands\n1. To see all available commands use:\n   ```sh\n   $ go run main.go\n   ```\n\n### Migration\n1. Use the command for migration\n   ```sh\n   $ go run main.go migrate\n   ```\n\n### Server\n1. Server accepts post request at `/callback` endpoint from `tester_server` (`Note`: necessary information about `tester_server` is given in the last part of this readme file.) and then It publishes them to RabbitMQ.\n2. A cron job scheduler is running which deletes records which are not seen in last 30s.\n3. To run use this command\n   ```sh\n   $ go run main.go serve \n   ```\n### Worker\n1. Worker will consume message form RabbitMQ and call a get request to `tester_server` using `tester_client` to get object details.\n2. It stores those details to postgresql database.\n3. To run the worker use the command:\n   ```sh\n   $ go run main.go worker \"\u003cn\u003e\"\n   ```\nHere `n` = `number of worker(s)` you want to consume message from the RabbitMQ. Just replace the `\u003cn\u003e` from the command.\n\n\n### Tester_Server\n\nWrite a rest-service that listens on localhost:9090 for POST requests on /callback. Run the go service attached to this task. It will send requests to your service\nat a fixed interval of 5 seconds. The request body will look like this:\n`{\n    \"object_ids\": [1,2,3,4,5,6]\n}`\nThe amount of IDs varies with each request. Expect up to 200 IDs.\nEvery ID is linked to an object whose details can be fetched from the provided service. Our service listens on localhost:9010/objects/:id and returns the following response:\n`{\n    \"id\": \u003cid\u003e,\n    \"online\": true|false \n}`\nNote that this endpoint has an unpredictable response time between 300ms and 4s!\nYour task is to request the object information for every incoming object_id and filter the objects by their \"online\" status.\n\nStore all objects in a PostgreSQL database along with a timestamp when the object was last seen.\nLet your service delete objects in the database when they have not been received for more than 30 seconds.\n\n**Important**: due to business constraints we are not allowed to miss any callback to our service. Write code in such a way that all errors are properly recovered\nand that the endpoint is always available. Optimize for very high throughput\nso that this service could work in production.\n\n\n#### Source Code of `tester_server.go`\n\n`package main\n\nimport (\n    \"bytes\"\n    \"fmt\"\n    \"log\"\n    \"math/rand\"\n    \"net/http\"\n    \"os\"\n    \"os/signal\"\n    \"strconv\"\n    \"strings\"\n    \"time\"\n)\n\nfunc main() {\n    rng := rand.New(rand.NewSource(time.Now().UnixNano()))\n\n\tgo func() {\n\t\tclient := \u0026http.Client{Timeout: 1 * time.Second}\n\n\t\tfor {\n\t\t\ttime.Sleep(5 * time.Second)\n\n\t\t\tids := make([]string, rng.Int31n(200))\n\t\t\tfor i := range ids {\n\t\t\t\tids[i] = strconv.Itoa(rng.Int() % 100)\n\t\t\t}\n\t\t\tbody := bytes.NewBuffer([]byte(fmt.Sprintf(`{\"object_ids\":[%s]}`, strings.Join(ids, \",\"))))\n\t\t\tresp, err := client.Post(\"http://localhost:9090/callback\", \"application/json\", body)\n\t\t\tif err != nil {\n\t\t\t\tfmt.Println(err)\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\t_ = resp.Body.Close()\n\t\t}\n\t}()\n\n\thttp.HandleFunc(\"/objects/\", func(w http.ResponseWriter, r *http.Request) {\n\t\ttime.Sleep(time.Duration(rng.Int63n(4000)+300) * time.Millisecond)\n\n\t\tidRaw := strings.TrimPrefix(r.URL.Path, \"/objects/\")\n\t\tlog.Println(idRaw)\n\t\tid, err := strconv.Atoi(idRaw)\n\t\tif err != nil {\n\t\t\thttp.Error(w, \"invalid id\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\tw.Write([]byte(fmt.Sprintf(`{\"id\":%d,\"online\":%v}`, id, id%2 == 0)))\n\t})\n\tgo func() { _ = http.ListenAndServe(\":9010\", nil) }()\n\n\tsig := make(chan os.Signal, 1)\n\tsignal.Notify(sig, os.Interrupt, os.Kill)\n\t\u003c-sig\n\n\tfmt.Println(\"closing\")\n}\n`\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnurislam03%2Fpostoffice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnurislam03%2Fpostoffice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnurislam03%2Fpostoffice/lists"}