{"id":18375109,"url":"https://github.com/zenorachi/dynamic-user-segmentation","last_synced_at":"2025-04-06T20:30:54.407Z","repository":{"id":190407195,"uuid":"682558064","full_name":"zenorachi/dynamic-user-segmentation","owner":"zenorachi","description":"Dynamic user segmentation service written in Golang is designed to add users to specific groups (segments) and remove users from segments.","archived":false,"fork":false,"pushed_at":"2023-09-26T13:17:42.000Z","size":307,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-22T06:25:37.642Z","etag":null,"topics":["docker","docker-compose","golang","golangci-lint","google-drive-api","migrations","nginx","postgresql","rest","sql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zenorachi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-08-24T12:35:58.000Z","updated_at":"2025-03-14T22:53:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"90a5fa89-0a8d-462b-9cc4-32db2058f4b7","html_url":"https://github.com/zenorachi/dynamic-user-segmentation","commit_stats":null,"previous_names":["zenorachi/dynamic-user-segmentation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zenorachi%2Fdynamic-user-segmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zenorachi%2Fdynamic-user-segmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zenorachi%2Fdynamic-user-segmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zenorachi%2Fdynamic-user-segmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zenorachi","download_url":"https://codeload.github.com/zenorachi/dynamic-user-segmentation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247547151,"owners_count":20956501,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","golang","golangci-lint","google-drive-api","migrations","nginx","postgresql","rest","sql"],"created_at":"2024-11-06T00:17:36.054Z","updated_at":"2025-04-06T20:30:54.053Z","avatar_url":"https://github.com/zenorachi.png","language":"Go","readme":"[![Golang](https://img.shields.io/badge/Go-v1.21-EEEEEE?logo=go\u0026logoColor=white\u0026labelColor=00ADD8)](https://go.dev/)\n[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n\n\u003cdiv align=\"center\"\u003e\n    \u003ch1\u003eDynamic User Segmentation service\u003c/h1\u003e\n    \u003ch5\u003e\n        A microservice written in the Go programming language is designed to add users to specific groups (segments) and remove users from segments with the capability to automate these processes. Additionally, it offers the ability to generate comprehensive reports on all operations, including the option to specify particular users, in the form of CSV files or links to CSV files.\n    \u003c/h5\u003e\n    \u003cp\u003e\n        English | \u003ca href=\"README.ru.md\"\u003eRussian\u003c/a\u003e \n    \u003c/p\u003e\n\u003c/div\u003e\n\n---\n\n## Technologies used:\n- [Golang](https://go.dev), [PostgreSQL](https://www.postgresql.org/)\n- [Docker](https://www.docker.com/), [Nginx](https://nginx.org/ru/)\n- [REST](https://ru.wikipedia.org/wiki/REST), [Swagger UI](https://swagger.io/tools/swagger-ui/)\n- [JWT Authentication](https://jwt.io/)\n\n---\n\n## Installation\n```shell\ngit clone git@github.com/zenorachi/dynamic-user-segmentation.git\n```\n\n---\n\n## Getting started\n#### [Detailed Guide to Google Drive Integration](./docs/examples/01-google-drive-setup.md)\n1. **Google Drive integration:**\n    * Register the application in [Google Cloud](https://developers.google.com/workspace/guides/create-project);\n    * Create a service account and generate a secret key for it;\n    * Add the received secret key to the `secrets/credentials` directory.;\n    * Modify the environment variable `GDRIVE_CREDENTIALS` in the .env file.\n\u003e **Hint:** the service can be launched without integrating with Google Drive. In this case, you need to leave the value of the `GDRIVE_CREDENTIALS` variable empty.\n\u003e In this case, when requesting a link to a CSV file, an error will occur stating that the Google Drive service is unavailable.\n2. **Setting up environment variables (create a .env file in the project root):**\n```dotenv\n# Database\nexport DB_HOST=\nexport DB_PORT=\nexport DB_USER=\nexport DB_NAME=\nexport DB_SSLMODE=\nexport DB_PASSWORD=\n\n# Local database\nexport LOCAL_DB_PORT=\n\n# Postgres service\nexport POSTGRES_PASSWORD=\n\n# Password Hasher\nexport HASH_SALT=\nexport HASH_SECRET=\n\n# Path to Google Drive credentials.json\nexport GDRIVE_CREDENTIALS=./secrets/credentials/your_credentials_file.json\n\n# GIN mode (optional, default - release)\nexport GIN_MODE=\n\n# Nginx \u0026 HTTPS\n# name of the app's service (app)\nexport APP_HOST=\n\n# app's port (as in main.yml)\nexport APP_PORT=\n\n# HTTPS connection (443, for example)\nexport HTTPS_PORT=\n```\n\u003e **Hint:**\nif you are running the project using Docker, set `DB_HOST` to \"**postgres**\" (as the service name of Postgres in the docker-compose).\n\n3. **(Optional) Adding certificates for Nginx to work correctly:**\n\u003e You need to generate certificates and place them in the `secrets/certs` directory in order to be able to access the service via HTTPS.\nYou can use the [**minica**](https://github.com/jsha/minica) utility.\n\n4. **Compile and run the project:**\n```shell\nmake\n```\n5. **To test the service's functionality, you can navigate to the address \nhttp://localhost:8080/docs/index.html to access the Swagger documentation.**\n\u003e **Hint:** to complete the authorization in Swagger UI after receiving the JWT token, you need \nto enter `Bearer \u003cyour_token\u003e` (without \"\u003c\" and \"\u003e\" symbols) in the input field.\n\n---\n\n## [Examples of requests](./docs/examples/02-requests.md)\n\n**[Users](./docs/examples/02-requests.md#Users)**\n* [Registration](./docs/examples/02-requests.md#1-registration)\n* [Authentication](./docs/examples/02-requests.md#2-authentication)\n* [Refresh token](./docs/examples/02-requests.md#3-refresh-token)\n\n**[Segments](./docs/examples/02-requests.md#Segments)**\n* [Create a segment](./docs/examples/02-requests.md#1-create-a-segment)\n* [Create a segment with an indication of the percentage of automatic addition](./docs/examples/02-requests.md#2-create-a-segment-with-an-indication-of-the-percentage-of-automatic-addition)\n* [Delete a segment by name](./docs/examples/02-requests.md#3-delete-a-segment-by-name)\n* [Delete a segment by ID](./docs/examples/02-requests.md#4-delete-a-segment-by-id)\n* [Get all segments](./docs/examples/02-requests.md#5-get-all-segments)\n* [Get a segment by ID](./docs/examples/02-requests.md#6-get-a-segment-by-id)\n\n**[User segment addition / removal operations](./docs/examples/02-requests.md#user-segment-addition--removal-operations)**\n* [Add segments to a user by a list of names](./docs/examples/02-requests.md#1-add-segments-to-a-user-by-a-list-of-names)\n* [Add segments to the user by a list of names with an indication of the ttl](./docs/examples/02-requests.md#2-add-segments-to-the-user-by-a-list-of-names-with-an-indication-of-the-ttl)\n* [Add segments to a user by a list of ID](./docs/examples/02-requests.md#3-add-segments-to-a-user-by-a-list-of-id)\n* [Add segments to the user by a list of ID with an indication of the ttl](./docs/examples/02-requests.md#4-add-segments-to-the-user-by-a-list-of-id-with-an-indication-of-the-ttl)\n* [Delete segments from a user by a list of names](./docs/examples/02-requests.md#5-delete-segments-from-a-user-by-a-list-of-names)\n* [Delete segments from a user by a list of ID](./docs/examples/02-requests.md#6-delete-segments-from-a-user-by-a-list-of-id)\n\n**[User-segment relations](./docs/examples/02-requests.md#user-segment-relations)**\n* [Get active segments of a user](./docs/examples/02-requests.md#1-get-active-segments-of-a-user)\n* [Get active users of a segment](./docs/examples/02-requests.md#2-get-active-users-of-a-segment)\n\n**[Reports](./docs/examples/02-requests.md#Reports)**\n* [Getting operation history](./docs/examples/02-requests.md#1-get-operation-history)\n* [Getting operation history in the form of a CSV file](./docs/examples/02-requests.md#2-get-operation-history-in-the-form-of-a-csv-file)\n* [Getting operation history in the form of a CSV file link](./docs/examples/02-requests.md#3-get-operation-history-in-the-form-of-a-csv-file-link)\n\n---\n\n## Additional features\n1. **Run tests**\n```shell\nmake test\n```\n\u003e **Hint:** for the tests to work correctly for configurations, it is necessary to comment\nout the first line in the Makefile: `include .env`.\n2. **Run the linter**\n```shell\nmake lint\n```\n3. **Create migration files**\n```shell\nmake migrate-create\n```\n4. **Migrations up / down**\n```shell\nmake migrate-up\n```\n```shell\nmake migrate-down\n```\n5. **Stop all running containers**\n```shell\nmake stop\n```\n\n---\n\n## Decisions\nIn the process of project implementation, I made the following decisions regarding certain contentious issues:\n\n* **How to implement a *many-to-many* relationship between users and segments?**\n\u003e **Decision:** it seemed to me that it would be more appropriate to use a linking table called `relations`.\nThis approach enables easy scalability of the application in the future without altering the current structure.\nIt simplifies the process of adding segments to a user and removing them, as well as streamlines the associated queries.\n\n* **Which request method is better to use when adding segments to a user (*POST* or *PUT*)?**\n\u003e **Decision:** it seemed to me that it would be more appropriate to use the *POST* method since new relations between users and segments are being created.\n\n* **Why in the connecting table `relations`, are the user ID and segment ID not implemented as references?**\n\u003e **Decision:** Initially, it was set up this way. The advantages are that no action is required \nto update relations if a segment is deleted. On the downside, if a segment is deleted, all associated relations\nare also removed. While this cleanup is beneficial, it results in the loss of operational history.\nHence, the decision was made to forego the use of references and implement a TRIGGER function.\nThis function responds to segment deletions by erasing all connections to the deleted segment and recording the actions in the `operations` table.\n\n* **How to implement automatic removal of users from segments?**\n\u003e **Decision:** there was a choice between creating a separate service that would periodically check whether\nthe relationship between a user and segments has expired (for example, using `time.Ticker` or the `cron` utility),\nor simply launching a separate goroutine if a TTL is specified in the request. This goroutine, using a `select` statement,\nwould wait for the specified time (or the context to be canceled) and then proceed to remove segments from users.\nThe chosen approach is relatively simple to implement, especially considering that the functionality to record\noperations in the `operations` table was already implemented in the segment removal method for users. \nThis greatly simplifies the process. However, it's worth noting that this approach has its downsides,\nsuch as potentially launching a large number of goroutines (so many that it could deplete the available memory),\nor other unfavorable scenarios that could disrupt the service. Therefore, if the application were not in a \"test\" mode,\nit would definitely be advisable to use the first approach. But in this case, to expedite the project,\nI decided to go with the second option.\n\n* **How to implement automatic user addition?**\n\u003e **Decision:** couldn't come up with anything better than to write a TRIGGER function that responds to new records being added to the `segments` table. \nIf new segments are added, the function checks the percentage of automatic addition\n(optionally specified in the segment creation request).\nIf the percentage is greater than zero, the function randomly selects users from the entire user pool\n(based on the specified percentage) and then adds the created segment to the selected users.\nIt also records information about the performed operation in the `operations` table. \nThis solution has its advantages, such as simplicity and implementation speed. \nHowever, there are downsides. The issue is that this approach only makes sense if we already have an existing user base.\nIf we don't, the entire purpose of the function disappears. It seemed somewhat logical to me to create new segments and test them on an existing user base, which is why I decided to stick with this solution.\n\n* **What service to use to generate a link to a CSV file?**\n\u003e **Decision:** as soon as I read the task, I immediately understood that I would likely use an S3 storage,\nsince I already had experience with S3 in the project [**ImageBox**](https://github.com/zenorachi/image-box).\nHowever, I can't deny that I was quite intrigued by the idea from the 2022 internship candidate of using\nGoogle Drive as a service. So, I decided to try something new for myself and implemented \nthe generation of a link to the CSV file using the Google Drive API integration.\n\n* **Should an error be returned if a user (segment) has no active segments (users)?**\n\u003e **Decision:** it seemed to me that this would be superfluous and not entirely appropriate, because the request is processed correctly,\nthere is simply no active connection between users and segments.\n\n* **Should I use *multistage build*?**\n\u003e **Decision:** I thought it might be unnecessary as it would significantly increase the build time.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzenorachi%2Fdynamic-user-segmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzenorachi%2Fdynamic-user-segmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzenorachi%2Fdynamic-user-segmentation/lists"}