{"id":21487181,"url":"https://github.com/ntsd/backend-engineer-challenge-new","last_synced_at":"2025-03-17T10:14:36.064Z","repository":{"id":188392664,"uuid":"611219710","full_name":"ntsd/backend-engineer-challenge-new","owner":"ntsd","description":null,"archived":false,"fork":false,"pushed_at":"2023-03-13T18:18:07.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-23T19:47:53.434Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ntsd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-03-08T11:23:13.000Z","updated_at":"2023-08-15T03:07:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"5bb08e19-2922-474d-988d-0b5c15d2ef64","html_url":"https://github.com/ntsd/backend-engineer-challenge-new","commit_stats":null,"previous_names":["ntsd/backend-engineer-challenge-new"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntsd%2Fbackend-engineer-challenge-new","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntsd%2Fbackend-engineer-challenge-new/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntsd%2Fbackend-engineer-challenge-new/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntsd%2Fbackend-engineer-challenge-new/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ntsd","download_url":"https://codeload.github.com/ntsd/backend-engineer-challenge-new/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244014186,"owners_count":20383716,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T13:27:08.619Z","updated_at":"2025-03-17T10:14:35.305Z","avatar_url":"https://github.com/ntsd.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Backend Engineer Coding Challenge\n\n## API Architecture\n\nNote: for Mermaid you may need a [Mermaid viewer](https://mermaid.live), by Default Github will render mermaid on Markdown.\n\n```mermaid\nC4Component\n    title API Architecture\n\n    Person(client, \"User\")\n\n    Boundary(process, \"Go Process\") {\n        Component(api, \"API\", \"Go\", \"Provides all the API requests.\")\n\n        ComponentQueue(scanner, \"Scanner\", \"Go Routine\", \"A worker pool for repository scanner.\")\n    }\n\n    ComponentDb(db, \"Database\", \"Postgres\", \"Store everything here\")\n\n    Rel_D(client, api, \"HTTP request\")\n    Rel_R(api, scanner, \"Create\", \"Go routine\")\n    Rel_D(scanner, db, \"Uses\")\n    Rel_D(api, db, \"Uses\")\n```\n\n## Requirements\n\n- [Go 1.8 or above](https://go.dev/doc/install)\n\n## Libraries\n\nThis package uses the following libraries to help with the development.\n\n- github.com/gin-gonic/gin - a HTTP web framework\n- github.com/golang/mock - a mocking framework\n- gorm.io/gorm - a ORM library\n- gorm.io/driver/postgres - a Postgres driver for Gorm\n- github.com/go-gormigrate/gormigrate/v2 - database migration for GORM\n- github.com/go-git/go-git - Git implementation for Go\n- github.com/sirupsen/logrus - a structured logger for Go\n\n## Project structure\n\nThe project structure is following [peoject-layout](https://github.com/golang-standards/project-layout) non official standard.\n\n```tree\n.\n├── cmd:                            - main command files\n├── deployments:                    - deployment files\n├── internal\n│   ├── config:                     - environment variables config\n│   ├── storage:                    - database connector and ORM query\n│   ├── handler:                    - HTTP handlers\n│   ├── model:                      - models of the database\n│   ├── scanner:                    - repository scanner worker\n```\n\n## Database design\n\n```mermaid\nerDiagram\n    Repository {\n        string id pk\n        string name \"repository name\"\n        string url \"repository URL\"\n        time created_at \"created time\"\n        time updated_at \"latest updated time\"\n        time deleted_at \"soft deleted time\"\n    }\n    Scan {\n        string id pk\n        string repository_id fk \"repository ID\"\n        string repository_url \"repository URL will not change after origin URL is changed\"\n        scan_status status \"Queued|In Progress|Success|Failure\"\n        json findings \"scan findings json object\"\n        time started_at \"scan started time\"\n        time finished_at \"scan finished time\"\n        time created_at \"scan created time\"\n        time updated_at \"latest updated time\"\n        time deleted_at \"soft deleted time\"\n    }\n    Repository ||--o{ Scan : \"\"\n```\n\n## Environment Variables\n\n| Key          | Description                             | Example                                       |\n| ------------ | --------------------------------------- | --------------------------------------------- |\n| APP_PORT     | port number, default: `8080`            | 8080                                          |\n| POSTGRES_URL | **Required** PostgreSQL URL             | postgresql://user:pass@localhost:5432/db_name |\n| SCAN_WORKERS | number of scanner workers, default: `2` | 2                                             |\n\n## Run (Docker)\n\nThis package has the Dockerfile and the Docker compose provide to make it easier to run.\n\n```sh\ndocker-compose build # or `make docker-build`\ndocker-compose up # or `make docker-up`\n```\n\n## Run local\n\n```sh\nmake run\n# or\ngo run cmd/run/main.go\n```\n\n## Run unit test\n\nThis package using gomock for the mocking framework.\n\n```sh\nmake test\n# or\ngo test ./...\n```\n\n## Generate mock\n\nTo generate mock require [gomock](https://github.com/golang/mock) mockgen\n\n```sh\ngo install github.com/golang/mock/mockgen@v1.6.0\n```\n\ncmd to generate mock\n\n```sh\nmake mockgen\n```\n\n## API Reference\n\n### Entity\n\nthe json response entity will following type\n\n#### Success Response\n\nsuccess response will response in json format\n\n| field | type   | description                                                                                  |\n| ----- | ------ | -------------------------------------------------------------------------------------------- |\n| data  | object | response object can be [Repository](#Repository), [[Repository](#Repository)], [Scan](#Scan) |\n\n#### Error Response\n\nhttp response status code other than `200` will be considor as an error\n\n| field   | type | description          |\n| ------- | ---- | -------------------- |\n| message | text | message of the error |\n\npossible http status and error message\n\n| HTTP status | message              | description                      |\n| ----------- | -------------------- | -------------------------------- |\n| 400         | bad request          | something wrong with the request |\n| 404         | not found            | data not found                   |\n| 422         | data is duplicated   | some of field is unique          |\n| 500         | something went wrong | internal server error            |\n\n#### Repository\n\nGit repository object. the URL need to be unique\n\n\\*Note: only the Get Repository endpoint will allow to query the scan results for performance purpose\n\n| field     | type            | description                               |\n| --------- | --------------- | ----------------------------------------- |\n| id        | text            | id of the repository                      |\n| name      | text            | name of the repository                    |\n| url       | text            | url of the repository                     |\n| createdAt | time            | the created time of the repository        |\n| updatedAt | time            | the latest updated time of the repository |\n| scans     | [[Scan](#Scan)] | scan list of the repository               |\n\n#### Scan\n\nGit scan result object will create every time you scan the repo\n\n| field         | type                  | description                                                    |\n| ------------- | --------------------- | -------------------------------------------------------------- |\n| id            | string                | id of the scan                                                 |\n| repositoryID  | string                | repositoryID will reference to Repository                      |\n| repositoryURL | string                | repositoryURL will not change after the repository url changed |\n| status        | string                | scan status (Queued, In Progress, Success, Failure)            |\n| findings      | [[Finding](#Finding)] | finding object after scan found something                      |\n| startedAt     | string                | the started time of the scan                                   |\n| finishedAt    | string                | the finished time of the scan                                  |\n| createdAt     | string                | the created time of the scan                                   |\n| updatedAt     | string                | the latest updated time of the scan                            |\n\n#### Finding\n\nlist of scan when found something\n\n| field       | type   | description            |\n| ----------- | ------ | ---------------------- |\n| path        | string | code path              |\n| line        | number | line of code           |\n| description | string | detail for the finding |\n\n### Endpoints\n\n#### Create Repository\n\nan endpoint to create Git repository\n\n`POST /repositories`\n\nRequest body\n\n| field | type | description            |\n| ----- | ---- | ---------------------- |\n| name  | text | name of the repository |\n| url   | text | url of the repository  |\n\nResponse: [Repository](#Repository)\n\nExample\n\n```sh\ncurl http://localhost:8080/repositories \\\n    --header 'Content-Type: application/json' \\\n    --data '{\n        \"name\": \"scan-test\",\n        \"url\": \"https://github.com/ntsd/scan-test\"\n    }'\n```\n\n#### List Repositories\n\nan endpoint to list all repositories\n\n`GET /repositories`\n\nResponse: array of [Repository](#Repository), this endpoint will not response `scans` for performance purpose.\n\nExample\n\n```sh\ncurl http://localhost:8080/repositories\n```\n\n#### Get Repository\n\nget one repository\n\n`GET /repositories/:repositoryID`\n\nResponse: [Repository](#Repository)\n\nExample\n\n```sh\ncurl http://localhost:8080/repositories/f775e48a-1515-411c-b004-994ec66e1111\n```\n\n#### Update Repository\n\npatch will allow to update the repository by fields, no need to add the full repository object\n\n`PATCH /repositories/:repositoryID`\n\nRequest body\n\n| field | type | description            |\n| ----- | ---- | ---------------------- |\n| name  | text | name of the repository |\n| url   | text | url of the repository  |\n\nResponse: an updated [Repository](#Repository), this endpoint will not response `scans`.\n\nExample\n\n```sh\ncurl --request PATCH http://localhost:8080/repositories/f775e48a-1515-411c-b004-994ec66e1111 \\\n    --header 'Content-Type: application/json' \\\n    --data '{\n        \"name\": \"scan-test-2\"\n    }'\n```\n\n#### Delete Repository\n\ndelete repository\n\n`DELETE /repositories/:repositoryID`\n\nExample\n\n```sh\ncurl --request DELETE 'http://localhost:8080/repositories/f775e48a-1515-411c-b004-994ec66e1111'\n```\n\n#### Scan Repository\n\nscan repository\n\n`POST /repositories/:repositoryID/scan`\n\nResponse: [Scan](#Scan)\n\nExample\n\n```sh\ncurl --request POST 'http://localhost:8080/repositories/f775e48a-1515-411c-b004-994ec66e1111/scan'\n```\n\n## Improvements\n\nan improvement list that can't finish in an amount of time.\n\n- separate scanner worker to another instance for a better scaling.\n- clone repository to temporary directory instead of re clone every scan, and git reset to remote origin.\n- better queue system, maybe implement a 3rd party tool.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntsd%2Fbackend-engineer-challenge-new","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fntsd%2Fbackend-engineer-challenge-new","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntsd%2Fbackend-engineer-challenge-new/lists"}