{"id":24194364,"url":"https://github.com/subho004/retail-pulse","last_synced_at":"2026-05-16T13:31:39.198Z","repository":{"id":272312306,"uuid":"916172690","full_name":"subho004/retail-pulse","owner":"subho004","description":"A Go-based microservice to process store visit images in parallel, validate store data, and track job statuses. Includes Docker support, automated testing, and comprehensive error handling. Ideal for high-performance image processing pipelines.","archived":false,"fork":false,"pushed_at":"2025-01-13T15:43:55.000Z","size":52,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-09T01:12:09.246Z","etag":null,"topics":["api","backend-development","concurrency","docker","go","go-mux","golang","goroutines","image-processing","job-tracking","microservices","parallel-processing","retail","store-data-validation","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/subho004.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-13T15:39:08.000Z","updated_at":"2025-01-13T15:43:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"a2027104-438e-4bd4-bf1d-85ac26f17cfd","html_url":"https://github.com/subho004/retail-pulse","commit_stats":null,"previous_names":["subho004/retail-pulse"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/subho004/retail-pulse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subho004%2Fretail-pulse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subho004%2Fretail-pulse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subho004%2Fretail-pulse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subho004%2Fretail-pulse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/subho004","download_url":"https://codeload.github.com/subho004/retail-pulse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subho004%2Fretail-pulse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33104421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T04:41:52.686Z","status":"ssl_error","status_checked_at":"2026-05-16T04:41:52.009Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","backend-development","concurrency","docker","go","go-mux","golang","goroutines","image-processing","job-tracking","microservices","parallel-processing","retail","store-data-validation","testing"],"created_at":"2025-01-13T18:25:25.026Z","updated_at":"2026-05-16T13:31:39.189Z","avatar_url":"https://github.com/subho004.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Retail Pulse Image Processing Service\n\nA Go-based microservice that processes store visit images in parallel, calculates image perimeters, and manages job statuses.\n\n## Description\n\nThis service provides APIs to:\n\n- Submit image processing jobs for store visits\n- Track job processing status\n- Handle multiple concurrent jobs\n- Process thousands of images in parallel\n- Validate store information against a master store list\n\n### Key Features\n\n- Concurrent image processing using goroutines\n- Job status tracking\n- Store master data validation\n- Error handling for failed downloads or invalid stores\n- Docker support\n- Comprehensive testing suite\n\n## Assumptions\n\n1. **Store Master Data**:\n\n   - Stores are pre-loaded from a CSV file named `StoreMasterAssignment.csv`\n   - CSV format: AreaCode, StoreName, StoreID\n   - Store IDs are unique\n\n2. **Image Processing**:\n\n   - Images are accessible via HTTP(S) URLs\n   - Processing time is simulated (0.1 to 0.4 seconds per image)\n   - Failed image downloads should fail the entire job\n\n3. **Performance**:\n   - Multiple jobs can run concurrently\n   - Each job can process multiple store visits in parallel\n   - Multiple images per store visit are processed in parallel\n\n## Installation \u0026 Setup\n\n### Prerequisites\n\n- Go 1.21 or later\n- Docker (optional)\n- jq (for running tests)\n\n### Local Setup\n\n1. Clone the repository:\n\n```bash\ngit clone https://github.com/subho004/retail-pulse\ncd retail-pulse\n```\n\n2. Initialize Go modules:\n\n```bash\ngo mod init retail-pulse\ngo mod tidy\n```\n\n3. Place store master data:\n\n```bash\n# Ensure StoreMasterAssignment.csv is in the project root\n```\n\n4. Run the server:\n\n```bash\ngo run cmd/server/main.go\n```\n\n### Docker Setup\n\n1. Build and run using Docker Compose:\n\n```bash\ndocker-compose up --build\n```\n\n## API Routes\n\n### 1. Submit Job\n\nSubmit a new image processing job.\n\n**Endpoint**: `POST /api/submit`\n\n**Request Body**:\n\n```json\n{\n  \"count\": 2,\n  \"visits\": [\n    {\n      \"store_id\": \"RP00001\",\n      \"image_url\": [\n        \"https://example.com/image1.jpg\",\n        \"https://example.com/image2.jpg\"\n      ],\n      \"visit_time\": \"2024-01-13T12:00:00Z\"\n    },\n    {\n      \"store_id\": \"RP00002\",\n      \"image_url\": [\"https://example.com/image3.jpg\"],\n      \"visit_time\": \"2024-01-13T12:30:00Z\"\n    }\n  ]\n}\n```\n\n**Success Response** (201 Created):\n\n```json\n{\n  \"job_id\": \"job_1234567890\"\n}\n```\n\n**Error Response** (400 Bad Request):\n\n```json\n{\n  \"error\": \"count does not match number of visits\"\n}\n```\n\n### 2. Get Job Status\n\nCheck the status of a submitted job.\n\n**Endpoint**: `GET /api/status?jobid=\u003cjob_id\u003e`\n\n**Success Response** (200 OK):\n\n```json\n{\n  \"status\": \"completed\",\n  \"job_id\": \"job_1234567890\"\n}\n```\n\n**Failed Job Response** (200 OK):\n\n```json\n{\n  \"status\": \"failed\",\n  \"job_id\": \"job_1234567890\",\n  \"error\": [\n    {\n      \"store_id\": \"RP00001\",\n      \"error\": \"store not found\"\n    }\n  ]\n}\n```\n\n**Error Response** (400 Bad Request):\n\n```json\n{\n  \"error\": \"job not found\"\n}\n```\n\n## Testing\n\n### Automated Testing\n\nRun the test script:\n\n```bash\n# Make script executable\nchmod +x test_api.sh\n\n# Run tests\n./test_api.sh\n```\n\nThe test script checks:\n\n1. Valid job submission\n2. Job status retrieval\n3. Invalid store ID handling\n4. Count mismatch validation\n5. Invalid job ID handling\n\n### Manual Testing\n\nUse curl commands:\n\n```bash\n# Submit a job\ncurl -X POST http://localhost:8080/api/submit \\\n-H \"Content-Type: application/json\" \\\n-d '{\n    \"count\": 2,\n    \"visits\": [\n        {\n            \"store_id\": \"RP00001\",\n            \"image_url\": [\n                \"https://www.gstatic.com/webp/gallery/2.jpg\",\n                \"https://www.gstatic.com/webp/gallery/3.jpg\"\n            ],\n            \"visit_time\": \"2024-01-13T12:00:00Z\"\n        },\n        {\n            \"store_id\": \"RP00002\",\n            \"image_url\": [\n                \"https://www.gstatic.com/webp/gallery/3.jpg\"\n            ],\n            \"visit_time\": \"2024-01-13T12:30:00Z\"\n        }\n    ]\n}'\n\n# Check job status (replace with actual job_id)\ncurl \"http://localhost:8080/api/status?jobid=job_1234567890\"\n```\n\n## Development Environment\n\n- **Operating System**: macOS/Linux\n- **Language**: Go 1.21\n- **IDE**: VSCode with Go extension\n- **Dependencies**:\n  - gorilla/mux: HTTP router\n  - rs/cors: CORS middleware\n  - Docker \u0026 Docker Compose\n  - jq: JSON processor for testing\n\n## Future Improvements\n\nGiven more time, the following improvements could be made:\n\n1. **Infrastructure**:\n\n   - Add database persistence for job status and results\n   - Implement redis for job queue management\n   - Set up CI/CD pipeline\n\n2. **Features**:\n\n   - Add authentication/authorization\n   - Implement rate limiting\n   - Add API versioning\n   - Add metrics collection\n\n3. **Code Quality**:\n   - Add comprehensive unit tests\n   - Add integration tests\n   - Implement input validation middleware\n   - Add API documentation (PostMan/Swagger/OpenAPI)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubho004%2Fretail-pulse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsubho004%2Fretail-pulse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubho004%2Fretail-pulse/lists"}