https://github.com/ckshitij/file-mgmt-srv
Service used to centrally store the different type media like images, video audio etc and can be accessible using the endpoints. Initally it is going to support the Upload and Download functionality using the APIs to access the resource.
https://github.com/ckshitij/file-mgmt-srv
go-kit mongodb-database upload-file
Last synced: 3 months ago
JSON representation
Service used to centrally store the different type media like images, video audio etc and can be accessible using the endpoints. Initally it is going to support the Upload and Download functionality using the APIs to access the resource.
- Host: GitHub
- URL: https://github.com/ckshitij/file-mgmt-srv
- Owner: ckshitij
- Created: 2025-06-11T15:57:14.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-06-23T07:09:01.000Z (3 months ago)
- Last Synced: 2025-07-14T13:33:39.840Z (3 months ago)
- Topics: go-kit, mongodb-database, upload-file
- Language: Go
- Homepage:
- Size: 31.3 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: ReadMe.md
Awesome Lists containing this project
README
# File Management Service
Service used to centrally store the differnt type media files like images, video audio etc and can be accessible using the endpoints.
Initally it is going to support the Upload and Download functionality using `mongo DB grid-fs` and `go-kit` to access the resource.
## Service Usage
- Clone the repository
```
git clone https://github.com/ckshitij/file-mgmt-srv.git
```
- Intitialize the project
```
go mod tidy
```
- Change the config under `resoure/config.yml` like mongoDB URL.
- Run the service
```
go run main.go
```
- Access the UI on ```http://localhost:8088/```
### Docker Compose
- Directly run docker-compose it will boot up the below services:
- mongo-db
- mongo-express
- ```http://localhost:8081/```
- user: admin
- password: pass
- file-mgmt-srv
- Run docker-compose command
```
docker-compose up --build -d
```
- Shutdown the docker-compose
```
docker-compose down
```
## About
- Upload file workflow has the below steps:
- Initialize the Upload process
- Generate a session ID
- Create a metadata in uploads collection
- Chunk Size (defined by the client)
- Total Chunks [file size/ divide by chunk size]
- ID [Session ID unique]
- Status : in-progress
- Uploads Chunks: Used add the chunks ids which are processed
- FileName
- Upload Chunks
- Chunks are store in the filesystem to under `tmp_uploads/sessionID` folders, just as a staging which can be used in case of partial completion.
- Upload Chunks request consist of below files:
- Chunk ID
- Session ID
- Data stream
- Create the file with `chunkID.chunk` under the `tmp_uploads/sessionID`.
- Complete Upload Status
- Once the complete api called check metadata whether all the chunks are uploaded, if yes then upload the data into `grid-fs bucket`.
- By default `grid-fs` stores file in to parts, `fs.chunks` and `fs.files` but since I change the bucket name to `uploads`.
- `uploads.chunks` stores main file contents below are the fields:
- _id
- file_id (unique id generated while upload the end data)
- n (chunk id)
- data (file data)
- `uploads.files` store the metadata about the file, which below fields:
- _id
- length [size of data uploaded in bytes]
- chunk size (default is 261120 bytes or 255 KB)
- UploadDate
- fileName
- Download a file by name