{"id":19558719,"url":"https://github.com/threeaccents/mahi","last_synced_at":"2025-04-26T23:32:07.484Z","repository":{"id":49272606,"uuid":"266191042","full_name":"threeaccents/mahi","owner":"threeaccents","description":"HTTP service for file uploading, processing, serving, and storage.","archived":false,"fork":false,"pushed_at":"2023-02-25T06:06:15.000Z","size":75820,"stargazers_count":38,"open_issues_count":10,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-26T11:17:00.333Z","etag":null,"topics":["chunk-upload","file-transform","file-upload","golang","image-processing","libvips"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/threeaccents.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-22T19:25:16.000Z","updated_at":"2023-08-31T19:06:33.000Z","dependencies_parsed_at":"2023-02-01T08:46:06.720Z","dependency_job_id":null,"html_url":"https://github.com/threeaccents/mahi","commit_stats":{"total_commits":154,"total_committers":3,"mean_commits":"51.333333333333336","dds":0.03246753246753242,"last_synced_commit":"62da8b5e42b40e05621c41f1066a8070150acfd8"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threeaccents%2Fmahi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threeaccents%2Fmahi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threeaccents%2Fmahi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threeaccents%2Fmahi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/threeaccents","download_url":"https://codeload.github.com/threeaccents/mahi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251068039,"owners_count":21531475,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chunk-upload","file-transform","file-upload","golang","image-processing","libvips"],"created_at":"2024-11-11T04:48:00.753Z","updated_at":"2025-04-26T23:32:07.084Z","avatar_url":"https://github.com/threeaccents.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"mahi-gopher.png\" height=\"300\" /\u003e\n\n# Mahi [![Go Report Card](http://goreportcard.com/badge/threeaccents/mahi)](https://goreportcard.com/report/threeaccents/mahi) [![GoDoc](http://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/threeaccents/mahi)\n\n\nMahi is an all-in-one HTTP service for file uploading, processing, serving, and storage. Mahi supports chunked, resumable, and concurrent uploads. Mahi uses Libvips behind the scenes making it extremely fast and memory efficient.\n\nMahi currently supports any s3 compatible storage, which includes (AWS s3, DO Spaces, Wasabi, Backblaze B2). The specific storage engine can be set when creating an [application](https://github.com/threeaccents/mahi#applications).\n\nMahi supports different databases for storing file meta-data and analytics. Currently, the 2 supported databased are PostgreSQL and BoltDB. The database of choice can be provided via the config file.\n\n## Features\n - [Uploads](https://github.com/threeaccents/mahi#uploads) upload files easily via our Web API.\n - [Chunked, Concurrent, Resumable Uploads](https://github.com/threeaccents/mahi#large-file-uploads) upload large files by chunking them and uploading them to our Web API.\n - [Flexible File Storage](https://github.com/threeaccents/mahi#applications) store your files in S3, Spaces, Wasabi, B2 with more options coming soon.\n - [Image Processing](https://github.com/threeaccents/mahi#file-transformations) resize, convert, and crop with ease.\n - [Usage Analytics](https://github.com/threeaccents/mahi#stats) get insights on how many transformations, how much storage and bandwidth you are processing.\n\n## Install\nLibvips must be installed on your machine. \n### Ubuntu\n```bash\nsudo apt install libvips libvips-dev libvips-tools\n```\n### MacOS\n```bash\nbrew install vips\n```\nFor other systems check out instructions [here](https://github.com/libvips/libvips/wiki#building-and-installing).\n\nInstalling mahid server.\n```bash\ngo get -u github.com/threeaccents/mahi/...\n```\nThis will install the `mahid` command in your `$GOPATH/bin` folder.\n## Usage\n```bash\nmahid -config=/path/to/config.toml\n```\nIf no config is passed Mahi will look for a `mahi.toml` file in the current directory.\n## Applications\nMahi has the concept of applications. Each application houses specific files and the storage engine for those files. This makes Mahi extremely flexible to use for different projects. If on one project you decide to use s3 as your storage engine and another DO Spaces, Mahi easily handles it for you.\n\nApplications can be created via our [Web API](https://mahi-api-docs.threeaccents.com/#req_10641a46be544cae978fa83b6fe1f00e).\n## Uploads\nFiles are uploaded to Mahi via `multipart/form-data` requests. Along with passing in the file data, you must also provide the `application_id`.\nMahi will handle processing and storing the file blob in the application's storage engine along with storing the file meta-data in the database.\nTo view an example upload response check out the [Web API](https://mahi-api-docs.threeaccents.com/#req_25f7dce3e796456e9f80ce43deba705b)\n## Large File Uploads\nWhen dealing with large files, it is best to split the file into small chunks and upload each chunk separately. Mahi easily handles chunked uploads storing each chunk and then re-building the whole file. Once the whole file is re-built Mahi uploads the file to the application's storage engine.\nTo view an example upload response check out the [Web API](https://mahi-api-docs.threeaccents.com/#req_649a25397026402b82397975292fbc4f)\n\nOther benefits of chunking up files are the ability to resume uploads and uploading multiple chunks concurrently. Mahi handles both scenarios for you with ease.\n## File Transformations (More Coming Soon)\nMahi supports file transformations via URL query params. Currently, the supported operations are:\n - Resize (width, height) `?width=100\u0026height=100`\n - Smart Crop `?crop=true`\n - Flip `?flip=true`\n - Flop `?flop=true`\n - Zoom `?zoom=2`\n - Black and White `?bw=true`\n - Quality(JPEG), Compression(PNG) `?quality=100` `?compression=10`\n - Format conversion `format is based on the file extension. To transform a png to webp, just use the .webp extension.`\n\nAll queries can be used together. For example, to resize the width, make the image black and white, and change the format to webp the params would look like this:\n```\nhttps://yourdomain.com/myimage.webp?width=100\u0026bw=true\n```\n## Stats\nMahi currently tracks these stats for both specific applications and the service as a hole:\n - Transformations: Total transformations\n - Unique Transformations: Unique transformations per file. \n - Bandwidth: Bytes served.\n - Storage: Bytes stored.\n - File Count: Total files.\n\nThese stats can be retrieved via our [Web API](https://mahi-api-docs.threeaccents.com/#req_4971b4c6a7854cad87e45d2150c7db64).\n## Config\nMahi's is configured via a toml file. Here are toml [config examples](https://github.com/threeaccents/mahi/tree/master/examples/config). Configuration options include:\n - ***db_engine:string***(default: bolt) The main database for mahi. Valid options are `postgres` and `bolt`. This is not to be confused with the storage engine. Storage engine is set per application via the [Web API](https://mahi-api-docs.threeaccents.com/#req_10641a46be544cae978fa83b6fe1f00e).\n - **http**\n    - ***port:int***(default: 4200) the port to run mahi on.\n    - ***https:boolean***(default: false) configures server to accept https requests.\n    - ***ssl_cert_path:string*** path to ssl certificate. Only required if `https` is set to true.\n    - ***ssl_key_path:string*** path to ssl key. Only required if `https` is set to true.\n - **security**\n    - ***auth_token:string*** token for authenticating requests\n    - ***aes_key:string*** key for use with AES-256 encryption. This is used to encrypt storage secrets.\n - **upload**\n    - ***chunk_upload_dir:string***(default: ./data/chunks) directory for storing chunks while an upload is happening. Once an upload is completed, the chunks are deleted.\n    - ***full_file_dir:string***(default: ./data/files) full_files are temp files used while building chunks or downloading files from the storage engine. These temp files are removed once the request is completed.\n    - ***max_chunk_size:int64***(default: 10MB) max size of a file chunk in bytes.\n    - ***max_file_size_upload:int64***(default: 50MB) max size of a file for a regular upload in bytes.\n    - ***max_transform_file_size:int64***(default: 50MB) max size of a file that can be transformed in bytes.\n - **bolt(only used if `db_engine` is set to bolt**\n    - ***dir:string***(default: ./data/mahi/mahi.db) directory for bolt db file.\n - **postgresql(only used if `db_engine` is set to postgres)**\n    - ***database:string***(default: mahi) name of database.\n    - ***host:string***(default: localhost) host of database.\n    - ***port:int***(default: 5432)  port of database.\n    - ***user:string***(default: mahi) username of database.\n    - ***password:string***(default: ) password of database.   \n    - ***max_conns:int***(default: 10 connections per CPU) maximum connections for database pool.   \n## Postgres\nTo use Postgres the necessary data tables must be created. SQL files are located in the [migrations](https://github.com/threeaccents/mahi/tree/master/cmd/migrations) folder. In the future, Mahi will come with a migrate command that will automatically handle creating the necessary tables for you. For now, you have 2 options. Install [tern](https://github.com/jackc/tern), `cd` into the migrations folder, and run `tern migrate`. The second option is just to copy and paste the SQL provided directly in a GUI or command line instance of Postgres.\n     \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreeaccents%2Fmahi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreeaccents%2Fmahi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreeaccents%2Fmahi/lists"}