{"id":24930526,"url":"https://github.com/mazrean/formstream","last_synced_at":"2025-07-27T04:05:42.131Z","repository":{"id":216037369,"uuid":"740239783","full_name":"mazrean/formstream","owner":"mazrean","description":"A Fast Streaming Parser for multipart/form-data in Golang","archived":false,"fork":false,"pushed_at":"2025-02-08T07:02:19.000Z","size":272,"stargazers_count":67,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-19T01:12:35.060Z","etag":null,"topics":["go","multipart","multipart-formdata","parser"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mazrean.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-07T23:05:58.000Z","updated_at":"2025-02-08T07:02:21.000Z","dependencies_parsed_at":"2024-03-05T17:45:30.838Z","dependency_job_id":"4d98b022-a7b6-4faa-9e7d-02548132bf28","html_url":"https://github.com/mazrean/formstream","commit_stats":null,"previous_names":["mazrean/conform","mazrean/formstream"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazrean%2Fformstream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazrean%2Fformstream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazrean%2Fformstream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mazrean%2Fformstream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mazrean","download_url":"https://codeload.github.com/mazrean/formstream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240783138,"owners_count":19856780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","multipart","multipart-formdata","parser"],"created_at":"2025-02-02T13:54:47.332Z","updated_at":"2025-02-26T03:15:58.721Z","avatar_url":"https://github.com/mazrean.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FormStream\n\n[![GitHub release](https://img.shields.io/github/release/mazrean/formstream.svg)](https://github.com/mazrean/formstream/releases/)\n![CI main](https://github.com/mazrean/formstream/actions/workflows/ci.yaml/badge.svg)\n[![codecov](https://codecov.io/gh/mazrean/formstream/branch/main/graph/badge.svg)](https://codecov.io/gh/mazrean/formstream)\n[![Go Reference](https://pkg.go.dev/badge/github.com/mazrean/formstream.svg)](https://pkg.go.dev/github.com/mazrean/formstream)\n\nFormStream is a Golang streaming parser for multipart data, primarily used in web form submissions and file uploads.\n\n## Features\n\n- Provides a streaming parser, eliminating the need to store entire files in memory or on disk in most cases.\n- Boasts extremely low memory usage.\n- Delivers high performance, significantly faster than traditional methods.\n\n## Benchmarks\n\nAcross all file sizes, FormStream outperforms the [`mime/multipart`](https://pkg.go.dev/mime/multipart) in both speed and memory efficiency.\n\n\u003cdetails\u003e\n\u003csummary\u003eTesting Environment\u003c/summary\u003e\n\n- OS: Ubuntu 22.04.2 LTS(WSL2 on Windows 11 Home)\n- CPU: AMD Ryzen 9 7950X 16-Core Processor\n- RAM: 32GB\n- Disk: 512GB\n- Go version: 1.22.0\n\u003c/details\u003e\n\n![](./docs/images/memory.png)\n![](./docs/images/time.png)\n\n\u003e [!NOTE]\n\u003e FormStream excels in speed by employing a stream for parsing multipart data that meets specific conditions (as shown in the `FastPath` on the graph).\n\u003e It remains significantly efficient even under less ideal conditions (`SlowPath` on the graph), marginally outperforming [`mime/multipart`](https://pkg.go.dev/mime/multipart).\n\u003e For more details, see [Technical Overview](./#technical-overview).\n\n## Installation\n\n```sh\ngo get github.com/mazrean/formstream@latest\n```\n\n## Usage\n\n### Basic Usage\n\n\u003cdetails\u003e\n\u003csummary\u003eExample Data\u003c/summary\u003e\n\n```text\n--boundary\nContent-Disposition: form-data; name=\"name\"\n\nmazrean\n--boundary\nContent-Disposition: form-data; name=\"password\"\n\npassword\n--boundary\nContent-Disposition: form-data; name=\"icon\"; filename=\"icon.png\"\nContent-Type: image/png\n\nicon contents\n--boundary--\n```\n\u003c/details\u003e\n\n```go\nparser, err := formstream.NewParser(r)\nif err != nil {\n    return err\n}\n\nerr = parser.Register(\"icon\", func(r io.Reader, header formstream.Header) error {\n    name, _, _ := parser.Value(\"name\")\n    password, _, _ := parser.Value(\"password\")\n\n    return saveUser(r.Context(), name, password, r)\n}, formstream.WithRequiredPart(\"name\"), formstream.WithRequiredPart(\"password\"))\nif err != nil {\n    return err\n}\n\nerr = parser.Parse()\nif err != nil {\n    return err\n}\n```\n\n### Integration with Web Frameworks\n\nFormStream offers wrappers for popular web frameworks:\n\n|Framework|Integration Package|\n|-|-|\n|[net/http](https://pkg.go.dev/net/http)|[httpform](./http)|\n|[Echo](https://echo.labstack.com/)|[echoform](./echo)|\n|[Gin](https://gin-gonic.com/)|[ginform](./gin)|\n\n## Technical Overview\n\nFormStream introduces a more efficient method for processing multipart data. \n\n### Understanding Multipart Data\n\nMultipart data is organized with defined boundaries separating each segment. Here's an example:\n\n```text\n--boundary\nContent-Disposition: form-data; name=\"description\"\n\nfile description\n--boundary\nContent-Disposition: form-data; name=\"file\"; filename=\"large.png\"\nContent-Type: image/png\n\nlarge png data...\n--boundary--\n```\n\nFor large files, streaming the data is vital for efficient memory usage. In the example above, streaming is made possible by sequentially processing each part from the beginning, which can be achieved using the [`(*Reader).NextPart`](https://pkg.go.dev/mime/multipart#Reader.NextPart) method in the [`mime/multipart`](https://pkg.go.dev/mime/multipart) package.\n\n### Alternative Parsing Method\n\nThe [`mime/multipart`](https://pkg.go.dev/mime/multipart) package also includes the [`(*Reader).ReadForm`](https://pkg.go.dev/mime/multipart#Reader.ReadForm) method. Unlike streaming, this method stores data temporarily in memory or on a file, leading to slower processing. It's widely used in frameworks like `net/http`, `Echo`, and `Gin` due to its ability to handle parts in any order. For instance:\n\n```text\n--boundary\nContent-Disposition: form-data; name=\"file\"; filename=\"large.png\"\nContent-Type: image/png\n\nlarge png data...\n--boundary\nContent-Disposition: form-data; name=\"description\"\n\nfile description\n--boundary--\n```\n\nWith [`(*Reader).NextPart`](https://pkg.go.dev/mime/multipart#Reader.NextPart), processing strictly follows sequential order, making it challenging to handle such data where later parts contain information necessary for processing earlier ones.\n\n### Efficient Processing Strategies\n\nOptimal multipart handling strategies include:\n- Stream processing with [`(*Reader).NextPart`](https://pkg.go.dev/mime/multipart#Reader.NextPart) when all necessary data is immediately available.\n- Temporarily storing data on disk or memory, then processing it with [`(*Reader).ReadForm`](https://pkg.go.dev/mime/multipart#Reader.ReadForm) when needed.\n\n### Advantages of FormStream\n\nFormStream enhances this process. It outpaces the [`(*Reader).ReadForm`](https://pkg.go.dev/mime/multipart#Reader.ReadForm) method and, unlike [`(*Reader).NextPart`](https://pkg.go.dev/mime/multipart#Reader.NextPart), can handle multipart data in any order. This adaptability makes FormStream suitable for a range of multipart data scenarios.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazrean%2Fformstream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmazrean%2Fformstream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmazrean%2Fformstream/lists"}