{"id":26906880,"url":"https://github.com/smooai/file","last_synced_at":"2026-04-24T06:03:42.562Z","repository":{"id":285254885,"uuid":"950968092","full_name":"SmooAI/file","owner":"SmooAI","description":"Multi-language file handling library (TypeScript, Python, Rust, Go) — unified interface for local filesystem, S3, URLs, and FormData with stream-first design, lazy byte loading, and intelligent type detection.","archived":false,"fork":false,"pushed_at":"2026-04-22T18:58:37.000Z","size":438,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-22T20:24:48.376Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://smoo.ai","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SmooAI.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-19T00:50:39.000Z","updated_at":"2026-04-22T18:52:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"61ffdf4a-71f0-4569-b31e-f08f65130412","html_url":"https://github.com/SmooAI/file","commit_stats":null,"previous_names":["smooai/file"],"tags_count":13,"template":false,"template_full_name":"SmooAI/library-template","purl":"pkg:github/SmooAI/file","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmooAI%2Ffile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmooAI%2Ffile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmooAI%2Ffile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmooAI%2Ffile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SmooAI","download_url":"https://codeload.github.com/SmooAI/file/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SmooAI%2Ffile/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32211386,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T03:15:14.334Z","status":"ssl_error","status_checked_at":"2026-04-24T03:15:11.608Z","response_time":64,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-01T11:20:20.171Z","updated_at":"2026-04-24T06:03:42.556Z","avatar_url":"https://github.com/SmooAI.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- Improved compatibility of back to top link: See: https://github.com/othneildrew/Best-README-Template/pull/73 --\u003e\n\n\u003ca name=\"readme-top\"\u003e\u003c/a\u003e\n\n\u003c!--\n*** Thanks for checking out the Best-README-Template. If you have a suggestion\n*** that would make this better, please fork the repo and create a pull request\n*** or simply open an issue with the tag \"enhancement\".\n*** Don't forget to give the project a star!\n*** Thanks again! Now go create something AMAZING! :D\n--\u003e\n\n\u003c!-- PROJECT SHIELDS --\u003e\n\u003c!--\n*** I'm using markdown \"reference style\" links for readability.\n*** Reference links are enclosed in brackets [ ] instead of parentheses ( ).\n*** See the bottom of this document for the declaration of the reference variables\n*** for contributors-url, forks-url, etc. This is an optional, concise syntax you may use.\n*** https://www.markdownguide.org/basic-syntax/#reference-style-links\n--\u003e\n\n\u003c!-- PROJECT LOGO --\u003e\n\u003cbr /\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://smoo.ai\"\u003e\n    \u003cimg src=\"images/logo.png\" alt=\"SmooAI Logo\" /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n\u003c!-- ABOUT THE PROJECT --\u003e\n\n## About Smoo AI\n\n**[Smoo AI](https://smoo.ai)** is an AI platform that helps businesses multiply their customer, employee, and developer experience — conversational AI for support and sales, paired with the production-grade developer tooling we use to build it.\n\nThis library is part of a small family of open-source packages we maintain to keep our own stack honest: contextual logging, typed HTTP, file storage, and agent orchestration. Use them in your stack, or take them as a reference for how we build.\n\n- 🌐 [smoo.ai](https://smoo.ai) — the product\n- 📦 [smoo.ai/open-source](https://smoo.ai/open-source) — every open-source package we ship\n- 🐙 [github.com/SmooAI](https://github.com/SmooAI) — the source\n\n## About @smooai/file\n\nA powerful file handling library for Node.js that provides a unified interface for working with files from local filesystem, S3, URLs, and more. Built with streaming in mind, it handles file bytes lazily where possible to minimize memory usage and improve performance.\n\n![NPM Version](https://img.shields.io/npm/v/%40smooai%2Ffile?style=for-the-badge)\n![NPM Downloads](https://img.shields.io/npm/dw/%40smooai%2Ffile?style=for-the-badge)\n![NPM Last Update](https://img.shields.io/npm/last-update/%40smooai%2Ffile?style=for-the-badge)\n\n![GitHub License](https://img.shields.io/github/license/SmooAI/file?style=for-the-badge)\n![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/SmooAI/file/release.yml?style=for-the-badge)\n![GitHub Repo stars](https://img.shields.io/github/stars/SmooAI/file?style=for-the-badge)\n\n### Install\n\n```sh\npnpm add @smooai/file\n```\n\n### Multi-Language Support\n\n@smooai/file is available as native implementations in **TypeScript**, **Python**, **Rust**, **Go**, and **.NET (C#)** — each built with idiomatic patterns for its ecosystem.\n\n| Language    | Package                                                           | Install                                 |\n| ----------- | ----------------------------------------------------------------- | --------------------------------------- |\n| TypeScript  | [`@smooai/file`](https://www.npmjs.com/package/@smooai/file)      | `pnpm add @smooai/file`                 |\n| Python      | [`smooai-file`](https://pypi.org/project/smooai-file/)            | `pip install smooai-file`               |\n| Rust        | [`smooai-file`](https://crates.io/crates/smooai-file)             | `cargo add smooai-file`                 |\n| Go          | `github.com/SmooAI/file/go/file`                                  | `go get github.com/SmooAI/file/go/file` |\n| .NET (core) | [`SmooAI.File`](https://www.nuget.org/packages/SmooAI.File)       | `dotnet add package SmooAI.File`        |\n| .NET (S3)   | [`SmooAI.File.S3`](https://www.nuget.org/packages/SmooAI.File.S3) | `dotnet add package SmooAI.File.S3`     |\n\nLanguage-specific source code lives in the [`python/`](./python/), [`rust/`](./rust/), [`go/`](./go/), and [`dotnet/`](./dotnet/) directories.\n\nThe .NET port uses [Mime-Detective](https://github.com/MediatedCommunications/Mime-Detective) for magic-byte MIME sniffing and splits S3 helpers into a sub-package so consumers who don't need AWS avoid pulling in the AWS SDK.\n\n### Key Features\n\n#### 🚀 Stream-First Design\n\n- Lazy loading of file contents\n- Memory-efficient processing\n- Automatic stream handling\n- Support for both Node.js and Web streams\n\n#### 📦 Multiple File Sources\n\n- **Local Filesystem**\n    - Read and write operations\n    - File system checks\n    - Metadata extraction\n\n- **URLs**\n    - Automatic download\n    - Stream-based transfer\n    - Header metadata extraction\n\n- **S3 Objects**\n    - Direct S3 integration (download and upload)\n    - Stream-based transfer\n    - Header metadata extraction\n    - Signed URL generation\n\n- **FormData**\n    - Ease of use for Multipart FormData upload\n\n#### 🔍 Intelligent File Type Detection\n\nPowered by [file-type](https://github.com/sindresorhus/file-type), providing:\n\n- Magic number detection\n- MIME type inference\n- File extension detection\n- Support for 100+ file types\n\n#### 📝 Rich Metadata\n\n- File name and extension\n- MIME type detection\n- File size\n- Last modified date\n- Creation date\n- File hash/checksum\n- URL and path information\n- Source type\n\n### Examples\n\n- [Basic Usage](#basic-usage)\n- [Streaming Operations](#streaming-operations)\n- [S3 Integration](#s3-integration)\n- [File Type Detection](#file-type-detection)\n- [FormData Support](#formdata-support)\n- [Web File / Blob (Hono, Next.js, Browser)](#web-file)\n- [Validation (size, mime, content-vs-claim)](#validation)\n- [Base64 Encoding](#base64)\n- [Presigned Upload URL](#presigned-upload)\n\n#### Basic Usage \u003ca name=\"basic-usage\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\n// Create a file from a local path\nconst file = await File.createFromFile('path/to/file.txt');\n\n// Read file contents (streams automatically)\nconst content = await file.readFileString();\nconsole.log(content);\n\n// Get file metadata\nconsole.log(file.metadata);\n// {\n//   name: 'file.txt',\n//   mimeType: 'text/plain',\n//   size: 1234,\n//   extension: 'txt',\n//   path: 'path/to/file.txt',\n//   lastModified: Date,\n//   createdAt: Date\n// }\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### Streaming Operations \u003ca name=\"streaming-operations\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\n// Create a file from a URL (streams automatically)\nconst file = await File.createFromUrl('https://example.com/large-file.zip');\n\n// Pipe to a destination (streams without loading entire file)\nawait file.pipeTo(someWritableStream);\n\n// Read as bytes (streams in chunks)\nconst bytes = await file.readFileBytes();\n\n// Save to filesystem (streams directly)\nconst { original, newFile } = await file.saveToFile('downloads/file.zip');\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### S3 Integration \u003ca name=\"s3-integration\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\n// Create from S3 (streams automatically)\nconst file = await File.createFromS3('my-bucket', 'path/to/file.jpg');\n\n// Upload to S3 (streams directly)\nawait file.uploadToS3('my-bucket', 'remote/file.jpg');\n\n// Save to S3 (creates new file instance)\nconst { original, newFile } = await file.saveToS3('my-bucket', 'remote/file.jpg');\n\n// Move to S3 (deletes local file if source was local)\nconst s3File = await file.moveToS3('my-bucket', 'remote/file.jpg');\n\n// Generate signed URL\nconst signedUrl = await s3File.getSignedUrl(3600); // URL expires in 1 hour\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### File Type Detection \u003ca name=\"file-type-detection\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\nconst file = await File.createFromFile('document.xml');\n\n// Get file type information (detected via magic numbers)\nconsole.log(file.mimeType); // 'application/xml'\nconsole.log(file.extension); // 'xml'\n\n// File type is automatically detected from:\n// - Magic numbers (via file-type)\n// - MIME type headers\n// - File extension\n// - Custom detectors\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### FormData Support \u003ca name=\"formdata-support\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\nconst file = await File.createFromFile('document.pdf');\n\n// Convert to FormData for uploads\nconst formData = await file.toFormData('document');\n\n// Use with fetch or other HTTP clients\nawait fetch('https://api.example.com/upload', {\n    method: 'POST',\n    body: formData,\n});\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### Web File / Blob (Hono, Next.js, Browser) \u003ca name=\"web-file\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\n// Hono multipart route\napp.post('/upload', async (c) =\u003e {\n    const form = await c.req.formData();\n    const webFile = form.get('file') as globalThis.File;\n\n    // Preserves the web File's name and type hints.\n    const file = await File.createFromWebFile(webFile);\n    // …validate, upload, etc.\n});\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### Validation (size, mime, content-vs-claim) \u003ca name=\"validation\"\u003e\u003c/a\u003e\n\n```typescript\nimport File, { FileValidationError } from '@smooai/file';\n\nconst file = await File.createFromWebFile(webFile);\n\ntry {\n    await file.validate({\n        maxSize: 5 * 1024 * 1024, // 5MB\n        allowedMimes: ['image/png', 'image/jpeg', 'image/webp'],\n        expectedMimeType: webFile.type, // compares magic-byte detection vs claimed Content-Type\n    });\n} catch (err) {\n    if (err instanceof FileValidationError) {\n        // FileSizeError | FileMimeError | FileContentMismatchError — map to HTTP 400\n        throw new HTTPException(400, { message: err.message });\n    }\n    throw err;\n}\n```\n\n`expectedMimeType` is the primary defense against mime-spoofing: a `.php` file uploaded with `Content-Type: image/png` will fail because magic-byte detection doesn't match the claim.\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### Base64 Encoding (email attachments, data URLs) \u003ca name=\"base64\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\nconst file = await File.createFromUrl('https://s3.example.com/invoice.pdf');\n\nawait sendEmail({\n    attachments: [\n        {\n            filename: 'invoice.pdf',\n            content: await file.toBase64(),\n            encoding: 'base64',\n        },\n    ],\n});\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n#### Presigned Upload URL (server signs, client uploads direct to S3) \u003ca name=\"presigned-upload\"\u003e\u003c/a\u003e\n\n```typescript\nimport File from '@smooai/file';\n\n// Server issues a time-limited signed URL the client uploads bytes to directly.\n// `maxSize` is baked into the signature so oversized uploads are rejected by S3.\nconst url = await File.createPresignedUploadUrl({\n    bucket: Resource.Bucket.name,\n    key: `avatars/${userId}.png`,\n    contentType: 'image/png',\n    expiresIn: 600,\n    maxSize: 2 * 1024 * 1024,\n});\n```\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#examples\"\u003eback to examples\u003c/a\u003e)\u003c/p\u003e\n\n### Built With\n\n- TypeScript\n- Node.js File System API\n- AWS SDK v3\n- [file-type](https://github.com/sindresorhus/file-type) for magic number-based MIME type detection\n- [@smooai/fetch](https://github.com/SmooAI/fetch) for URL downloads\n- [@smooai/logger](https://github.com/SmooAI/logger) for structured logging\n\n## Contributing\n\nContributions are welcome! This project uses [changesets](https://github.com/changesets/changesets) to manage versions and releases.\n\n### Development Workflow\n\n1. Fork the repository\n2. Create your branch (`git checkout -b amazing-feature`)\n3. Make your changes\n4. Add a changeset to document your changes:\n\n    ```sh\n    pnpm changeset\n    ```\n\n    This will prompt you to:\n    - Choose the type of version bump (patch, minor, or major)\n    - Provide a description of the changes\n\n5. Commit your changes (`git commit -m 'Add some amazing feature'`)\n6. Push to the branch (`git push origin feature/amazing-feature`)\n7. Open a Pull Request\n\n### Pull Request Guidelines\n\n- Reference any related issues in your PR description\n\nThe maintainers will review your PR and may request changes before merging.\n\n\u003c!-- CONTACT --\u003e\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#readme-top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n## Contact\n\nBrent Rager\n\n- [Email](mailto:brent@smoo.ai)\n- [LinkedIn](https://www.linkedin.com/in/brentrager/)\n- [BlueSky](https://bsky.app/profile/brentragertech.bsky.social)\n- [TikTok](https://www.tiktok.com/@brentragertech)\n- [Instagram](https://www.instagram.com/brentragertech/)\n\nSmoo Github: [https://github.com/SmooAI](https://github.com/SmooAI)\n\n\u003cp align=\"right\"\u003e(\u003ca href=\"#readme-top\"\u003eback to top\u003c/a\u003e)\u003c/p\u003e\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n\n[sst.dev-url]: https://reactjs.org/\n[sst]: https://img.shields.io/badge/sst-EDE1DA?style=for-the-badge\u0026logo=sst\u0026logoColor=E27152\n[sst-url]: https://sst.dev/\n[next]: https://img.shields.io/badge/next.js-000000?style=for-the-badge\u0026logo=nextdotjs\u0026logoColor=white\n[next-url]: https://nextjs.org/\n[aws]: https://img.shields.io/badge/aws-232F3E?style=for-the-badge\u0026logo=amazonaws\u0026logoColor=white\n[aws-url]: https://tailwindcss.com/\n[tailwindcss]: https://img.shields.io/badge/tailwind%20css-0B1120?style=for-the-badge\u0026logo=tailwindcss\u0026logoColor=#06B6D4\n[tailwindcss-url]: https://tailwindcss.com/\n[zod]: https://img.shields.io/badge/zod-3E67B1?style=for-the-badge\u0026logoColor=3E67B1\n[zod-url]: https://zod.dev/\n[sanity]: https://img.shields.io/badge/sanity-F36458?style=for-the-badge\n[sanity-url]: https://www.sanity.io/\n[vitest]: https://img.shields.io/badge/vitest-1E1E20?style=for-the-badge\u0026logo=vitest\u0026logoColor=#6E9F18\n[vitest-url]: https://vitest.dev/\n[pnpm]: https://img.shields.io/badge/pnpm-F69220?style=for-the-badge\u0026logo=pnpm\u0026logoColor=white\n[pnpm-url]: https://pnpm.io/\n[turborepo]: https://img.shields.io/badge/turborepo-000000?style=for-the-badge\u0026logo=turborepo\u0026logoColor=#EF4444\n[turborepo-url]: https://turbo.build/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmooai%2Ffile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmooai%2Ffile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmooai%2Ffile/lists"}