{"id":18486315,"url":"https://github.com/maxgfr/huge-async-storage","last_synced_at":"2026-02-08T00:10:18.289Z","repository":{"id":64162993,"uuid":"573838342","full_name":"maxgfr/huge-async-storage","owner":"maxgfr","description":"A wrapper of async-storage that allows you to store huge data on react-native","archived":false,"fork":false,"pushed_at":"2024-10-25T19:13:06.000Z","size":1598,"stargazers_count":1,"open_issues_count":7,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-26T15:12:38.210Z","etag":null,"topics":["async-storage","asyncstorage","huge-data","local-storage","react-native","react-native-async-storage","storage","typescript","wrapper"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/huge-async-storage","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maxgfr.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-03T15:40:59.000Z","updated_at":"2024-10-25T19:12:26.000Z","dependencies_parsed_at":"2023-09-27T00:21:48.527Z","dependency_job_id":"79d9461a-c53e-4c68-8232-3e19d0b1773c","html_url":"https://github.com/maxgfr/huge-async-storage","commit_stats":{"total_commits":214,"total_committers":3,"mean_commits":71.33333333333333,"dds":0.09345794392523366,"last_synced_commit":"bfd6b68bd99752cb5b106ed664e3a774f23be139"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":"maxgfr/rn-simple-modal","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxgfr%2Fhuge-async-storage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxgfr%2Fhuge-async-storage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxgfr%2Fhuge-async-storage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxgfr%2Fhuge-async-storage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maxgfr","download_url":"https://codeload.github.com/maxgfr/huge-async-storage/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223343334,"owners_count":17129947,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-storage","asyncstorage","huge-data","local-storage","react-native","react-native-async-storage","storage","typescript","wrapper"],"created_at":"2024-11-06T12:48:59.870Z","updated_at":"2026-02-08T00:10:18.282Z","avatar_url":"https://github.com/maxgfr.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# huge-async-storage\n\nA robust wrapper around [React Native's AsyncStorage](https://react-native-async-storage.github.io/async-storage/) that enables storage of large data exceeding typical size limitations by intelligently chunking the data.\n\n## Features\n\n- **Store unlimited data size**: Automatically splits data into 1MB chunks\n- **TypeScript support**: Full type safety with generics\n- **Promise-based API**: Modern async/await syntax\n- **Error handling**: Comprehensive error messages for debugging\n- **Idempotent operations**: Safe to call multiple times\n- **Cleanup on failure**: Automatically removes partial data on storage errors\n- **Null-safety**: Handles missing or corrupted data gracefully\n\n## Installation\n\n```sh\nyarn add huge-async-storage\n# or\nnpm install huge-async-storage\n```\n\n## Quick Start\n\n```tsx\nimport { storeAsync, getAsync, removeAsync } from \"huge-async-storage\";\n\n// Store large data (automatically chunked)\nawait storeAsync('users', { list: Array(1000000).fill({ id: 1, name: 'John' }) });\n\n// Retrieve the data\nconst users = await getAsync('users');\n\n// Remove the data\nawait removeAsync('users');\n```\n\n## API Reference\n\n### `storeAsync\u003cT\u003e(key: string, data: T): Promise\u003cvoid\u003e`\n\nStores data by splitting it into manageable chunks and saving them to AsyncStorage.\n\n#### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `key` | `string` | Yes | The storage key. Must be non-empty and not whitespace-only. |\n| `data` | `T` | Yes | The data to store. Will be JSON serialized. |\n\n#### Throws\n\n- `Error` - If key is empty or whitespace-only\n- `Error` - If data cannot be serialized (e.g., circular references)\n- `Error` - If storage operation fails (with automatic cleanup of partial data)\n\n#### Side Effects\n\n- Creates multiple keys in AsyncStorage: `key0`, `key1`, ..., `keyN` for data chunks\n- Stores chunk count under the original `key`\n- On failure, removes all partially written chunks\n\n#### Examples\n\n```typescript\n// Simple object\nawait storeAsync('user', { name: 'Alice', age: 30 });\n\n// Large array (will be chunked automatically)\nconst largeData = { items: Array(1000000).fill('data') };\nawait storeAsync('large', largeData);\n\n// Null values are supported\nawait storeAsync('settings', null);\n\n// Complex nested objects\nawait storeAsync('config', {\n  database: { host: 'localhost', port: 5432 },\n  features: ['auth', 'logging', 'caching']\n});\n```\n\n---\n\n### `getAsync\u003cT\u003e(key: string): Promise\u003cT\u003e`\n\nRetrieves and reconstructs data that was stored using `storeAsync`.\n\n#### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `key` | `string` | Yes | The storage key to retrieve. |\n\n#### Returns\n\n`Promise\u003cT\u003e` - The reconstructed data with the original type.\n\n#### Throws\n\n- `Error` - If key is empty or whitespace-only\n- `Error` - If no data exists for the given key\n- `Error` - If chunk count is invalid or corrupted\n- `Error` - If any chunk is missing (storage corruption)\n- `Error` - If data cannot be parsed as valid JSON\n\n#### Examples\n\n```typescript\n// With explicit type\ninterface User {\n  name: string;\n  age: number;\n  active: boolean;\n}\n\nconst user = await getAsync\u003cUser\u003e('user');\nconsole.log(user.name); // Type-safe access\n\n// Type inference\nconst settings = await getAsync\u003c{ theme: 'light' | 'dark' }\u003e('settings');\n\n// Array types\nconst items = await getAsync\u003cnumber[]\u003e('numbers');\nitems.map(n =\u003e n * 2); // Type-safe operations\n```\n\n---\n\n### `removeAsync(key: string): Promise\u003cvoid\u003e`\n\nRemoves all chunks and metadata associated with a key.\n\n#### Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `key` | `string` | Yes | The storage key to remove. |\n\n#### Throws\n\n- `Error` - If key is empty or whitespace-only\n- `Error` - If chunk count is invalid\n- `Error` - If removal operation fails\n\n#### Side Effects\n\n- Removes all chunk keys (`key0`, `key1`, ..., `keyN`)\n- Removes the metadata key\n\n#### Behavior\n\n- **Idempotent**: Calling `removeAsync` multiple times on the same key is safe\n- **Silent success**: If the key doesn't exist, the operation succeeds without error\n\n#### Examples\n\n```typescript\n// Remove stored data\nawait removeAsync('tempData');\n\n// Safe to call multiple times\nawait removeAsync('cache');\nawait removeAsync('cache'); // No error thrown\n\n// Non-existent keys are handled gracefully\nawait removeAsync('neverStored'); // Success, no error\n```\n\n---\n\n## How It Works\n\n### Chunking Strategy\n\n1. Data is serialized to JSON using `JSON.stringify()`\n2. The serialized string is split into chunks of 1,000,000 characters\n3. Each chunk is stored with a numeric suffix: `key0`, `key1`, `key2`, etc.\n4. The total number of chunks is stored under the original key\n\n### Example Flow\n\n```\nstoreAsync('myData', largeObject)\n  ↓\nJSON.stringify(largeObject) → '{\"items\":[...]}' (2.5M chars)\n  ↓\nSplit into chunks:\n  - myData0: 1,000,000 chars\n  - myData1: 1,000,000 chars\n  - myData2: 500,000 chars\n  ↓\nStore chunk count:\n  - myData: \"3\"\n```\n\n### Storage Layout\n\n```\nAsyncStorage Keys:\n┌─────────────────────────────────────┐\n│ myData    → \"3\" (chunk count)       │\n│ myData0   → chunk 1 (1MB)           │\n│ myData1   → chunk 2 (1MB)           │\n│ myData2   → chunk 3 (remaining)     │\n└─────────────────────────────────────┘\n```\n\n## Error Handling\n\nAll functions provide detailed error messages to help diagnose issues:\n\n```typescript\ntry {\n  await getAsync('corrupted');\n} catch (error) {\n  // Error: Storage corruption: chunk 2 of 5 missing for key \"corrupted\"\n  console.error(error.message);\n}\n```\n\n### Common Errors\n\n| Error | Cause | Solution |\n|-------|-------|----------|\n| `Storage key cannot be empty` | Empty or whitespace key | Provide a valid key |\n| `No data found for key \"x\"` | Key doesn't exist | Check if data was stored |\n| `Invalid chunk count for key \"x\"` | Corrupted metadata | Remove and re-store data |\n| `Storage corruption: chunk N of M missing` | Partial data loss | Remove and re-store data |\n| `Failed to parse data for key \"x\"` | Invalid JSON | Check data integrity |\n\n## Best Practices\n\n### 1. Use Type Guards\n\n```typescript\ninterface UserProfile {\n  id: string;\n  name: string;\n  email?: string;\n}\n\nconst user = await getAsync\u003cUserProfile\u003e('user');\nif (user?.email) {\n  sendEmail(user.email);\n}\n```\n\n### 2. Handle Errors Gracefully\n\n```typescript\nasync function loadConfig() {\n  try {\n    const config = await getAsync\u003cConfig\u003e('config');\n    return config ?? defaultConfig;\n  } catch (error) {\n    console.warn('Failed to load config, using defaults');\n    return defaultConfig;\n  }\n}\n```\n\n### 3. Clean Up Unused Data\n\n```typescript\n// Remove old data when no longer needed\nawait removeAsync('tempCache');\n```\n\n### 4. Avoid Very Large Keys\n\nShort keys are more efficient:\n```typescript\n// Good\nawait storeAsync('usr', userData);\n\n// Avoid\nawait storeAsync('veryLongKeyNameThatWastesMemory', userData);\n```\n\n## Limitations\n\n- **Chunk Size**: Fixed at 1MB per chunk for compatibility across platforms\n- **Synchronous Operations**: Each operation is atomic; concurrent writes to the same key may conflict\n- **Memory Usage**: Retrieving very large data loads everything into memory\n\n## Platform Support\n\n| Platform | Total Storage | Per-Entry Limit | Chunk Size | Notes |\n|----------|---------------|-----------------|------------|-------|\n| **iOS** | ~6MB default | ~6MB | 1MB | Safe within default limits |\n| **Android** | ~6MB default (configurable) | ~2MB (WindowCursor SQLite) | 1MB | ✅ **Below Android's 2MB per-entry limit** |\n\n### Why 1MB Chunk Size?\n\nThe 1MB chunk size is specifically designed to work within Android's **WindowCursor SQLite limit** of approximately 2MB per entry:\n\n\u003e *\"Per-entry is limited by a size of a WindowCursor, a buffer used to read data from SQLite. Currently it's size is around 2 MB.\"* — AsyncStorage Documentation\n\nBy using 1MB chunks, this library:\n- ✅ Stays safely below Android's 2MB per-entry limit\n- ✅ Allows storing data larger than the 6MB total limit through chunking\n- ✅ Works across iOS and Android without platform-specific code\n\n## License\n\nMIT","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxgfr%2Fhuge-async-storage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaxgfr%2Fhuge-async-storage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxgfr%2Fhuge-async-storage/lists"}