{"id":30277117,"url":"https://github.com/neon-x-hub/segmon","last_synced_at":"2026-04-08T16:31:45.010Z","repository":{"id":309624845,"uuid":"1036971946","full_name":"neon-x-hub/segmon","owner":"neon-x-hub","description":"A lightweight, high performance segmented JSON database with custom IDs, dual-size limits, and flexible filtering and pagination.","archived":false,"fork":false,"pushed_at":"2025-08-12T23:01:39.000Z","size":24,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-13T00:11:46.940Z","etag":null,"topics":["bun","database","json","lowdb","nosql","segment","storage"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/segmon","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/neon-x-hub.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-12T21:35:13.000Z","updated_at":"2025-08-12T23:01:43.000Z","dependencies_parsed_at":"2025-08-13T00:11:49.815Z","dependency_job_id":"a53d3fec-710f-4fea-86e3-a530b85a2a4c","html_url":"https://github.com/neon-x-hub/segmon","commit_stats":null,"previous_names":["neon-x-hub/segmon"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/neon-x-hub/segmon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neon-x-hub%2Fsegmon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neon-x-hub%2Fsegmon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neon-x-hub%2Fsegmon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neon-x-hub%2Fsegmon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/neon-x-hub","download_url":"https://codeload.github.com/neon-x-hub/segmon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neon-x-hub%2Fsegmon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31564867,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bun","database","json","lowdb","nosql","segment","storage"],"created_at":"2025-08-16T11:13:48.684Z","updated_at":"2026-04-08T16:31:44.996Z","avatar_url":"https://github.com/neon-x-hub.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# **Segmon** 🗃️\n[![npm](https://img.shields.io/npm/v/segmon)](https://www.npmjs.com/package/segmon)\n\n*A lightweight, high performance segmented JSON database with custom IDs, dual-size limits, and flexible filtering and pagination.*\n\nPerfect for:\n\n✅ **Small to medium projects** needing structured storage\n\n✅ **Apps where MongoDB is overkill** but filesystem storage is too basic\n\n✅ **CLI tools** requiring fast, atomic operations\n\n✅ **Prototyping** with zero dependencies\n\n---\n\n## **📑 Table of Contents**\n\n1. **[✨ Features](#-features)**\n   - Dual Segmentation\n   - Custom ID Generation\n   - Bulk Operations\n   - Flexible Queries\n   - Atomic Writes\n\n2. **[🚀 Install](#-install)**\n\n3. **[📖 Usage](#-usage)**\n   - Initialization\n   - CRUD Operations\n   - Advanced Queries\n\n4. **[🚀 Performance Benchmarks](#-performance-benchmarks)**\n   - Key Operations Comparison\n   - Runtime Highlights\n   - Benchmark Execution\n\n5. **[🛠 API Reference](#-api-reference)**\n   - Configuration Options\n   - Core Methods\n\n6. **[📜 License](#-license)**\n---\n\n## **✨ Features**\n\n- **Dual Segmentation**\n  Limit by file size (`segmentSize`) **or** item count (`maxItemsPerSegment`).\n\n- **Custom ID Generation**\n  Bring your own ID logic (UUIDs, nanoid, etc.) or use the built-in generator with custom ID length.\n\n- **Optimized Bulk Operations**\n  Insert/update/delete multiple documents with minimal I/O.\n\n- **Flexible Queries**\n  Filter with:\n  - **Ranges**: `{ age: { min: 18, max: 30 } }`\n  - **Custom logic**: `onFilter: (doc, filter) =\u003e ...`\n  - **Deep matching**: Nested object/array support.\n  - **Built-in Pagination**: `db.find(collection, {} ,{limit: 10, offset: 50})`\n\n- **Atomic Writes**\n  File-level locking prevents corruption.\n\n---\n\n## **🚀 Install**\n\n```bash\nnpm install segmon\n```\n\n---\n\n## **📖 Usage**\n\n### **1. Initialize**\n```javascript\nimport Segmon from 'segmon';\n\nconst db = new Segmon({\n  basePath: './data',           // Storage directory\n  segmentSize: 50000,           // 50KB per file (optional)\n  maxItemsPerSegment: 1000,     // OR max docs per file (optional)\n  idGenerator: customIdFn,      // Optional: () =\u003e 'your-id'\n  onFilter: customFilter        // Override default filtering\n});\n```\n\n### **2. CRUD Operations**\n```javascript\n// Create\nconst user = await db.create('users', { name: 'Alice', age: 30 });\n\n// Bulk insert\nawait db.bulkCreate('users', [...largeArray]);\n\n// Query\nconst adults = await db.find('users', { age: { min: 18 } }, { limit: 10 });\n\n// Update\nawait db.update('users', user.id, { age: 31 });\n\n// Delete\nawait db.delete('users', user.id);\n```\n\n### **3. Advanced Queries**\n```javascript\n// Custom filter function\nconst db = new Segmon({\n  onFilter: (doc, filter) =\u003e doc.tags?.includes(filter.tag)\n});\n\n// Find all docs with 'special' tag\nawait db.find('posts', { tag: 'special' });\n```\n\n---\n\n## **🚀 Performance Benchmarks**\n\nSegmon delivers exceptional performance across JavaScript runtimes, with Bun showing particularly impressive results:\n\n### **Key Operations Comparison**\n| Operation  | 100 Docs (Node/Bun)  | 10K Docs (Node/Bun)    | Bun Advantage           |\n| ---------- | -------------------- | ---------------------- | ----------------------- |\n| **Insert** | 5.11ms / **10.55ms** | 121.24ms / **96.28ms** | **20% faster** at scale |\n| **Query**  | 2.18ms / **2.30ms**  | 56.51ms / **40.82ms**  | **28% faster**          |\n| **Update** | 2.32ms / **1.45ms**  | 1.26ms / **1.12ms**    | **15-40% faster**       |\n| **Delete** | 1.91ms / **1.25ms**  | 1.04ms / **0.88ms**    | **15-35% faster**       |\n\n**✨ Runtime Highlights:**\n- **Bun dominates** at scale with 20-28% faster operations for large datasets\n- **Node.js shows** slightly better performance for small datasets (\u003c1,000 docs)\n- **Update/Delete operations** are consistently fast (\u003c2ms) in both runtimes\n\n### **What Makes These Results Fire**\n1. **10K Document Benchmark**\n   - Bun completes bulk inserts in **\u003c100ms** (vs 121ms in Node)\n   - Demonstrates Segmon's optimized segmentation system\n\n2. **Real-World Ready**\n   - Sub-2ms updates/deletes mean perfect fit for:\n     - High-traffic APIs\n     - Real-time applications\n     - Data-intensive scripts\n\n3. **Runtime Flexibility**\n```bash\n   # Works beautifully in both environments\n   node your-app.js\n   bun your-app.js\n```\n\n*Test Environment:*\n- 16GB RAM / NVMe SSD\n- Clean database state for all tests\n- Node.js v20.9.0 vs Bun v1.1.16\n\n\n### How to Run Benchmarks\n\n1. Clone the repository:\n```bash\n   git clone https://github.com/neon-x-hub/segmon.git\n   cd segmon\n```\n\n2. Run the benchmark script\n```bash\n    # or use `bun` if you want\n    node benchmark/bench.js\n    # or simply:\n    npm run bench\n```\n\n### Note:\n\nBenchmarks measure cold-start operations on a clean database. Your results may vary based on:\n• Hardware specifications (SSD vs HDD)\n• Node.js version\n• System load during testing\n\n---\n\n## **🛠 API Reference**\n\n### **`new Segmon(config)` Configuration Options**\n\n| Parameter             | Type       | Default Value       | Description                                                                 |\n|-----------------------|------------|---------------------|-----------------------------------------------------------------------------|\n| `basePath`           | `string`   | `\"./segmon-data\"`   | Base directory where collections will be stored                             |\n| `segmentSize`        | `number`   | `51200` (50KB)      | Maximum size of each segment file in bytes                                  |\n| `maxItemsPerSegment` | `number`   | `null` (unlimited)  | Maximum number of documents per segment file (overrides size limit if set)  |\n| `idGenerator`        | `function` | `generateId`        | Custom function to generate document IDs: `(idLength) =\u003e string`                    |\n| `idLength`           | `number`   | `6`                 | Length of auto-generated IDs   |\n| `onFilter`           | `function` | `null`              | Custom filter function: `(doc, filter) =\u003e boolean`                          |\n| `normaliseDocument`  | `function` | `(doc) =\u003e doc`      | Document pre-processor function applied before storage/querying             |\n\n**Notes:**\n- Either `segmentSize` or `maxItemsPerSegment` can limit segmentation\n- When both are set, whichever limit is hit first triggers new segment creation\n- `idGenerator` receives the desired ID length (idLength) and should return a unique string\n- `normaliseDocument` runs on all documents before they're filtered\n\n\n### **Core Methods**\n| Method                            | Description                             | Bulk Version                      |\n| --------------------------------- | --------------------------------------- | --------------------------------- |\n| `create(collection, data)`        | Insert a single document                | `bulkCreate(collection, array)`   |\n| `find(collection, filter, opts)`  | Query documents with optional filtering | -                                 |\n| `findById(collection, id)`        | Fetch a single document by ID           | `bulkFindByIds(collection, ids)`  |\n| `update(collection, id, changes)` | Modify a single document                | `bulkUpdate(collection, updates)` |\n| `delete(collection, id)`          | Remove a single document                | `bulkDelete(collection, ids)`     |\n\n**Key Features:**\n- **Bulk operations** are more optimized for batch processing\n- **Filtering and Pagination** automatically supported by find method\n- **Atomic writes** ensure data integrity\n- **Segment-aware** queries automatically scan all relevant files\n\n---\n\n## **📜 License**\nMIT © Memmou Abderrahmane (neon-x-hub)\n\n---\n\n**Enjoy Segmon?** ⭐️ [GitHub Repo](https://github.com/neon-x-hub/segmon) | Report issues [here](#).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneon-x-hub%2Fsegmon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneon-x-hub%2Fsegmon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneon-x-hub%2Fsegmon/lists"}