{"id":36855230,"url":"https://github.com/go9sky/pytuck","last_synced_at":"2026-01-23T19:01:17.508Z","repository":{"id":331615183,"uuid":"1131561078","full_name":"go9sky/Pytuck","owner":"go9sky","description":"A lightweight, pure Python document database with multi-engine support. No SQL required - manage your data through Python objects and methods.  ✨纯Python实现的轻量级文档数据库，支持多种存储引擎，无SQL，通过对象和方法管理数据。","archived":false,"fork":false,"pushed_at":"2026-01-16T18:33:33.000Z","size":657,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-01-16T21:36:14.206Z","etag":null,"topics":["database","orm","python","zero-dependency"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/go9sky.png","metadata":{"files":{"readme":"README.EN.md","changelog":"CHANGELOG.EN.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-10T08:57:18.000Z","updated_at":"2026-01-16T18:33:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/go9sky/Pytuck","commit_stats":null,"previous_names":["go9sky/pytuck"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/go9sky/Pytuck","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go9sky%2FPytuck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go9sky%2FPytuck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go9sky%2FPytuck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go9sky%2FPytuck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/go9sky","download_url":"https://codeload.github.com/go9sky/Pytuck/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go9sky%2FPytuck/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28698343,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T17:25:48.045Z","status":"ssl_error","status_checked_at":"2026-01-23T17:25:47.153Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","orm","python","zero-dependency"],"created_at":"2026-01-12T14:42:50.938Z","updated_at":"2026-01-23T19:01:17.498Z","avatar_url":"https://github.com/go9sky.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pytuck - Lightweight Python Document Database\n\n[![Gitee](https://img.shields.io/badge/Gitee-go9sky%2Fpytuck-red)](https://gitee.com/go9sky/pytuck)\n[![GitHub](https://img.shields.io/badge/GitHub-go9sky%2Fpytuck-blue)](https://github.com/go9sky/pytuck)\n\n[![PyPI version](https://badge.fury.io/py/pytuck.svg)](https://badge.fury.io/py/pytuck)\n[![Python Versions](https://img.shields.io/pypi/pyversions/pytuck.svg)](https://pypi.org/project/pytuck/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n[中文](README.md) | English\n\nA lightweight, pure Python document database with multi-engine support. No SQL required - manage your data through Python objects and methods.\n\n## Repository Mirrors\n\n- **GitHub**: https://github.com/go9sky/pytuck\n- **Gitee**: https://gitee.com/go9sky/pytuck\n\n## Key Features\n\n- **No SQL Required** - Work entirely with Python objects and methods\n- **Multi-Engine Support** - Binary, JSON, CSV, SQLite, Excel, XML storage formats\n- **Pluggable Architecture** - Zero dependencies by default, optional engines on demand\n- **SQLAlchemy 2.0 Style API** - Modern query builders (`select()`, `insert()`, `update()`, `delete()`)\n- **Generic Type Hints** - Complete generic support with precise IDE type inference (`List[User]` instead of `List[PureBaseModel]`)\n- **Pythonic Query Syntax** - Use native Python operators (`User.age \u003e= 18`)\n- **Index Optimization** - Hash indexes for accelerated queries\n- **Type Safety** - Automatic type validation and conversion (loose/strict modes), supports 10 field types\n- **Relationships** - Supports one-to-many and many-to-one with lazy loading + auto caching\n- **Independent Data Models** - Accessible after session close, usable like Pydantic\n- **Persistence** - Automatic or manual data persistence to disk\n\n## Quick Start\n\n### Installation\n\n```bash\n# Basic installation (binary engine only, zero dependencies)\npip install pytuck\n\n# Install specific engines\npip install pytuck[excel]   # Excel engine (requires openpyxl)\npip install pytuck[xml]     # XML engine (requires lxml)\n\n# Install all engines\npip install pytuck[all]\n\n# Development environment\npip install pytuck[dev]\n```\n\n### Basic Usage\n\nPytuck offers two usage modes:\n\n#### Mode 1: Pure Model (Default, Recommended)\n\nOperate data through Session, following SQLAlchemy 2.0 style:\n\n```python\nfrom typing import Type\nfrom pytuck import Storage, declarative_base, Session, Column\nfrom pytuck import PureBaseModel, select, insert, update, delete\n\n# Create database (default: binary engine)\ndb = Storage(file_path='mydb.db')\nBase: Type[PureBaseModel] = declarative_base(db)\n\n# Define model\nclass Student(Base):\n    __tablename__ = 'students'\n\n    id = Column('id', int, primary_key=True)\n    name = Column('name', str, nullable=False, index=True)\n    age = Column('age', int)\n    email = Column('email', str, nullable=True)\n\n# Create Session\nsession = Session(db)\n\n# Insert records\nstmt = insert(Student).values(name='Alice', age=20, email='alice@example.com')\nresult = session.execute(stmt)\nsession.commit()\nprint(f\"Created student, ID: {result.inserted_primary_key}\")\n\n# Query records\nstmt = select(Student).where(Student.id == 1)\nresult = session.execute(stmt)\nalice = result.scalars().first()\nprint(f\"Found: {alice.name}, {alice.age} years old\")\n\n# Conditional query (Pythonic syntax)\nstmt = select(Student).where(Student.age \u003e= 18).order_by('name')\nresult = session.execute(stmt)\nadults = result.scalars().all()\nfor student in adults:\n    print(f\"  - {student.name}\")\n\n# Identity Map example (0.3.0 NEW, object uniqueness guarantee)\nstudent1 = session.get(Student, 1)  # Load from database\nstmt = select(Student).where(Student.id == 1)\nstudent2 = session.execute(stmt).scalars().first()  # Get through query\nprint(f\"Same object? {student1 is student2}\")  # True, same instance\n\n# merge() operation example (0.3.0 NEW, merge external data)\nexternal_student = Student(id=1, name=\"Alice Updated\", age=22)  # External data\nmerged = session.merge(external_student)  # Intelligently merge into Session\nsession.commit()  # Update takes effect\n\n# Update records\n# Method 1: Use update statement (bulk update)\nstmt = update(Student).where(Student.id == 1).values(age=21)\nsession.execute(stmt)\nsession.commit()\n\n# Method 2: Attribute assignment update (0.3.0 NEW, more intuitive)\nstmt = select(Student).where(Student.id == 1)\nresult = session.execute(stmt)\nalice = result.scalars().first()\nalice.age = 21  # Attribute assignment auto-detected and updates database\nsession.commit()  # Automatically writes changes to database\n\n# Delete records\nstmt = delete(Student).where(Student.id == 1)\nsession.execute(stmt)\nsession.commit()\n\n# Close\nsession.close()\ndb.close()\n```\n\n#### Mode 2: Active Record\n\nModels with built-in CRUD methods for simpler operations:\n\n```python\nfrom typing import Type\nfrom pytuck import Storage, declarative_base, Column\nfrom pytuck import CRUDBaseModel\n\n# Create database\ndb = Storage(file_path='mydb.db')\nBase: Type[CRUDBaseModel] = declarative_base(db, crud=True)  # Note: crud=True\n\n# Define model\nclass Student(Base):\n    __tablename__ = 'students'\n\n    id = Column('id', int, primary_key=True)\n    name = Column('name', str, nullable=False)\n    age = Column('age', int)\n\n# Create record (auto-save)\nalice = Student.create(name='Alice', age=20)\nprint(f\"Created: {alice.name}, ID: {alice.id}\")\n\n# Or save manually\nbob = Student(name='Bob', age=22)\nbob.save()\n\n# Query records\nstudent = Student.get(1)  # Query by primary key\nstudents = Student.filter(Student.age \u003e= 18).all()  # Conditional query\nstudents = Student.filter_by(name='Alice').all()  # Equality query\nall_students = Student.all()  # Get all\n\n# Update records\nalice.age = 21  # Active Record mode already supports attribute assignment updates\nalice.save()    # Explicitly save to database\n\n# Delete records\nalice.delete()\n\n# Close\ndb.close()\n```\n\n**How to Choose?**\n- **Pure Model Mode**: Suited for larger projects, team development, clear data access layer separation\n- **Active Record Mode**: Suited for smaller projects, rapid prototyping, simple CRUD operations\n\n## Storage Engines\n\nPytuck supports multiple storage engines, each suited for different scenarios:\n\n### Binary Engine (Default)\n\n**Features**: Zero dependencies, compact, high performance, encryption support\n\n```python\nfrom pytuck.common.options import BinaryBackendOptions\n\n# Basic usage\ndb = Storage(file_path='data.db', engine='binary')\n\n# Enable encryption (three levels: low/medium/high)\nopts = BinaryBackendOptions(encryption='high', password='mypassword')\ndb = Storage(file_path='secure.db', engine='binary', backend_options=opts)\n\n# Open encrypted database (auto-detects encryption level)\nopts = BinaryBackendOptions(password='mypassword')\ndb = Storage(file_path='secure.db', engine='binary', backend_options=opts)\n```\n\n**Encryption Levels**:\n| Level | Algorithm | Security | Use Case |\n|-------|-----------|----------|----------|\n| `low` | XOR obfuscation | Prevents casual viewing | Prevent accidental file opening |\n| `medium` | LCG stream cipher | Prevents regular users | General protection needs |\n| `high` | ChaCha20 | Cryptographically secure | Sensitive data protection |\n\n**Encryption Performance Benchmark** (1000 records, ~100 bytes each):\n| Level | Write Time | Read Time | File Size | Read Overhead |\n|-------|------------|-----------|-----------|---------------|\n| None | 41ms | 17ms | 183KB | (baseline) |\n| low | 33ms | 33ms | 183KB | +100% |\n| medium | 82ms | 86ms | 183KB | +418% |\n| high | 342ms | 335ms | 183KB | +1928% |\n\n\u003e **Note**: Encryption uses pure Python implementation to maintain zero dependencies. For better performance, consider using `low` or `medium` levels.\n\u003e Run `examples/benchmark_encryption.py` to test performance in your environment.\n\n**Use Cases**:\n- Production deployment\n- Embedded applications\n- Sensitive data protection\n- Minimum footprint required\n\n### JSON Engine\n\n**Features**: Human-readable, debug-friendly, standard format\n\n```python\nfrom pytuck.common.options import JsonBackendOptions\n\n# Configure JSON options\njson_opts = JsonBackendOptions(indent=2, ensure_ascii=False)\ndb = Storage(file_path='data.json', engine='json', backend_options=json_opts)\n```\n\n**Use Cases**:\n- Development and debugging\n- Configuration storage\n- Data exchange\n\n### CSV Engine\n\n**Features**: Excel compatible, tabular format, data analysis friendly\n\n```python\nfrom pytuck.common.options import CsvBackendOptions\n\n# Configure CSV options\ncsv_opts = CsvBackendOptions(encoding='utf-8', delimiter=',')\ndb = Storage(file_path='data_dir', engine='csv', backend_options=csv_opts)\n```\n\n**Use Cases**:\n- Data analysis\n- Excel import/export\n- Tabular data\n\n### SQLite Engine\n\n**Features**: Mature, stable, ACID compliance, SQL support\n\n```python\nfrom pytuck.common.options import SqliteBackendOptions\n\n# Configure SQLite options (optional)\nsqlite_opts = SqliteBackendOptions()  # Use default config\ndb = Storage(file_path='data.sqlite', engine='sqlite', backend_options=sqlite_opts)\n```\n\n**Use Cases**:\n- Need SQL queries\n- Need transaction guarantees\n- Large datasets\n\n### Excel Engine (Optional)\n\n**Requires**: `openpyxl\u003e=3.0.0`\n\n```python\nfrom pytuck.common.options import ExcelBackendOptions\n\n# Configure Excel options (optional)\nexcel_opts = ExcelBackendOptions(read_only=False)  # Use default config\ndb = Storage(file_path='data.xlsx', engine='excel', backend_options=excel_opts)\n```\n\n**Use Cases**:\n- Business reports\n- Visualization needs\n- Office automation\n\n### XML Engine (Optional)\n\n**Requires**: `lxml\u003e=4.9.0`\n\n```python\nfrom pytuck.common.options import XmlBackendOptions\n\n# Configure XML options\nxml_opts = XmlBackendOptions(encoding='utf-8', pretty_print=True)\ndb = Storage(file_path='data.xml', engine='xml', backend_options=xml_opts)\n```\n\n**Use Cases**:\n- Enterprise integration\n- Standardized exchange\n- Configuration files\n\n## Advanced Features\n\n### Generic Type Hints\n\nPytuck provides complete generic type support, enabling IDEs to precisely infer the specific types of query results and significantly enhancing the development experience:\n\n#### IDE Type Inference Effects\n\n```python\nfrom typing import List, Optional\nfrom pytuck import Storage, declarative_base, Session, Column\nfrom pytuck import select, insert, update, delete\n\ndb = Storage('mydb.db')\nBase = declarative_base(db)\n\nclass User(Base):\n    __tablename__ = 'users'\n    id = Column('id', int, primary_key=True)\n    name = Column('name', str)\n    age = Column('age', int)\n\nsession = Session(db)\n\n# Statement builder type inference\nstmt = select(User)  # IDE infers: Select[User] ✅\nchained = stmt.where(User.age \u003e= 18)  # IDE infers: Select[User] ✅\n\n# Session execution type inference\nresult = session.execute(stmt)  # IDE infers: Result[User] ✅\n\n# Result processing precise types\nusers = result.scalars().all()  # IDE infers: List[User] ✅ (no longer List[PureBaseModel])\nuser = result.scalars().first()  # IDE infers: Optional[User] ✅\n\n# IDE knows specific attribute types\nfor user in users:\n    user_name: str = user.name  # ✅ IDE knows this is str\n    user_age: int = user.age    # ✅ IDE knows this is int\n    # user.invalid_field        # ❌ IDE warns attribute doesn't exist\n```\n\n#### Type Safety Features\n\n- **Precise Type Inference**: `select(User)` returns `Select[User]`, not generic `Select`\n- **Smart Code Completion**: IDE accurately suggests model attributes and methods\n- **Compile-time Error Detection**: MyPy can detect type errors at compile time\n- **Method Chain Type Preservation**: All chained calls maintain specific generic types\n- **100% Backward Compatibility**: Existing code works unchanged and automatically gains type hint enhancement\n\n#### Comparison Effects\n\n**Before:**\n```python\nusers = result.scalars().all()  # IDE: List[PureBaseModel] 😞\nuser.name                       # IDE: doesn't know what attributes exist 😞\n```\n\n**Now:**\n```python\nusers = result.scalars().all()  # IDE: List[User] ✅\nuser.name                       # IDE: knows this is str type ✅\nuser.age                        # IDE: knows this is int type ✅\n```\n\n### Data Persistence\n\nPytuck provides flexible data persistence mechanisms.\n\n#### Pure Model Mode (Session)\n\n```python\ndb = Storage(file_path='data.db')  # auto_flush=False (default)\n\n# Data changes only in memory\nsession.execute(insert(User).values(name='Alice'))\nsession.commit()  # Commits to Storage memory\n\n# Manually write to disk\ndb.flush()  # Method 1: Explicit flush\n# or\ndb.close()  # Method 2: Auto-flush on close\n```\n\nEnable auto persistence:\n\n```python\ndb = Storage(file_path='data.db', auto_flush=True)\n\n# Each commit automatically writes to disk\nsession.execute(insert(User).values(name='Alice'))\nsession.commit()  # Automatically writes to disk, no manual flush needed\n```\n\n#### Active Record Mode (CRUDBaseModel)\n\nCRUDBaseModel has no Session, operates directly on Storage:\n\n```python\ndb = Storage(file_path='data.db')  # auto_flush=False (default)\nBase = declarative_base(db, crud=True)\n\nclass User(Base):\n    __tablename__ = 'users'\n    id = Column('id', int, primary_key=True)\n    name = Column('name', str)\n\n# create/save/delete only modify memory\nuser = User.create(name='Alice')\nuser.name = 'Bob'\nuser.save()\n\n# Manually write to disk\ndb.flush()  # Method 1: Explicit flush\n# or\ndb.close()  # Method 2: Auto-flush on close\n```\n\nEnable auto persistence:\n\n```python\ndb = Storage(file_path='data.db', auto_flush=True)\nBase = declarative_base(db, crud=True)\n\n# Each create/save/delete automatically writes to disk\nuser = User.create(name='Alice')  # Automatically writes to disk\nuser.name = 'Bob'\nuser.save()  # Automatically writes to disk\n```\n\n#### Persistence Method Summary\n\n| Method | Mode | Description |\n|--------|------|-------------|\n| `session.commit()` | Pure Model | Commits transaction to Storage memory; if `auto_flush=True`, also writes to disk |\n| `Model.create/save/delete()` | Active Record | Modifies Storage memory; if `auto_flush=True`, also writes to disk |\n| `storage.flush()` | Both | Forces in-memory data to be written to disk |\n| `storage.close()` | Both | Closes database, automatically calls `flush()` |\n\n**Recommendations**:\n- Use `auto_flush=True` in production for data safety\n- Use default mode for batch operations, call `flush()` at the end for better performance\n\n### Transaction Support\n\nPytuck supports memory-level transactions with automatic rollback on exceptions:\n\n```python\n# Session transaction (recommended)\nwith session.begin():\n    session.add(User(name='Alice'))\n    session.add(User(name='Bob'))\n    # Auto-commit on success, auto-rollback on exception\n\n# Storage-level transaction\nwith db.transaction():\n    db.insert('users', {'name': 'Alice'})\n    db.insert('users', {'name': 'Bob'})\n    # Auto-rollback to pre-transaction state on exception\n```\n\n### Session Context Manager\n\nSession supports context manager for automatic commit/rollback:\n\n```python\nwith Session(db) as session:\n    stmt = insert(User).values(name='Alice')\n    session.execute(stmt)\n    # Auto-commit on exit, auto-rollback on exception\n```\n\n### Auto-commit Mode\n\n```python\nsession = Session(db, autocommit=True)\n# Each operation auto-commits\nsession.add(User(name='Alice'))  # Auto-committed\n```\n\n### Object State Tracking\n\nSession provides complete object state tracking:\n\n```python\n# Add single object\nsession.add(user)\n\n# Batch add\nsession.add_all([user1, user2, user3])\n\n# Flush to database (without committing transaction)\nsession.flush()\n\n# Commit transaction\nsession.commit()\n\n# Rollback transaction\nsession.rollback()\n```\n\n### Auto Flush\n\nEnable `auto_flush` for automatic disk persistence on each write:\n\n```python\ndb = Storage(file_path='data.db', auto_flush=True)\n\n# Insert automatically writes to disk\nstmt = insert(Student).values(name='Bob', age=21)\nsession.execute(stmt)\nsession.commit()\n```\n\n### Index Queries\n\nAdd indexes to fields to accelerate queries:\n\n```python\nclass Student(Base):\n    __tablename__ = 'students'\n    name = Column('name', str, index=True)  # Create index\n\n# Index query (automatically optimized)\nstmt = select(Student).filter_by(name='Bob')\nresult = session.execute(stmt)\nbob = result.scalars().first()\n```\n\n### Query Operators\n\nSupported Pythonic query operators:\n\n```python\n# Equal\nstmt = select(Student).where(Student.age == 20)\n\n# Not equal\nstmt = select(Student).where(Student.age != 20)\n\n# Greater than / Greater than or equal\nstmt = select(Student).where(Student.age \u003e 18)\nstmt = select(Student).where(Student.age \u003e= 18)\n\n# Less than / Less than or equal\nstmt = select(Student).where(Student.age \u003c 30)\nstmt = select(Student).where(Student.age \u003c= 30)\n\n# IN query\nstmt = select(Student).where(Student.age.in_([18, 19, 20]))\n\n# Multiple conditions (AND)\nstmt = select(Student).where(Student.age \u003e= 18, Student.age \u003c 30)\n\n# Simple equality query (filter_by)\nstmt = select(Student).filter_by(name='Alice', age=20)\n```\n\n### Sorting and Pagination\n\n```python\n# Sorting\nstmt = select(Student).order_by('age')\nstmt = select(Student).order_by('age', desc=True)\n\n# Pagination\nstmt = select(Student).limit(10)\nstmt = select(Student).offset(10).limit(10)\n\n# Count\nstmt = select(Student).where(Student.age \u003e= 18)\nresult = session.execute(stmt)\nadults = result.scalars().all()\ncount = len(adults)\n```\n\n## Data Model Features\n\nPytuck's data models have unique characteristics that make them behave like both ORM and pure data containers.\n\n### Independent Data Objects\n\nPytuck model instances are completely independent Python objects that are immediately materialized to memory after query:\n\n- ✅ **Accessible After Session Close**: No DetachedInstanceError\n- ✅ **Operable After Storage Close**: Loaded objects are completely independent\n- ✅ **No Lazy Loading**: All direct attributes are loaded immediately\n- ✅ **Serializable**: Supports JSON, Pickle, and other serialization formats\n- ✅ **Usable as Data Containers**: Use like Pydantic models\n\n```python\nfrom pytuck import Storage, declarative_base, Session, Column, select\n\ndb = Storage(file_path='data.db')\nBase = declarative_base(db)\n\nclass User(Base):\n    __tablename__ = 'users'\n    id = Column('id', int, primary_key=True)\n    name = Column('name', str)\n\nsession = Session(db)\nstmt = select(User).where(User.id == 1)\nuser = session.execute(stmt).scalars().first()\n\n# Close session and storage\nsession.close()\ndb.close()\n\n# Still accessible!\nprint(user.name)  # ✅ Works\nprint(user.to_dict())  # ✅ Works\n```\n\n**Comparison with SQLAlchemy**:\n\n| Feature | Pytuck | SQLAlchemy |\n|---------|--------|------------|\n| Access after Session close | ✅ Supported | ❌ DetachedInstanceError |\n| Lazy loading relationships | ✅ Supported (with cache) | ✅ Supported |\n| Model as pure data container | ✅ Yes | ❌ No (bound to session) |\n\n### Relationships\n\nPytuck supports one-to-many and many-to-one relationships with lazy loading and caching:\n\n```python\nfrom pytuck.core.orm import Relationship\n\n# Define relationships\nclass User(Base):\n    __tablename__ = 'users'\n    id = Column('id', int, primary_key=True)\n    name = Column('name', str)\n    # One-to-many: one user has many orders\n    orders = Relationship('Order', foreign_key='user_id')\n\nclass Order(Base):\n    __tablename__ = 'orders'\n    id = Column('id', int, primary_key=True)\n    user_id = Column('user_id', int)\n    amount = Column('amount', float)\n    # Many-to-one: one order belongs to one user\n    user = Relationship(User, foreign_key='user_id')\n\n# Use relationships\nuser = User.get(1)\norders = user.orders  # Lazy loaded on first access\nfor order in orders:\n    print(f\"Order: {order.amount}\")\n\n# Reverse access\norder = Order.get(1)\nuser = order.user  # Many-to-one query\nprint(f\"User: {user.name}\")\n```\n\n**Relationship Features**:\n\n- ✅ **Lazy Loading**: Queries database only on first access\n- ✅ **Auto Caching**: Caches results to avoid repeated queries\n- ✅ **Bidirectional**: Supports back_populates parameter\n- ✅ **After Storage Close**: Already loaded relationships remain accessible (uses cache)\n- ⚠️ **Requires Eager Loading**: Access once before storage close to trigger loading\n\n```python\n# Eager loading strategy\nuser = User.get(1)\norders = user.orders  # Access before storage close to load and cache\n\ndb.close()\n\n# Still accessible after close (uses cache)\nfor order in orders:\n    print(order.amount)  # ✅ Works\n```\n\n### Type Validation and Conversion\n\nPytuck provides zero-dependency automatic type validation and conversion:\n\n```python\nclass User(Base):\n    __tablename__ = 'users'\n    id = Column('id', int, primary_key=True)\n    age = Column('age', int)  # Declared as int\n\n# Loose mode (default): auto conversion\nuser = User(age='25')  # ✅ Automatically converts '25' → 25\n\n# Strict mode: no conversion, raises error on type mismatch\nclass StrictUser(Base):\n    __tablename__ = 'strict_users'\n    id = Column('id', int, primary_key=True)\n    age = Column('age', int, strict=True)  # Strict mode\n\nuser = StrictUser(age='25')  # ❌ ValidationError\n```\n\n**Type Conversion Rules (Loose Mode)**:\n\n| Python Type | Conversion Rule | Example |\n|------------|----------------|---------|\n| int | int(value) | '123' → 123 |\n| float | float(value) | '3.14' → 3.14 |\n| str | str(value) | 123 → '123' |\n| bool | Special rules* | '1', 'true', 1 → True |\n| bytes | encode() if str | 'hello' → b'hello' |\n| datetime | ISO 8601 parse | '2024-01-15T10:30:00' → datetime |\n| date | ISO 8601 parse | '2024-01-15' → date |\n| timedelta | Total seconds | 3600.0 → timedelta(hours=1) |\n| list | JSON parse | '[1,2,3]' → [1, 2, 3] |\n| dict | JSON parse | '{\"a\":1}' → {'a': 1} |\n| None | Allowed if nullable=True | None → None |\n\n*bool conversion rules:\n- True: `True`, `1`, `'1'`, `'true'`, `'True'`, `'yes'`, `'Yes'`\n- False: `False`, `0`, `'0'`, `'false'`, `'False'`, `'no'`, `'No'`, `''`\n\n**Use Cases**:\n\n```python\n# Web API development: return directly after query, no connection concerns\n@app.get(\"/users/{id}\")\ndef get_user(id: int):\n    session = Session(db)\n    stmt = select(User).where(User.id == id)\n    user = session.execute(stmt).scalars().first()\n    session.close()\n\n    # Return model, no concern about closed session\n    return user.to_dict()\n\n# Data transfer: model objects can be passed freely between functions\ndef process_users(users: List[User]) -\u003e List[dict]:\n    return [u.to_dict() for u in users]\n\n# JSON serialization\nimport json\nuser_json = json.dumps(user.to_dict())\n```\n\n## Performance Benchmark\n\nHere are v4 version benchmark results.\n\n### Test Environment\n\n- **System**: Windows 11, Python 3.12.10\n- **Test Data**: 100,000 records\n- **Mode**: Extended test (including index comparison, range queries, batch reads, lazy loading)\n\n### Performance Comparison\n\n| Engine | Insert | Indexed | Non-Indexed | Speedup | Range | Save | Load | Lazy | Size |\n|--------|--------|---------|-------------|---------|-------|------|------|------|------|\n| Binary | 794.57ms | 1.39ms | 7.13s | 5124x | 333.29ms | 869.68ms | 1.01s | 319.88ms | 11.73MB |\n| JSON | 844.76ms | 1.42ms | 8.95s | 6279x | 337.01ms | 845.77ms | 319.37ms | - | 18.90MB |\n| CSV | 838.89ms | 1.47ms | 7.24s | 4939x | 346.85ms | 453.50ms | 472.90ms | - | 731.9KB |\n| SQLite | 879.05ms | 1.40ms | 7.21s | 5145x | 333.84ms | 325.80ms | 393.39ms | - | 6.97MB |\n| Excel | 897.48ms | 1.41ms | 7.25s | 5150x | 340.40ms | 5.75s | 7.63s | - | 2.84MB |\n| XML | 1.23s | 1.41ms | 7.41s | 5248x | 333.87ms | 2.49s | 2.03s | - | 34.54MB |\n\n**Notes**:\n- **Indexed**: 100 indexed field equality lookups (millisecond level)\n- **Non-Indexed**: 100 non-indexed field full table scans (second level)\n- **Speedup**: Index query vs non-indexed query speedup ratio\n- **Range**: Range condition queries (e.g., `age \u003e= 20 AND age \u003c 62`)\n- **Lazy**: Only Binary engine supports lazy loading (loads index only, not data)\n\n### Engine Feature Comparison\n\n| Engine | Query Perf | I/O Perf | Storage Eff | Human Readable | Dependencies | Recommended Use |\n|--------|-----------|----------|-------------|----------------|--------------|-----------------|\n| Binary | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐ | ❌ | None | **Production First Choice** |\n| JSON | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ✅ | None | Development, Config Storage |\n| CSV | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ✅ | None | Data Exchange, Minimum Size |\n| SQLite | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ❌ | None | SQL Needed, ACID Guarantee |\n| Excel | ⭐⭐⭐⭐ | ⭐ | ⭐⭐⭐⭐ | ✅ | openpyxl | Visual Editing, Reports |\n| XML | ⭐⭐⭐⭐ | ⭐⭐ | ⭐ | ✅ | lxml | Enterprise Integration |\n\n**Conclusions**:\n- **Binary** fastest insert (794ms), supports lazy loading and encryption, **production first choice**\n- **JSON** fastest load (319ms), easy debugging, suitable for development and config storage\n- **CSV** smallest file (732KB, ZIP compressed), excellent I/O, suitable for data exchange\n- **SQLite** best I/O (save 325ms), well-balanced, suitable for ACID requirements\n- **Excel** slower I/O (7.63s load), suitable for visual editing scenarios\n- **XML** largest file (34.54MB), suitable for enterprise integration\n\n## Installation Methods\n\n### Install from PyPI\n\n```bash\n# Basic installation\npip install pytuck\n\n# With specific extras\npip install pytuck[all]      # All optional engines\npip install pytuck[excel]    # Excel support only\npip install pytuck[xml]      # XML support only\npip install pytuck[dev]      # Development tools\n```\n\n### Install from Source\n\n```bash\n# Clone repository\ngit clone https://github.com/go9sky/pytuck.git\ncd pytuck\n\n# Editable install\npip install -e .\n\n# With all extras\npip install -e .[all]\n\n# Development mode\npip install -e .[dev]\n```\n\n### Build and Publish\n\n```bash\n# Install build tools\npip install build twine\n\n# Build wheel and source distribution\npython -m build\n\n# Upload to PyPI\npython -m twine upload dist/*\n\n# Upload to Test PyPI\npython -m twine upload --repository testpypi dist/*\n```\n\n## Data Migration\n\nMigrate data between different engines:\n\n```python\nfrom pytuck.tools.migrate import migrate_engine\nfrom pytuck.common.options import JsonBackendOptions\n\n# Configure target engine options\njson_opts = JsonBackendOptions(indent=2, ensure_ascii=False)\n\n# Migrate from binary to JSON\nmigrate_engine(\n    source_path='data.db',\n    source_engine='binary',\n    target_path='data.json',\n    target_engine='json',\n    target_options=json_opts  # Use strongly-typed options\n)\n```\n\n## Architecture\n\n```\n┌─────────────────────────────────────┐\n│       Application Layer             │\n│   BaseModel, Column, Query API      │\n└─────────────────────────────────────┘\n               ↓\n┌─────────────────────────────────────┐\n│          ORM Layer (orm.py)         │\n│   Model definitions, validation,    │\n│   relationship mapping              │\n└─────────────────────────────────────┘\n               ↓\n┌─────────────────────────────────────┐\n│     Storage Layer (storage.py)      │\n│   Table management, CRUD ops,       │\n│   query execution                   │\n└─────────────────────────────────────┘\n               ↓\n┌─────────────────────────────────────┐\n│    Backend Layer (backends/)        │\n│  BinaryBackend | JSONBackend | ...  │\n└─────────────────────────────────────┘\n               ↓\n┌─────────────────────────────────────┐\n│      Common Layer (common/)         │\n│   Exceptions, Utils, Options        │\n└─────────────────────────────────────┘\n```\n\n## Roadmap\n\n### Completed\n- Core ORM and in-memory storage\n- Pluggable multi-engine persistence\n- SQLAlchemy 2.0 style API\n- Basic transaction support\n\n## Current Limitations\n\nPytuck is a lightweight embedded database designed for simplicity. Here are the current limitations:\n\n| Limitation | Description |\n|------------|-------------|\n| **No JOIN support** | Single table queries only, no multi-table joins |\n| **No OR conditions** | Query conditions only support AND logic |\n| **No aggregate functions** | No COUNT, SUM, AVG, MIN, MAX support |\n| **No relationship loading** | No lazy loading or eager loading of related objects |\n| **No migration tools** | Schema changes require manual handling |\n| **Single writer** | No concurrent write support, suitable for single-process use |\n| **Full rewrite on save** | Non-binary/SQLite backends rewrite entire file on each save |\n| **No nested transactions** | Only single-level transactions supported |\n\n## Roadmap / TODO\n\n### Completed\n\n- [x] **Extended Field Type Support** ✨NEW✨\n  - [x] Added `datetime`, `date`, `timedelta`, `list`, `dict` five new types\n  - [x] Unified TypeRegistry codec, all backends use consistent serialization interface\n  - [x] JSON backend format optimization, removed redundant `_type`/`_value` wrapper\n- [x] **Binary Engine v4 Format** ✨NEW✨\n  - [x] WAL (Write-Ahead Log) for O(1) write latency\n  - [x] Dual Header mechanism for atomic switching and crash recovery\n  - [x] Index region zlib compression (saves ~81% space)\n  - [x] Batch I/O and codec caching optimizations\n  - [x] Three-tier encryption support (low/medium/high), pure Python implementation\n- [x] **Primary Key Query Optimization** (affects ALL storage engines) ✨NEW✨\n  - [x] `WHERE pk = value` queries use O(1) direct access\n  - [x] Single update/delete performance improved ~1000x\n- [x] Complete SQLAlchemy 2.0 Style Object State Management\n  - [x] Identity Map (Object Uniqueness Management)\n  - [x] Automatic Dirty Tracking (Attribute assignment auto-detected and updates database)\n  - [x] merge() Operation (Merge detached objects)\n  - [x] Query Instance Auto-Registration to Session\n- [x] Unified database connector architecture (`pytuck/connectors/` module)\n- [x] Data migration tools (`migrate_engine()`, `import_from_database()`)\n- [x] Import from external relational databases feature\n- [x] Unified engine version management (`pytuck/backends/versions.py`)\n- [x] Table and column comment support (`comment` parameter)\n- [x] Complete generic type hints system\n- [x] Strongly-typed configuration options system (dataclass replaces **kwargs)\n\n### Planned Features\n\n\u003e 📋 For detailed development plans, please refer to [TODO.md](./TODO.md)\n\n- [ ] **Web UI Interface Support** - Provide API support for independent Web UI library\n- [ ] **ORM Event Hooks System** - Complete event system based on SQLAlchemy event pattern\n- [ ] **JOIN Support** - Multi-table relational queries\n- [ ] **OR Condition Support** - Complex logical query conditions\n- [ ] **Aggregate Functions** - COUNT, SUM, AVG, MIN, MAX, etc.\n- [ ] **Relationship Lazy Loading** - Optimize associated data loading performance\n- [ ] **Schema Migration Tools** - Database structure version management\n- [ ] **Concurrent Access Support** - Multi-process/thread-safe access\n\n### Planned Engines\n\n- [ ] DuckDB - Analytical database engine\n- [ ] TinyDB - Pure Python document database\n- [ ] PyDbLite3 - Pure Python in-memory database\n- [ ] diskcache - Disk-based cache engine\n\n### Planned Optimizations\n\n- [ ] Incremental save for non-binary backends (currently full rewrite on each save)\n- [ ] Binary engine Compaction (space reclaim) mechanism\n- [ ] Use `tempfile` module for safer temporary file handling\n- [ ] Streaming read/write for large datasets\n- [ ] Connection pooling for SQLite backend\n- [ ] Relationship and lazy loading enhancements\n\n## Examples\n\nSee the `examples/` directory for more examples:\n\n- `sqlalchemy20_api_demo.py` - Complete SQLAlchemy 2.0 style API example (recommended)\n- `all_engines_test.py` - All storage engine functionality tests\n- `transaction_demo.py` - Transaction management example\n- `type_validation_demo.py` - Type validation and conversion example\n- `data_model_demo.py` - Data model independence features example\n- `backend_options_demo.py` - Backend configuration options demo (new)\n- `migration_tools_demo.py` - Data migration tools demo (new)\n\n## Contributing\n\nIssues and Pull Requests are welcome!\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n\n## Acknowledgments\n\nInspired by SQLAlchemy, Django ORM, and TinyDB.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgo9sky%2Fpytuck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgo9sky%2Fpytuck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgo9sky%2Fpytuck/lists"}