{"id":43471252,"url":"https://github.com/1amageek/database-framework","last_synced_at":"2026-04-21T12:00:58.648Z","repository":{"id":327367349,"uuid":"1107470904","full_name":"1amageek/database-framework","owner":"1amageek","description":"Build your own database with pluggable indexes — vector, full-text, spatial, graph, and more on FoundationDB","archived":false,"fork":false,"pushed_at":"2026-04-18T07:50:45.000Z","size":7178,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-18T09:36:10.969Z","etag":null,"topics":["database","foundationdb","full-text-search","knowledge-graph","owl","server-side-swift","sparql","spatial-index","swift","vector-search"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/1amageek.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"Docs/SECURITY_RULE_DESIGN.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-01T07:22:27.000Z","updated_at":"2026-04-18T07:50:49.000Z","dependencies_parsed_at":"2026-01-19T07:00:59.626Z","dependency_job_id":"4fcad6aa-e6a6-4689-9e9e-e0aec5fca1e1","html_url":"https://github.com/1amageek/database-framework","commit_stats":null,"previous_names":["1amageek/database-framework"],"tags_count":33,"template":false,"template_full_name":null,"purl":"pkg:github/1amageek/database-framework","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1amageek%2Fdatabase-framework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1amageek%2Fdatabase-framework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1amageek%2Fdatabase-framework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1amageek%2Fdatabase-framework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/1amageek","download_url":"https://codeload.github.com/1amageek/database-framework/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1amageek%2Fdatabase-framework/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32091053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-21T11:25:29.218Z","status":"ssl_error","status_checked_at":"2026-04-21T11:25:28.499Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","foundationdb","full-text-search","knowledge-graph","owl","server-side-swift","sparql","spatial-index","swift","vector-search"],"created_at":"2026-02-03T07:07:29.613Z","updated_at":"2026-04-21T12:00:58.564Z","avatar_url":"https://github.com/1amageek.png","language":"Swift","funding_links":[],"categories":[],"sub_categories":[],"readme":"# database-framework\n\nServer-side database engine built on FoundationDB, implementing index maintenance for [database-kit](https://github.com/1amageek/database-kit) models.\n\n## Overview\n\ndatabase-framework is the **server execution layer** that provides:\n\n- **FDBContainer / FDBContext**: Change-tracking data access with ACID transactions\n- **12 index maintainers**: Scalar, Vector, FullText, Spatial, Graph, Aggregation, and more\n- **Schema Registry**: pg_catalog-style metadata persistence\n- **Migration System**: Online index building and schema evolution\n- **DatabaseServer**: WebSocket server for [database-client](https://github.com/1amageek/database-client) connections\n- **DatabaseCLI**: Interactive REPL for data inspection\n\n```\n┌──────────────────────────────────────────────────────────┐\n│                      database-kit                        │\n│  @Persistable models, IndexKind protocols, QueryIR       │\n└──────────┬───────────────────────────────┬───────────────┘\n           │                               │\n           ▼                               ▼\n┌─────────────────────┐       ┌─────────────────────────┐\n│  database-framework │       │    database-client       │\n│  FDBContainer       │◄─────│    DatabaseContext        │\n│  Index Maintainers  │  WS  │    KeyPath queries       │\n│  FoundationDB       │       │    iOS / macOS           │\n└─────────────────────┘       └─────────────────────────┘\n```\n\n## Installation\n\n### Prerequisites\n\n- **FoundationDB 7.3+** installed and running\n- **Swift 6.2+**\n- **macOS 15+** or Linux\n\n```bash\n# macOS\nbrew install foundationdb\nbrew services start foundationdb\n```\n\n### Swift Package Manager\n\n```swift\ndependencies: [\n    .package(url: \"https://github.com/1amageek/database-framework.git\", from: \"26.0411.1\")\n]\n```\n\n## Quick Start\n\n```swift\nimport Database\nimport Core\n\n// 1. Define models (from database-kit)\n@Persistable\nstruct User {\n    #Directory\u003cUser\u003e(\"app\", \"users\")\n    #Index(ScalarIndexKind\u003cUser\u003e(fields: [\\.email]), unique: true)\n\n    var email: String\n    var name: String\n}\n\n// 2. Create container\nlet schema = Schema([User.self])\nlet container = try await FDBContainer(for: schema)\n\n// 3. Insert data\nlet context = container.newContext()\ncontext.insert(User(email: \"alice@example.com\", name: \"Alice\"))\ncontext.insert(User(email: \"bob@example.com\", name: \"Bob\"))\ntry await context.save()\n\n// 4. Query\nlet users = try await context.fetch(User.self)\n    .where(\\.name == \"Alice\")\n    .execute()\n```\n\n## Modules\n\n### Core\n\n| Module | Description |\n|--------|-------------|\n| `DatabaseEngine` | FDBContainer, FDBContext, IndexMaintainer protocol, Schema Registry |\n| `Database` | All-in-one re-export of all index modules |\n| `QueryAST` | SQL/SPARQL parser, AST builder, serializer |\n| `DatabaseServer` | WebSocket server for database-client connections |\n| `DatabaseCLICore` | Interactive REPL library for data inspection |\n\n### Index Maintainers\n\n| Module | IndexKind | Features |\n|--------|-----------|----------|\n| `ScalarIndex` | `ScalarIndexKind` | Range queries, sorting, unique constraints |\n| `VectorIndex` | `VectorIndexKind` | HNSW and flat similarity search |\n| `FullTextIndex` | `FullTextIndexKind` | Tokenization, inverted index |\n| `SpatialIndex` | `SpatialIndexKind` | S2 / Geohash / Morton encoding |\n| `RankIndex` | `RankIndexKind` | Skip-list based rankings |\n| `GraphIndex` | `GraphIndexKind` | Graph traversal, SPARQL, OWL DL reasoning |\n| `AggregationIndex` | `Count/Sum/Min/Max/AverageIndexKind` | Atomic aggregations |\n| `VersionIndex` | `VersionIndexKind` | FDB versionstamp history |\n| `BitmapIndex` | `BitmapIndexKind` | Roaring bitmap for categorical data |\n| `LeaderboardIndex` | `TimeWindowLeaderboardIndexKind` | Time-windowed leaderboards |\n| `PermutedIndex` | `PermutedIndexKind` | Alternative field orderings |\n| `RelationshipIndex` | `RelationshipIndexKind` | Cross-type relationship queries |\n\n## Architecture\n\n### Container / Context Pattern\n\n| Component | Role | Transactions |\n|-----------|------|-------------|\n| `FDBContainer` | Resource manager (DB connection, Schema, Directory) | Does NOT create |\n| `FDBContext` | User-facing API, change tracking, batch operations | Creates |\n| `FDBDataStore` | Low-level operations within transactions | Receives |\n\n### Lifecycle Management\n\n```\nFDBContainer (owns FDBDatabase)\n  └── newContext() → FDBContext (references container, does NOT own database)\n\nStartup:  FDBClient.initialize() → FDBContainer.init → container.newContext()\nTeardown: context = nil → container = nil → FDBClient.shutdown() [optional]\n```\n\n| Component | Owns Database? | Cleanup |\n|-----------|---------------|---------|\n| `FDBContainer` | Yes (strong ref) | ARC — releases database on dealloc |\n| `FDBContext` | No (references container) | ARC — discards pending changes |\n\n**Process exit**: Safe without explicit cleanup. OS reclaims all resources.\n\n**Explicit shutdown** (servers, tests):\n\n```swift\n// 1. Release contexts (discard pending changes)\ncontext = nil\n\n// 2. Release container (triggers fdb_database_destroy via ARC)\ncontainer = nil\n\n// 3. Stop network (optional — OS handles this at process exit)\nFDBClient.shutdown()\n```\n\n**Server pattern**: One container per process, one context per request.\n**CLI pattern**: One container, one context, `Foundation.exit(0)` is safe.\n**Test pattern**: One container per suite, `FDBClient.shutdown()` in teardown.\n\n### Change Tracking\n\n```swift\nlet context = container.newContext()\n\n// Stage changes\ncontext.insert(user1)\ncontext.insert(user2)\ncontext.delete(user3)\n\n// Commit all in one ACID transaction\ntry await context.save()\n```\n\n### Protocol Bridge\n\ndatabase-kit defines **what** to index. database-framework defines **how** to maintain it.\n\n```swift\n// database-kit: metadata only\npublic struct ScalarIndexKind: IndexKind {\n    public static let identifier = \"scalar\"\n}\n\n// database-framework: FDB-dependent implementation\nextension ScalarIndexKind: IndexKindMaintainable {\n    public func makeIndexMaintainer\u003cItem: Persistable\u003e(...) -\u003e any IndexMaintainer\u003cItem\u003e {\n        return ScalarIndexMaintainer(...)\n    }\n}\n```\n\n## Features\n\n### Security\n\n```swift\n// Declarative access control\nextension Post: SecurityPolicy {\n    static func allowGet(resource: Post, auth: (any AuthContext)?) -\u003e Bool {\n        resource.isPublic || resource.authorID == auth?.userID\n    }\n}\n\n// Encryption at rest (AES-256-GCM)\nlet config = TransformConfiguration(\n    encryptionEnabled: true,\n    keyProvider: RotatingKeyProvider(...)\n)\nlet container = try FDBContainer(for: schema, transformConfiguration: config)\n```\n\n### Hybrid SQL/SPARQL\n\n```swift\nlet sql = \"\"\"\nSELECT * FROM User\nWHERE id IN (SPARQL(Connection, 'SELECT ?from WHERE { ?from \"follows\" \"bob\" }'))\n  AND age \u003e 18\n\"\"\"\nlet users = try await context.executeSQL(sql, as: User.self)\n```\n\n### Migration\n\n```swift\nlet migration = Migration(\n    fromVersion: Schema.Version(1, 0, 0),\n    toVersion: Schema.Version(2, 0, 0),\n    description: \"Add email index\"\n) { ctx in\n    try await ctx.addIndex(emailIndexDescriptor)\n}\n```\n\n### Ontology Integration\n\nLink Persistable types to OWL ontology classes with `@OWLClass` and `@OWLDataProperty` / `@OWLObjectProperty` macros (defined in [database-kit](https://github.com/1amageek/database-kit) Graph module). `@Persistable` handles pure persistence; `@OWLClass` handles OWL class mapping independently.\n\n```swift\nimport Database\nimport Core\nimport Graph\n\n// 1. Define ontology-aware types\n@Persistable\n@OWLClass(\"http://example.org/onto#Employee\")\nstruct Employee {\n    #Directory\u003cEmployee\u003e(\"app\", \"employees\")\n\n    @OWLDataProperty(\"http://example.org/onto#name\")\n    var name: String\n\n    @OWLDataProperty(\"http://example.org/onto#worksFor\", to: \\Department.id)\n    var departmentID: String?\n}\n\n@Persistable\n@OWLClass(\"http://example.org/onto#Department\")\nstruct Department {\n    #Directory\u003cDepartment\u003e(\"app\", \"departments\")\n    var name: String\n}\n\n// 2. Define object property (edge type) with @OWLObjectProperty\n@Persistable\n@OWLObjectProperty(\"http://example.org/onto#worksFor\", from: \"employeeID\", to: \"departmentID\")\nstruct WorksFor {\n    #Directory\u003cWorksFor\u003e(\"app\", \"assignments\")\n    var employeeID: String = \"\"\n    var departmentID: String = \"\"\n\n    @OWLDataProperty(\"http://example.org/onto#since\")\n    var startDate: Date = Date()\n}\n\n// 3. Define RDF triple store\n@Persistable\nstruct RDFTriple {\n    #Directory\u003cRDFTriple\u003e(\"app\", \"knowledge\")\n    var subject: String = \"\"\n    var predicate: String = \"\"\n    var object: String = \"\"\n\n    #Index(GraphIndexKind\u003cRDFTriple\u003e(\n        from: \\.subject, edge: \\.predicate, to: \\.object,\n        strategy: .hexastore\n    ))\n}\n\n// 4. Create container\nlet schema = Schema([Employee.self, Department.self, WorksFor.self, RDFTriple.self])\nlet container = try await FDBContainer(for: schema)\n\n// 5. Load ontology independently via OntologyStore\nlet ontology = OWLOntology(iri: \"http://example.org/onto\") {\n    Class(\"ex:Employee\", subClassOf: \"ex:Person\")\n    Class(\"ex:Department\")\n    ObjectProperty(\"ex:worksFor\", domain: \"ex:Employee\", range: \"ex:Department\")\n}\n\nlet context = container.newContext()\ntry await context.ontology.load(ontology)\n```\n\n**What happens automatically**:\n- `@OWLClass` generates `OWLClassEntity` conformance with `ontologyClassIRI`\n- `@OWLClass` scans `@OWLDataProperty` annotations to build `ontologyPropertyDescriptors`\n- `@OWLObjectProperty` generates `OWLObjectPropertyEntity` conformance with `objectPropertyIRI`, `fromFieldName`, `toFieldName`, and auto-generates a `GraphIndexKind` for the edge type\n- `@OWLDataProperty(to:)` generates a reverse index on `departmentID` for efficient lookups\n- `context.ontology.load()` persists the ontology to `/_ontology/` via OntologyStore\n- GraphIndex uses the ontology for OWL 2 RL materialization on triple writes\n\nSee [GraphIndex README](Sources/GraphIndex/README.md) for SPARQL, OWL reasoning, and graph algorithms.\n\n### Polymorphable\n\n```swift\n@Polymorphable\nprotocol Document: Polymorphable {\n    var id: String { get }\n    var title: String { get }\n}\n\n// Query all conforming types from a shared directory\nlet docs = try await context.fetchPolymorphic(Article.self)\n```\n\n## Data Layout\n\n```\n[directory]/R/[type]/[id]              → Protobuf-encoded item\n[directory]/I/[indexName]/[values]/[id] → Index entry\n[directory]/state/[indexName]           → Index state (readable/writeOnly/disabled)\n/_catalog/[typeName]                    → JSON-encoded TypeCatalog\n```\n\n## Build and Test\n\n```bash\nswift build                                    # Debug build\nswift build -c release                         # Release build\nswift test                                     # All tests (requires FoundationDB)\nswift test --filter DatabaseEngineTests        # Engine tests only\n```\n\n## Benchmarks\n\nPerformance benchmarks live in the `PerformanceBenchmarks` test target and require a healthy FoundationDB cluster.\n\n```bash\nswift test --filter 'PerformanceBenchmarks.CoveringIndexBenchmark'\nswift test --filter 'PerformanceBenchmarks.FDBFrameworkCRUDBenchmarkTests'\nswift test --filter 'PerformanceBenchmarks.IndexedQueryAndWriteBenchmarkTests'\nswift test --filter 'PerformanceBenchmarks.MinMaxBatchBenchmark'\nswift test --filter 'PerformanceBenchmarks.RangeTreeBenchmark'\nswift test --filter 'PerformanceBenchmarks.SerializationBenchmark'\n```\n\nThe `DatabaseEngine` benchmarks use `.serialized` suites, `FDBTestSetup.shared.withSerializedAccess`, unique benchmark IDs, and explicit directory cleanup so each run stays isolated and does not deadlock on shared FoundationDB state.\n\n### Latest Snapshot (2026-04-11)\n\n**Environment**: macOS 26.3, Apple M4 Max, local Docker FoundationDB cluster\n\n| Module | Scenario | Result |\n|--------|----------|--------|\n| `ScalarIndex` | Fetch all users (300 records) | p95 `3.60ms`, `293 ops/s` |\n| `ScalarIndex` | Scan scalability (200 records) | p95 `6.45ms`, `184 ops/s` |\n| `AggregationIndex` | MIN/MAX grouped query | p95 `2.64ms`, `377 ops/s` |\n| `AggregationIndex` | Combined MIN+MAX vs separate queries | p95 `2.33ms`, throughput `+29.05%` |\n| `RankIndex` | Top-100 from 1,000 players | p95 `10.61ms`, `97 ops/s` |\n| `RankIndex` | Top-K scaling | p95 stayed near `10.72-11.02ms` |\n| `BitmapIndex` | JSON serialization | p95 `0.07ms`, `14,910 ops/s` |\n| `BitmapIndex` | JSON round-trip (10,000 values) | p95 `1148.92ms` |\n\nDetailed benchmark reports:\n- `ScalarIndex`: `Sources/ScalarIndex/README.md`\n- `AggregationIndex`: `Sources/AggregationIndex/README.md`\n- `RankIndex`: `Sources/RankIndex/README.md`\n- `BitmapIndex`: `Sources/BitmapIndex/README.md`\n\n## Platform Support\n\n| Platform | Support |\n|----------|---------|\n| macOS | 15.0+ |\n| Linux | Swift 6.2+ |\n\n\u003e iOS/watchOS/tvOS/visionOS are not supported (requires FoundationDB).\n\n## Related Packages\n\n| Package | Role | Platform |\n|---------|------|----------|\n| **[database-kit](https://github.com/1amageek/database-kit)** | Model definitions, IndexKind protocols, QueryIR | iOS, macOS, Linux |\n| **[database-client](https://github.com/1amageek/database-client)** | Client SDK with KeyPath queries and WebSocket transport | iOS, macOS |\n\n## License\n\nMIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1amageek%2Fdatabase-framework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F1amageek%2Fdatabase-framework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1amageek%2Fdatabase-framework/lists"}