{"id":13409103,"url":"https://github.com/kirilltitov/FDBSwift","last_synced_at":"2025-03-14T14:30:58.845Z","repository":{"id":34359789,"uuid":"140037231","full_name":"kirilltitov/FDBSwift","owner":"kirilltitov","description":"FoundationDB client for Swift","archived":false,"fork":false,"pushed_at":"2022-11-03T16:27:39.000Z","size":301,"stargazers_count":45,"open_issues_count":2,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-20T08:39:07.432Z","etag":null,"topics":["backend","fdb","foundationdb","nio","swift","swift-nio","swiftnio"],"latest_commit_sha":null,"homepage":"","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kirilltitov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-07-06T23:12:17.000Z","updated_at":"2024-10-15T00:01:26.000Z","dependencies_parsed_at":"2022-08-08T00:16:35.435Z","dependency_job_id":null,"html_url":"https://github.com/kirilltitov/FDBSwift","commit_stats":null,"previous_names":[],"tags_count":68,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kirilltitov%2FFDBSwift","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kirilltitov%2FFDBSwift/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kirilltitov%2FFDBSwift/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kirilltitov%2FFDBSwift/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kirilltitov","download_url":"https://codeload.github.com/kirilltitov/FDBSwift/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243593286,"owners_count":20316160,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","fdb","foundationdb","nio","swift","swift-nio","swiftnio"],"created_at":"2024-07-30T20:00:58.033Z","updated_at":"2025-03-14T14:30:58.488Z","avatar_url":"https://github.com/kirilltitov.png","language":"Swift","funding_links":[],"categories":["Bindings"],"sub_categories":[],"readme":"# FDBSwift v5 \u003cimg src=\"https://img.shields.io/badge/Swift-5.5-brightgreen.svg\" alt=\"Swift: 5.5\" /\u003e\n\u003e _Episode V: The Swift Awaitening_\n\nThis is FoundationDB client for Swift. It's quite low-level, (almost) `Foundation`less and can into `async`/`await`\n(Swift-NIO not included and not required).\n\n## Installation\n\nObviously, you need to install `FoundationDB` first. Download it from\n[official website](https://www.foundationdb.org/download/). Next part is tricky because CFDB module (C bindings)\nwon't link `libfdb_c` library on its own, and FoundationDB doesn't yet ship `pkg-config` during installation.\nTherefore you must install it yourself. Run\n\n```bash\nchmod +x ./scripts/install_pkgconfig.sh\n./scripts/install_pkgconfig.sh\n```\n\nor copy `scripts/libfdb.pc` (choose your platform) to `/usr/local/lib/pkgconfig/` on macOS or\n`/usr/lib/pkgconfig/libfdb.pc` on Linux.\n\n## Migration from v4 to v5\n\nv5 is a huge update in terms of internals and API. The most important update is `async`/`await` adoption, of course.\nBecause of that, Swift-NIO dependency has been dropped as redundant. Naturally, a lot of API just vanished as obsolete.\nHowever, the remaining API looks pretty much the same, except it's not using event loops and futures.\n\nIn v4 (and earlier versions) there was a blocking API. Naturally, barely anyone used it because, well, why would you.\nHowever, during v5 development it turned out that this API is a perfect foundation for adopting `async`/`await`,\njust add `async` in signature and you're ready to go. Like it's been waiting for it really. It's a good thing I've been\nmaintaining this API along with NIO API. Good for me. \n\nSo now there are only two ways of using FDBSwift:\n\n1. Oneshot API with autocommit in `AnyFDB` (hence, `FDB`, the default impl), like `get(key:)`, `set(key:value:)`,\n`atomic(_ op:key:value)` etc.\n1. Transactional API which comes in two flavours:\n    1. Manual transaction management with flat flow: `fdb.begin()`, some operations and `transaction.commit()`\n    (basically you have full control over transaction and have to catch errors manually and process retry error yourself\n    as well, see details and respective section below).\n    1. Wrapped transactions: `fdb.withTransaction { transaction in /* some operations */ }` which manages retries\n    for you, still you have to be aware of other errors.\n\nOne more not so minor change is that error case `FDB.Error.transactionRetry` doesn't have an `AnyFDBTransaction`\nassociated value anymore. It was handy in v4 when you could begin a NIO transaction and not have a reference to it\nat ANY moment, including `flatMapError`/`recover` etc. (I personally had a lot of such cases).\nNow this problem is gone for good and this associated value isn't needed at all.\n\n## Usage\n\n### Root concepts\n\nBy default (and in the very core) this wrapper, as well as C API, operates with byte keys and values (not pointers, but\n`Array\u003cUInt8\u003e`). See [Keys, tuples and subspaces](#keys-tuples-and-subspaces) section for more details.\n\nValues are always bytes (`typealias Bytes = [UInt8]`) (or `nil` if key not found). Why not `Data` you may ask? I'd like\nto stay `Foundation`less for as long as I can (srsly, import half of the world just for `Data` object which is a fancy\nwrapper around `NSData` which is a fancy wrapper around `[UInt8]`?) (Hast thou forgot that you need to wrap all your\n`Data` objects with `autoreleasepool` or otherwise you get _fancy_ memory leaks?) (except for Linux tho, yes), you can\nalways convert bytes to `Data` with `Data(bytes: myBytes)` initializer (why would you want to do that? oh yeah, right,\nJSON... ok, but do it yourself please, extensions to the rescue).\n\n### Connection\n\n```swift\n// Default cluster file path depending on your OS\nlet fdb = FDB()\n\n// OR\nlet fdb = FDB(clusterFile: \"/usr/local/etc/foundationdb/fdb.cluster\")\n```\n\nOptionally you may pass network stop timeout.\n\nKeep in mind that at this point connection has not yet been established, it's automatically established on first actual\ndatabase operation. If you would like to explicitly connect to database and catch possible errors, just call:\n\n```swift\ntry fdb.connect()\n```\n\nDisconnect is automatic, on `deinit`. But you may also call `disconnect()` method directly. Be warned that if\nanything goes wrong during disconnect, you will get uncatchable fatal error. It's not that bad because disconnect\nshould happen only once, when your application shuts down (and you shouldn't really care about fatal errors at that\npoint). Also you _very_ ought to ensure that FDB really disconnected before actual shutdown (trap `SIGTERM` signal and\nwait for `disconnect` to finish), otherwise you might experience undefined behaviour (I personally haven't really\nencountered that yet, but it's not a phantom menace; when you don't follow FoundationDB recommendations things get\nquite messy indeed).\n\nBefore you connected to FDB cluster you may also set network options:\n\n```swift\ntry fdb.setOption(.TLSCertPath(path: \"/opt/fdb/tls/chain.pem\"))\ntry fdb.setOption(.TLSPassword(password: \"changeme\"))\ntry fdb.setOption(.buggifyEnable)\n```\n\nSee [`FDB+NetworkOptions.swift`](Sources/FDB/FDB%2BNetworkOptions.swift) file for complete set of network options.\n\n### Keys, tuples and subspaces\n\nAll keys are `AnyFDBKey` which is a protocol:\n\n```swift\npublic protocol AnyFDBKey {\n    func asFDBKey() -\u003e Bytes\n}\n```\n\nThis protocol is adopted by `String`, `StaticString`, `Tuple` (NOT Tuple type from Swift), `Subspace` and `Bytes`\n(aka `Array\u003cUInt8\u003e`), so you may freely use any of these types, or adopt this protocol in your custom types.\n\nSince you would probably like to have some kind of key namespacing in your application, you should stick to `Subspace`\nwhich is an extremely useful instrument for creating namespaces. Under the hood it utilizes the Tuple concept. You\noughtn't really bother delving into it (in short: basically a discount MsgPack, a tricky binary protocol), just remember\nthat currently subspaces accept `String`, `Int`, `Float` (aka `Float32`), `Double`, `Bool`, `UUID`, `Tuple`\n(hence `FDBTuplePackable`), `FDB.Null` (why would you do that?) and `Bytes` as arguments.\n\n```swift\n// dump subspace if you would like to see how it looks from the inside\nlet rootSubspace = FDB.Subspace(\"root\")\n\n// also check Subspace.swift for more details and usecases\nlet childSubspace = rootSubspace[\"child\"][\"subspace\"]\n\n// OR\nlet childSubspace = rootSubspace[\"child\", \"subspace\"]\n\n// Talking about tuples:\nlet tuple = FDB.Tuple(\n    Bytes([0, 1, 2]),\n    322,\n    -322,\n    FDB.Null(),\n    \"foo\",\n    FDB.Tuple(\"bar\", 1337, \"baz\"),\n    FDB.Tuple(),\n    FDB.Null()\n)\nlet packed: Bytes = tuple.pack()\nlet unpacked: FDB.Tuple = try FDB.Tuple(from: packed)\nlet tupleBytes: Bytes? = unpacked.tuple[0] as? Bytes\nlet tupleInt: Int? = unpacked.tuple[1] as? Int\n// ...\nlet tupleEmptyTuple: FDB.Tuple? = unpacked.tuple[6] as? FDB.Tuple\nlet tupleNull: FDB.Null? = unpacked.tuple[7] as? FDB.Null\nif tupleNull is FDB.Null || unpacked.tuple[7] is FDB.Null {}\n\n// you get the idea\n```\n\n### Setting values\n\nSimple as that:\n\n```swift\ntry await fdb.set(key: \"somekey\", value: someBytes)\n\n// OR\ntry await fdb.set(key: Bytes([0, 1, 2, 3]), value: someBytes)\n\n// OR\ntry await fdb.set(key: FDB.Tuple(\"foo\", FDB.Null(), \"bar\", FDB.Tuple(\"baz\", \"sas\"), \"lul\"), value: someBytes)\n\n// OR\ntry await fdb.set(key: Subspace(\"foo\", \"bar\"), value: someBytes)\n```\n\n### Getting values\n\nValue is always `Bytes?` (`nil` if key not found), you should unwrap it before use.\nKeys are, of course, still `AnyFDBKey`s.\n\n```swift\nlet value = try await fdb.get(key: \"someKey\")\n```\n\n### Range get (multi get)\n\nSince in FoundationDB keys are lexicographically ordered over the underlying bytes, you can get all subspace values\n(or even from whole DB) by querying range from key `somekey\\x00` to key `somekey\\xFF` (from byte 0 to byte 255).\nYou shouldn't do it manually though, as `Subspace` object has a shortcut that does it for you.\n\nAdditionally, `get(range:)` (and its versions) method returns not `Bytes`, but a special box structure\n`FDB.KeyValuesResult` which holds an array of `FDB.KeyValue` structures and a flag indicating whether DB can provide\nmore results (pagination, kinda):\n\n```swift\npublic extension FDB {\n    /// A holder for key-value pair\n    public struct KeyValue {\n        public let key: Bytes\n        public let value: Bytes\n    }\n    \n    /// A holder for key-value pairs result returned from range get\n    public struct KeyValuesResult {\n        /// Records returned from range get\n        public let records: [FDB.KeyValue]\n\n        /// Indicates whether there are more results in FDB\n        public let hasMore: Bool\n    }\n}\n```\n\nIf range call returned zero records, it would result in an empty `FDB.KeyValuesResult` struct (not `nil`).\n\n```swift\nlet subspace = FDB.Subspace(\"root\")\nlet range = subspace.range\n/*\n  these three calls are completely equal (can't really come up with case when you need second form,\n  but whatever, I've seen worse whims)\n*/\nlet result: FDB.KeyValuesResult = try await fdb.get(range: range)\nlet result: FDB.KeyValuesResult = try await fdb.get(begin: range.begin, end: range.end)\nlet result: FDB.KeyValuesResult = try await fdb.get(subspace: subspace)\n\n// although call below is not equal to above one because `key(subspace:)` overload implicitly loads range\n// this one will load bare subspace key\nlet result: FDB.KeyValuesResult = try await fdb.get(key: subspace)\n\nresult.records.forEach {\n    dump(\"\\($0.key) - \\($0.value)\")\n}\n```\n\n### Clearing values\n\nClearing (removing, deleting, you name it) records is simple as well.\n\n```swift\ntry await fdb.clear(key: childSubspace[\"concrete_record\"])\n\n// OR\ntry await fdb.clear(key: rootSubspace[\"child\"][\"subspace\"][\"concrete_record\"])\n\n// OR EVEN\ntry await fdb.clear(key: rootSubspace[\"child\", \"subspace\", \"concrete_record\"])\n\n// OR EVEN (this is not OK, but still possible :)\ntry await fdb.clear(key: rootSubspace[\"child\", FDB.Null, FDB.Tuple(\"foo\", \"bar\"), \"concrete_record\"])\n\n// clears whole subspace, including \"concrete_record\" key\ntry await fdb.clear(range: childSubspace.range)\n```\n\n### Atomic operations\n\nFoundationDB also supports atomic operations like `ADD`, `AND`, `OR`, `XOR` and stuff like that\n(please refer to [docs](https://apple.github.io/foundationdb/api-c.html#c.FDBMutationType)).\nYou can perform any of these operations with `atomic(_ op:key:value:)` method:\n\n```swift\ntry await fdb.atomic(.add, key: key, value: 1)\n```\n\nKnowing that most popular atomic operation is increment (or decrement), I added handy syntax sugar:\n\n```swift\ntry await fdb.increment(key: key)\n\n// OR returning incremented value, which is always Int64\nlet result: Int64 = try await fdb.increment(key: key)\n\n// OR\nlet result = try await fdb.increment(key: key, value: 2)\n```\n\nHowever, keep in mind that example above isn't atomic anymore.\n\nAnd decrement, which is just a proxy for `increment(key:value:)`, just inverting the `value`:\n\n```swift\nlet result = try await fdb.decrement(key: key)\n\n// OR\nlet result = try await fdb.decrement(key: key, value: 2)\n```\n\n### Transactions\n\nAll previous examples are utilizing `FDB` object methods which are implicitly transactional. If you would like\nto perform more than one operation within one transaction (and experience all delights\nof [ACID](https://en.wikipedia.org/wiki/ACID_(computer_science))), you should first begin transaction using\n`begin()` method on `FDB` object context and then do your stuff (just don't forget to `commit()` it in the end,\nby default transactions roll back if not committed explicitly, or after timeout of 5 seconds):\n\n```swift\nlet transaction = try fdb.begin()\n\ntransaction.set(key: \"someKey\", value: someBytes)\n\ntry await transaction.commit()\n\n// OR\ntransaction.reset()\n\n// OR\ntransaction.cancel()\n```\n\nOr you can just leave transaction object in place and it resets \u0026 destroys itself on `deinit`.\nConsider it auto-rollback. Please refer to official docs on reset and cancel behaviour:\nhttps://apple.github.io/foundationdb/api-c.html#c.fdb_transaction_reset\n\nIt's not really necessary to commit readonly transaction though :)\n\nAdditionally you may set transaction options using `transaction.setOption(_:)` method:\n\n```swift\nlet transaction: AnyFDBTransaction = ...\ntry transaction.setOption(.transactionLoggingEnable(identifier: \"debuggable_transaction\"))\ntry transaction.setOption(.snapshotRywDisable)\n```\n\nSee [`Transaction+Options.swift`](Sources/FDB/Transaction/Transaction%2BOptions.swift) for a complete list of options.\n\n### Conflicts, retries and `withTransaction`\n\nSince FoundationDB is _quite_ a transactional database, sometimes `commit`s might not succeed due to serialization\nfailures (more commonly and mistakenly known as deadlocks). This can happen when two or more transactions create\noverlapping conflict ranges. Or, simply speaking, when they try to access or modify same keys\n(unless they are not in `snapshot` read mode) at the same time. This is expected (and, in a way, welcomed) behaviour,\nbecause this is how ACID works.\n\nIn these [not-so-rare] cases transaction is allowed to be replayed again.\nHow do you know if your transaction can be replayed? It's failed with a special error case`FDB.Error.transactionRetry`.\nIf your transaction is failed with this particular error, it means that the transaction has already been rolled back\nto its initial state and is ready to be executed again.\n\nYou can implement this retry logic manually or you can just use `FDB` instance method `withTransaction`.\nFollowing example should be self-explanatory:\n\n```swift\nlet maybeString: String? = try await fdb.withTransaction { transaction in\n    guard let bytes: Bytes = try await transaction.get(key: key) else {\n        return nil\n    }\n    try await transaction.commit()\n    return String(bytes: bytes, encoding: .ascii)\n}\n```\n\nThus your block of code will be gently retried until transaction is successfully committed\n(or until databases decides that it's been retried enough times and _it's time to let it go_).\n\n### Writing to Versionstamped Keys\n\nAs a special type of atomic operation, values can be written to special keys that are guaranteed to be unique.\nThese keys make use of an incomplete versionstamp within their tuples, which will be completed by the underlying cluster\nwhen data is written. The Versionstamp that was used within a transaction can then be retrieved so it can be\nreferenced elsewhere.\n\nAn incomplete versionstamp can be created and added to tuples using the `FDB.Versionstamp()` initializer.\nThe `userData` field is optional, and serves to further order keys if multiple are written within the same transaction.\n\nWithin a transaction's block, the `set(versionstampedKey:value:)` method can be used to write to keys with incomplete\nversionstamps. This method will search the key for an incomplete versionstamp, and if one is found, will flag it to be\nreplaced by a complete versionstamp once it's written to the cluster. If an incomplete versionstamp was not found,\na `FDB.Error.missingIncompleteVersionstamp` error will be thrown.\n\nIf you need the complete versionstamp that was used within the key, you can call `getVersionstamp()` before\nthe transaction is committed. Note that this method must be called within the same transaction that a versionstamped key\nwas written in, otherwise it won't know which versionstamp to return. Also note that this versionstamp does not include\nany user data that was associated with it, since it will be the same versionstamp no matter how many versionstamped keys\nwere written.\n\n```swift\nlet keyWithVersionstampPlaceholder = self.subspace[FDB.Versionstamp(userData: 42)][\"anotherKey\"]\nlet valueToWrite: String = \"Hello, World!\"\n\nvar versionstamp: FDB.Versionstamp = try await fdb.withTransaction { transaction in\n    try transaction.set(versionstampedKey: keyWithVersionstampPlaceholder, value: Bytes(valueToWrite.utf8))\n    try await transaction.commit()\n    return try await transaction.getVersionstamp()\n}\n\nversionstamp.userData = 42\nlet actualKey = self.subspace[versionstamp][\"anotherKey\"]\n\n// ... return it to user, save it as a reference to another entry, etc…\n```\n\n### Complete example\n\n```swift\nlet key = FDB.Subspace(\"1337\")[\"322\"]\n\nlet resultString: String = try await fdb.withTransaction { transaction in\n    try transaction.setOption(.timeout(milliseconds: 5000))\n    try transaction.setOption(.snapshotRywEnable)\n\n    transaction.set(key: key, value: Bytes([1, 2, 3]))\n\n    guard let bytes = try await transaction.get(key: key, snapshot: true) else {\n        throw MyApplicationError.Something(\"Bytes are not bytes\")\n    }\n    guard let string = String(bytes: bytes, encoding: .ascii) else {\n        throw MyApplicationError.Something(\"String is not string\")\n    }\n\n    try await transaction.commit()\n\n    return string\n}\n\nprint(\"My string is '\\(resultString)'\")\n```\n\n### Debugging/logging\n\nFDBSwift supports official community [Swift-Log](https://github.com/apple/swift-log) library with a custom backend\n[LGNLog](https://github.com/1711-Games/LGN-Log), therefore FDBSwift will use `Logger.current` logger (which may be bound\nto a TaskLocal logger via `Logger.$current.withValue(contextLogger) { ... }`) and obey log level and other LGNLogger\nconfig entries, see respective docs. \n\n## Troubleshooting\n\n### Package doesn't compile, something like `Undefined symbols for architecture` and tons of similar crap. Send help.\n\nYou haven't properly installed `pkg-config` for FoundationDB, see [Installation section](#installation).\n\n### Package does compile in macOS, but in runtime I'm getting error `The bundle “FDBTests” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle`. What do?\n\nExecute this magic command in console:\n`install_name_tool -id /usr/local/lib/libfdb_c.dylib /usr/local/lib/libfdb_c.dylib`.\n\nShoutout to [@dimitribouniol](https://github.com/dimitribouniol) and his\n[marvelous investigation](https://github.com/kirilltitov/FDBSwift/issues/70#issuecomment-726421104).\n\n### I'm getting strange error on second operation: `API version already set`. Should I rethink my life?\n\n(Rethinking hence analyzing things is always good) You tried to create more than one instance of FDB class, which is\na) prohibited\nb) not needed at all since one instance is just enough for any application\n(if not, consider horizontal scaling, FDB absolutely shouldn't be a bottleneck of your application).\nStrictly speaking, it's not very ok, there should be a way of creating more than one of FDB connection in a runtime,\nand I will definitely try to make it possible. Still, I don't think that FDB connection pooling is a good idea,\nit already does everything for you.\n\n## Warnings\n\nThough I aim for full interlanguage compatibility of Tuple layer, I don't guarantee it. During development I refered\nto Python implementation, but there might be slight differences (like unicode string and byte string packing,\nsee [design doc](https://github.com/apple/foundationdb/blob/master/design/tuple.md) on strings\nand [my comments](Tests/FDBTests/TupleTests.swift) on that). In general it's should be quite compatible already.\nProbably one day I'll spend some time on ensuring packing compatibility, but that's not high priority for me.\n\n## TODOs\n\n* Enterprise support, vendor WSDL, rewrite on ~Java~ ~Scala~ ~Kotlin~ Java 10\n* Drop enterprise support, rewrite on golang using react-native (pretty sure it will be a thing by that time)\n* Blockchain? ICO? VR? AR?\n* Rehab\n* ✅ Proper errors\n* ✅ Transactions rollback\n* ✅ Tuples\n* ✅ Tuples pack\n* ✅ Tuples unpack\n* ✅ Integer tuples\n* ✅ Ranges\n* ✅ Subspaces\n* ✅ Atomic operations\n* ✅ Tests\n* ✅ Properly test on Linux\n* ✅ 🎉 Asynchronous methods (Swift-NIO)\n* ✅ More verbose\n* ✅ Even more verbose\n* ✅ Transaction options\n* ✅ Network options\n* ✅ Docblocks and built-in documentation\n* ✅ Auto transaction retry if allowed and appropriate\n* ✅ 🎉 Even morer verbose (Swift-Log)\n* ✅ The rest of tuple pack/unpack (only floats, I think?) (also Bool and UUID)\n* ✅ Adopt `async/await`, yeah, boiiiiiiiiii\n* ✅ Drop Swift-NIO support (not because it's something bad, but because it's not really necessary here anymore;\nwe still love it tho)\n* More sugar for atomic operations\n* The rest of C API (watches?)\n* Directories\n* Drop VR support\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkirilltitov%2FFDBSwift","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkirilltitov%2FFDBSwift","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkirilltitov%2FFDBSwift/lists"}