{"id":19293956,"url":"https://github.com/igrishaev/dynamodb","last_synced_at":"2025-04-22T07:32:32.246Z","repository":{"id":65813225,"uuid":"597037144","full_name":"igrishaev/dynamodb","owner":"igrishaev","description":"Pure Clojure driver for Dynamo DB (with GraalVM/native-image support)","archived":false,"fork":false,"pushed_at":"2023-03-08T15:34:58.000Z","size":194,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-01T20:51:26.638Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/igrishaev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-03T13:43:36.000Z","updated_at":"2023-07-27T12:08:50.000Z","dependencies_parsed_at":"2024-11-09T22:36:50.870Z","dependency_job_id":null,"html_url":"https://github.com/igrishaev/dynamodb","commit_stats":{"total_commits":116,"total_committers":1,"mean_commits":116.0,"dds":0.0,"last_synced_commit":"19d69f331c4f966ed70bd2e8562e8f38ad1a30f0"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igrishaev%2Fdynamodb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igrishaev%2Fdynamodb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igrishaev%2Fdynamodb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igrishaev%2Fdynamodb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/igrishaev","download_url":"https://codeload.github.com/igrishaev/dynamodb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250195054,"owners_count":21390230,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T22:36:40.679Z","updated_at":"2025-04-22T07:32:32.239Z","avatar_url":"https://github.com/igrishaev.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DynamoDB\n\n[dynamodb]: https://aws.amazon.com/dynamodb/\n\nA [DynamoDB][dynamodb] driver written in pure Clojure. Lightweight\ndependencies. GraalVM/native-image friendly.\n\n## Table of Contents\n\n\u003c!-- toc --\u003e\n\n- [Benefits](#benefits)\n- [Installation](#installation)\n- [Documentation](#documentation)\n- [API Implemented](#api-implemented)\n- [Who Uses It](#who-uses-it)\n- [Usage](#usage)\n  * [Encoding](#encoding)\n  * [Decoding](#decoding)\n  * [The Client](#the-client)\n  * [Create a Table](#create-a-table)\n  * [List Tables](#list-tables)\n  * [Put Item](#put-item)\n  * [Get Item](#get-item)\n  * [Update Item](#update-item)\n    + [Set Attributes](#set-attributes)\n    + [Add Attributes](#add-attributes)\n    + [Remove Attributes](#remove-attributes)\n    + [Delete Attributes](#delete-attributes)\n  * [Delete Item](#delete-item)\n  * [Query](#query)\n  * [Scan](#scan)\n  * [Other API](#other-api)\n- [Raw API access](#raw-api-access)\n- [Specs](#specs)\n- [Tests](#tests)\n\n\u003c!-- tocstop --\u003e\n\n## Benefits\n\n[http-kit]: https://github.com/http-kit/http-kit\n[cheshire]: https://github.com/dakrone/cheshire\n\n[native-image]: https://www.graalvm.org/22.0/reference-manual/native-image/\n[ydb]: https://cloud.yandex.com/en-ru/services/ydb\n\n- Free from AWS SDK. Everything is implemented with pure JSON + HTTP.\n- Quite narrow dependencies: just [HTTP Kit][http-kit] and [Cheshire][cheshire].\n- Compatible with [Native Image][native-image]! Thus, easy to use as a binary\n  file in AWS Lambda.\n- Clojure-friendly: supports fully qualified keyword attributes and handles\n  properly them in SQL expressions.\n- Both encoding \u0026 decoding are extendable with protocols \u0026 multimethods.\n- Raw API access for special cases.\n- Specs for better input validation.\n- Compatible with [Yandex DB][ydb].\n\n## Installation\n\nLeiningen/Boot:\n\n```\n[com.github.igrishaev/dynamodb \"0.1.4\"]\n```\n\nClojure CLI/deps.edn:\n\n```\ncom.github.igrishaev/dynamodb {:mvn/version \"0.1.4\"}\n```\n\n## Documentation\n\n[docs]: https://cljdoc.org/d/com.github.igrishaev/dynamodb/0.1.4/doc/readme\n\n[At cljdoc.org][docs] (automatic build).\n\n## API Implemented\n\n[api-ops]: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations_Amazon_DynamoDB.html\n\nAt the moment, only the most important [API targets][api-ops] are\nimplemented. The rest of them is a matter of time and copy-paste. Let me know if\nyou need something missing in the table below.\n\n\u003cdetails\u003e\n\u003csummary\u003eCheck out the table\u003c/summary\u003e\n\n| Target                              | Done? | Comment |\n|-------------------------------------|-------|---------|\n| BatchExecuteStatement               |       |         |\n| BatchGetItem                        | +     |         |\n| BatchWriteItem                      |       |         |\n| CreateBackup                        | +     |         |\n| CreateGlobalTable                   |       |         |\n| CreateTable                         | +     |         |\n| DeleteBackup                        |       |         |\n| DeleteItem                          | +     |         |\n| DeleteTable                         | +     |         |\n| DescribeBackup                      | +     |         |\n| DescribeContinuousBackups           |       |         |\n| DescribeContributorInsights         |       |         |\n| DescribeEndpoints                   |       |         |\n| DescribeExport                      |       |         |\n| DescribeGlobalTable                 |       |         |\n| DescribeGlobalTableSettings         |       |         |\n| DescribeImport                      |       |         |\n| DescribeKinesisStreamingDestination |       |         |\n| DescribeLimits                      |       |         |\n| DescribeTable                       | +     |         |\n| DescribeTableReplicaAutoScaling     |       |         |\n| DescribeTimeToLive                  |       |         |\n| DisableKinesisStreamingDestination  |       |         |\n| EnableKinesisStreamingDestination   |       |         |\n| ExecuteStatement                    |       |         |\n| ExecuteTransaction                  |       |         |\n| ExportTableToPointInTime            |       |         |\n| GetItem                             | +     |         |\n| ImportTable                         |       |         |\n| ListBackups                         |       |         |\n| ListContributorInsights             |       |         |\n| ListExports                         |       |         |\n| ListGlobalTables                    |       |         |\n| ListImports                         |       |         |\n| ListTables                          | +     |         |\n| ListTagsOfResource                  |       |         |\n| PutItem                             | +     |         |\n| Query                               | +     |         |\n| RestoreTableFromBackup              |       |         |\n| RestoreTableToPointInTime           |       |         |\n| Scan                                | +     |         |\n| TagResource                         | +     |         |\n| TransactGetItems                    |       |         |\n| TransactWriteItems                  |       |         |\n| UntagResource                       |       |         |\n| UpdateContinuousBackups             |       |         |\n| UpdateContributorInsights           |       |         |\n| UpdateGlobalTable                   |       |         |\n| UpdateGlobalTableSettings           |       |         |\n| UpdateItem                          | +     |         |\n| UpdateTable                         |       |         |\n| UpdateTableReplicaAutoScaling       |       |         |\n| UpdateTimeToLive                    |       |         |\n\n\u003c/details\u003e\n\n## Who Uses It\n\n[teleward]: https://github.com/igrishaev/teleward\n\nDynamoDB is a part of [Teleward][teleward] — a Telegram captcha bot. The bot is\nhosted in Yandex Cloud as a binary file compiled with GraalVM. It uses the\nlibrary to track the state in Yandex DB. In turn, Yandex DB is a cloud database\nthat mimics DynamoDB and serves a subset of its HTTP API.\n\n## Usage\n\nFirst, import the library:\n\n```clojure\n(require '[dynamodb.api :as api])\n(require '[dynamodb.constant :as const])\n```\n\nThe `constant` module is needed sometimes to refer to common DynamoDB values\nlike `\"PAY_PER_REQUEST\"`, `\"PROVISIONED\"` and so on.\n\n### Encoding\n\nThe `dynamodb.encode` namespace provides the `IEncode` protocol with the\n`(-encode [this])` method. The library extends the `Boolean`, `String`, `Number`\nand other types to encode them properly. There is also a default implementation\nfor `Object` type that encodes any value as a string.\n\nTo establish your own encoding rules for some certain type, extend it with\nprotocol as follows:\n\n```clojure\n(extend-protocol IEncode\n  MyDateTimeType\n  (-encode [this]\n    (-encode (.milliseconds this))))\n```\n\nIn the example above, a custom `MyDateTimeType` type gets transferred into\nmilliseconds and gets encoded as a number. Thus, the result will be something\nlike `{:N \"1676628480718\"}`.\n\n### Decoding\n\nAt the moment, the values are decoded using the `case` form with no way for\nextension. This a subject to improve/rework this behaviour (see the\n`dynamodb.decode` namespace).\n\n### The Client\n\nPrepare a client object. The first four parameters are mandatory:\n\n```clojure\n(def CLIENT\n  (api/make-client \"aws-public-key\"\n                   \"aws-secret-key\"\n                   \"https://aws.dynamodb.endpoint.com/some/path\"\n                   \"aws-region\"\n                   {...}))\n```\n\nFor Yandex DB, the region is something like \"ru-central1\".\n\nBoth public and secret AWS keys are masked with a special wrapper that prevents\nthem from being logged or printed.\n\nThe fifth parameter is a map of options to override:\n\n| Parameter   | Default      | Description                                    |\n|-------------|--------------|------------------------------------------------|\n| `:throw?`   | `true`       | Whether to throw a negative DynamoDB response. |\n| `:version`  | `\"20120810\"` | DynamoDB API version.                          |\n| `:http-opt` | (see below)  | A map of HTTP Kit default settings.            |\n\nThe default HTTP settings are:\n\n```clojure\n{:user-agent \"com.github.igrishaev/dynamodb\"\n :keepalive (* 30 1000)\n :insecure? true\n :follow-redirects false}\n```\n\n### Create a Table\n\nTo create a new table, pass its name, the schema map, and the primary key\nmapping:\n\n```clojure\n(api/create-table CLIENT\n                  \"SomeTable\"\n                  {:user/id :N\n                   :user/name :S}\n                  {:user/id const/key-type-hash\n                   :user/name const/key-type-range}\n                  {:tags {:foo \"hello\"}\n                   :table-class const/table-class-standard\n                   :billing-mode const/billing-mode-pay-per-request})\n```\n\n### List Tables\n\nTables can be listed by pages. The default page size is 100. Once you've reached\nthe limit, check out the `LastEvaluatedTableName` field. Pass it to the\n`:start-table` optional argument to propagate to the next page:\n\n```clojure\n(def resp1\n  (api/list-tables CLIENT {:limit 10}))\n\n(def last-table\n  (:LastEvaluatedTableName resp1))\n\n(def resp2\n  (api/list-tables CLIENT\n                   {:limit 10\n                    :start-table last-table}))\n```\n\n### Put Item\n\nTo upsert an item, pass a map that contains the primary attributes:\n\n```clojure\n(api/put-item CLIENT\n              \"SomeTable\"\n              {:user/id 1\n               :user/name \"Ivan\"\n               :user/foo 1}\n              {:return-values const/return-values-none})\n```\n\nPass `:sql-condition` to make the operation conditional. In the example above,\nthe `:user/foo` attribute is 1. The second upsert operation checks if\n`:user/foo` is either 1, 2, or 3, which is true. Thus, it will fail:\n\n```clojure\n(api/put-item CLIENT\n              \"SomeTable\"\n              {:user/id 1\n               :user/name \"Ivan\"\n               :user/test 3}\n              {:sql-condition \"#foo in (:one, :two, :three)\"\n               :attr-names {\"#foo\" :user/foo}\n               :attr-values {\":one\" 1\n                             \":two\" 2\n                             \":three\" 3}\n               :return-values const/return-values-all-old})\n```\n\n### Get Item\n\nTo get an item, provide its primary key:\n\n```clojure\n(api/get-item CLIENT\n              \"SomeTable\"\n              {:user/id 1\n               :user/name \"Ivan\"})\n\n{:Item #:user{:id 1\n              :name \"Ivan\"\n              :foo 1}}\n```\n\nThere is an option to get only the attributes you need or even sub-attributes\nfor nested maps or lists:\n\n```clojure\n;; put some complex values\n(api/put-item CLIENT\n              \"SomeTable\"\n              {:user/id 1\n               :user/name \"Ivan\"\n               :test/kek \"123\"\n               :test/foo 1\n               :abc nil\n               :foo \"lol\"\n               :bar {:baz [1 2 3]}})\n\n;; pass a list of attributes/paths into the `:attrs-get` param\n(api/get-item CLIENT\n              \"SomeTable\"\n              {:user/id 1\n               :user/name \"Ivan\"}\n              {:attrs-get [:test/kek \"bar.baz[1]\" \"abc\" \"foo\"]})\n\n;; the result:\n{:Item {:test/kek \"123\"\n        :bar {:baz [2]}\n        :abc nil\n        :foo \"lol\"}}\n```\n\n### Update Item\n\n[faraday]: https://github.com/Taoensso/faraday\n\nThis operation is the most complex. In AWS SDK or [Faraday][faraday], to update\nan item's secondary attributes, one should manually build a SQL expression that\ninvolves string formatting, concatenation and similar boring stuff.\n\n```sql\nSET username = :username, email = :email, ...\n```\n\nThe `ADD`, `DELETE`, and `REMOVE` expressions require manual work as well.\n\nThe present library solves this problem for you. The `update-item` function\naccepts `:add`, `:set`, `:delete`, and `:remove` parameters, either maps or\nvectors.\n\nThe `:sql-condition` argument accepts a plain SQL expression. Should it\nevaluates as falseness, the item won't be affected and you'll get a negative\nresponse.\n\n#### Set Attributes\n\n```clojure\n(api/update-item CLIENT\n                 table\n                 {:user/id 1\n                  :user/name \"Ivan\"}\n                 {:attr-names {\"#counter\" :test/counter}\n                  :attr-values {\":one\" 1}\n                  :set {\"Foobar\" 123\n                        :user/email \"test@test.com\"\n                        \"#counter\" (api/sql \"#counter + :one\")}})\n```\n\nThe example above covers three various options for the `:set` argument. Namely:\n\n1. The attribute is a plain string `(\"Foobar\")`, and the value is plain as well.\n2. The attribute is a complex keyword (`:user/email`) which cannot be placed in\n   a SQL expression directly. Under the hood, the library produces an alias for\n   it and injects it into `ExpressionAttributeNames`.\n3. The attribute is an alias, and the value is a raw expression. To distinguish\n   an expression from a regular string (e.g. email), there is a wrapper\n   `api/sql`. The alias `#counter` should be declared in the `:attr-names` map.\n\n#### Add Attributes\n\nThe `:add` parameter accepts a map of an attribute or an alias to a\nvalue. Imagine you have the following item in the db:\n\n```clojure\n{:user/id 1\n :user/name \"Ivan\"\n :amount 3\n :test/colors #{\"r\" \"g\"}}\n```\n\nTo increase the amount and add a new color into the colors set, perfrom:\n\n```clojure\n(api/update-item CLIENT\n                 table\n                 {:user/id 1\n                  :user/name \"Ivan\"}\n                 {:add {\"amount\" 1\n                        :test/colors #{\"b\"}}})\n```\n\nResult:\n\n```clojure\n{:Item\n {:amount 4\n  :user/id 1\n  :test/colors #{\"b\" \"r\" \"g\"}\n  :user/name \"Ivan\"}}\n```\n\n#### Remove Attributes\n\nTo remove an attribute, pass the `:remove` vector. Each item of that vector is\neither a keyword attribute, a raw string expression, or an alias.\n\n```clojure\n(api/update-item CLIENT\n                 table\n                 {:user/id 1\n                  :user/name \"Ivan\"}\n                 {:attr-names {\"#kek\" :test/kek}\n                  :remove [\"#kek\" \"abc\" :test/lol]})\n```\n\nTo remove an item from a list, pass a string like this:\n\n```clojure\n;; item in the databalse\n{:tags [\"foo\" \"bar\" \"baz\"]}\n\n{:remove [\"tags[1]\"]}\n```\n\nUse an alias when the attribute is a keyword with a namespace:\n\n```clojure\n(api/update-item CLIENT\n                 table\n                 {:user/id 1\n                  :user/name \"Ivan\"}\n                 {:attr-names {\"#tags\" :user/tags}\n                  :remove [\"#tags[1]\"]})\n```\n\n#### Delete Attributes\n\nIn DynamoDB, the `DELETE` clause is used to remove items from sets. The\n`update-item` function accepts the `:delete` argument which is a map. The key is\neither a keyword or a string alias. The value is always a set:\n\nThe item:\n\n```clojure\n{:user/id 1\n :user/name \"Ivan\"\n :user/colors #{\"r\" \"g\" \"b\"}}\n```\n\nAPI call:\n\n```clojure\n(api/update-item CLIENT\n                 table\n                 {:user/id 1\n                  :user/name \"Ivan\"}\n                 {:delete {:user/colors #{\"r\" \"b\"}}})\n```\n\nResult:\n\n```clojure\n{:Item #:user{:colors #{\"g\"} :id 1 :name \"Ivan\"}}\n```\n\n### Delete Item\n\nSimple deletion of an item:\n\n```clojure\n(api/delete-item CLIENT\n                 table\n                 {:user/id 1 :user/name \"Ivan\"})\n```\n\nConditional deletion: throws an exception when the expression fails.\n\n```clojure\n(api/put-item CLIENT\n              table\n              {:user/id 1\n               :user/name \"Ivan\"\n               :test/kek 99})\n\n(api/delete-item CLIENT\n                 table\n                 {:user/id 1 :user/name \"Ivan\"}\n                 {:sql-condition \"#kek in (:foo, :bar, :baz)\"\n                  :attr-names {\"#kek\" :test/kek}\n                  :attr-values {\":foo\" 1\n                                \":bar\" 2\n                                \":baz\" 3}})\n```\n\nIn the example above, the `\"#kek in (:foo, :bar, :baz)\"` expression fails as the\n`:test/kek` attribute is of value 99. The item stays in the database, and you'll\nget an exception with ex-info:\n\n```clojure\n{:error? true\n :status 400\n :path \"com.amazonaws.dynamodb.v20120810\"\n :exception \"ConditionalCheckFailedException\"\n :message \"The conditional request failed\"\n :payload\n {:TableName table\n  :Key #:user{:id {:N \"1\"} :name {:S \"Ivan\"}}\n  :ConditionExpression \"#kek in (:foo, :bar, :baz)\"\n  :ExpressionAttributeNames {\"#kek\" :test/kek}\n  :ExpressionAttributeValues\n  {\":foo\" {:N \"1\"} \":bar\" {:N \"2\"} \":baz\" {:N \"3\"}}}\n :target \"DeleteItem\"}\n```\n\n### Query\n\nThe Query target allows searching items that match a primary key partially or\nmatch some range. Imagine the primary key of a table is `:user/id :HASH` and\n`:user/name :RANGE`. Here is what you have in the database:\n\n```clojure\n{:user/id 1\n :user/name \"Ivan\"\n :test/foo 1}\n\n{:user/id 1\n :user/name \"Juan\"\n :test/foo 2}\n\n{:user/id 2\n :user/name \"Huan\"\n :test/foo 3}\n```\n\nNow, to find the items whose `:user/id` is 1, execute:\n\n```clojure\n(api/query CLIENT\n           table\n           {:sql-key \"#id = :one\"\n            :attr-names {\"#id\" :user/id}\n            :attr-values {\":one\" 1}\n            :limit 1})\n```\n\nResult:\n\n```\n{:Items [{:user/id 1\n          :test/foo 1\n          :user/name \"Ivan\"}]\n :Count 1\n :ScannedCount 1\n :LastEvaluatedKey #:user{:id 1\n                          :name \"Ivan\"}}\n```\n\nTo propagate to the next page, fetch the `LastEvaluatedKey` field from the\nresult and pass it into the `:start-key` Query parameter.\n\n\n### Scan\n\nThe Scan API goes through the whole table collecting the items that match an\nexpression. This is not optimal yet required sometimes.\n\n```clojure\n(api/scan CLIENT\n          table\n          {:sql-filter \"#foo = :two\"\n           :attrs-get [:test/foo \"#name\"]\n           :attr-names {\"#foo\" :test/foo\n                        \"#name\" :user/name}\n           :attr-values {\":two\" 2}\n           :limit 2})\n```\n\nResult:\n\n```\n{:Items [{:test/foo 2\n          :user/name \"Ivan\"}]\n :Count 1\n :ScannedCount 2\n :LastEvaluatedKey #:user{:id 1\n                          :name \"Ivan\"}}\n```\n\nBoth `LastEvaluatedKey` and `:start-key` parameters work as described above.\n\n### Other API\n\nSee the tests, specs, and `dynamodb.api` module for more information.\n\n## Raw API access\n\nThe `api-call` function allows you to interact with DynamoDB on a low level. It\naccepts the client, the target name, and a raw payload you'd like to send to\nDB. The payload gets sent as-is with no kind of processing or interference.\n\n```clojure\n(api/api-call CLIENT\n             \"NotImplementedTarget\"\n             {:ParamFoo ... :ParamBar ...})\n```\n\n## Specs\n\nThe library provides a number of specs for the API. Find them in the\n`dynamodb.spec` module. It's not imported by default to prevent the binary file\nfrom growing when compiled with GraalVM. That's a known issue when introducing\n`clojure.spec` adds +20 Mbs to the file.\n\nStill, those specs are useful for testing and documentation. Import the specs,\nthen instrument the functions by calling the `instrument` function:\n\n```clojure\n(require 'dynamodb.spec)\n(require '[clojure.spec.test.alpha :as spec.test])\n\n(spec.test/instrument)\n```\n\nNow if you pass something wrong into one of the library functions, you'll get a\nspec exception.\n\n## Tests\n\nThe primary testing module called `api_test.clj` relies on a local DynamoDB\ninstance running in Docker. To bootstrap it, execute the command:\n\n```bash\nmake docker-up\n```\n\nIt spawns `amazon/dynamodb-local` image on port 8000. Now connect to the REPL\nand run the API tests from your editor as usual.\n\nIvan Grishaev, 2023\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Figrishaev%2Fdynamodb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Figrishaev%2Fdynamodb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Figrishaev%2Fdynamodb/lists"}