https://github.com/shelfio/dynamodb-query-optimized
https://github.com/shelfio/dynamodb-query-optimized
Last synced: 7 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/shelfio/dynamodb-query-optimized
- Owner: shelfio
- License: mit
- Created: 2022-01-04T15:50:34.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2025-05-06T21:13:44.000Z (8 months ago)
- Last Synced: 2025-05-06T22:25:35.155Z (8 months ago)
- Language: TypeScript
- Size: 148 KB
- Stars: 14
- Watchers: 20
- Forks: 0
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: license
Awesome Lists containing this project
README
# dynamodb-query-optimized [](https://circleci.com/gh/shelfio/dynamodb-query-optimized/tree/master)
> 2x faster DynamoDB queries when you need to query 2+ MB of data
Read the blog post article explaining how it works: https://vladholubiev.medium.com/how-to-speed-up-long-dynamodb-queries-by-2x-c66a2987d53a
## Install
```
$ yarn add @shelf/dynamodb-query-optimized
```
## Benchmark
```
Regular query: <1 MB of items: 650ms
Optimized query: <1 MB of items: 704ms
Regular query: ~21 MB of items: 9.023s
Optimized query: ~21 MB of items: 4.988s # almost 2x faster
```
## Usage
For now, it supports `aws-sdk` v2. Feel free to submit a PR to support `aws-sdk` v3!
### Optimized query for 2+ MB of data
Queries DDB from both ends of the query in parallel. Stops and returns results when the middle is reached.
```js
import {queryOptimized} from '@shelf/dynamodb-query-optimized';
import DynamoDB from 'aws-sdk/clients/dynamodb';
const ddb = new DynamoDB.DocumentClient({region: 'us-east-1'});
const results = await queryOptimized({
queryFunction: ddb.query.bind(ddb),
queryParams: {
TableName: 'example_table',
ProjectionExpression: 'hash_key, range_key',
KeyConditionExpression: '#hash_key = :hash_key AND begins_with(#range_key, :range_key)',
ExpressionAttributeNames: {
'#hash_key': 'hash_key',
'#range_key': 'range_key',
},
ExpressionAttributeValues: {
':hash_key': hash_key,
':range_key': range_key,
},
},
});
console.log(results);
/*
[{hash_key: 'foo', range_key: 'bar'}, {hash_key: 'foo', range_key: 'baz'}]
*/
```
### Regular query for <2 MB of data
Queries DDB and continues to paginate through all results until query is exhausted.
```js
import {queryRegular} from '@shelf/dynamodb-query-optimized';
import DynamoDB from 'aws-sdk/clients/dynamodb';
const ddb = new DynamoDB.DocumentClient({region: 'us-east-1'});
const results = await queryRegular({
queryFunction: ddb.query.bind(ddb),
queryParams: {
TableName: 'example_table',
ProjectionExpression: 'hash_key, range_key',
KeyConditionExpression: '#hash_key = :hash_key AND begins_with(#range_key, :range_key)',
ExpressionAttributeNames: {
'#hash_key': 'hash_key',
'#range_key': 'range_key',
},
ExpressionAttributeValues: {
':hash_key': hash_key,
':range_key': range_key,
},
},
});
console.log(results);
/*
[{hash_key: 'foo', range_key: 'bar'}, {hash_key: 'foo', range_key: 'baz'}]
*/
```
## Publish
```sh
$ git checkout master
$ yarn version
$ yarn publish
$ git push origin master --tags
```
## License
MIT © [Shelf](https://shelf.io)