https://github.com/gchudnov/sqsmove
Moves SQS messages between queues
https://github.com/gchudnov/sqsmove
copy move sqs sqs-queue
Last synced: 8 months ago
JSON representation
Moves SQS messages between queues
- Host: GitHub
- URL: https://github.com/gchudnov/sqsmove
- Owner: gchudnov
- License: mit
- Created: 2021-09-10T10:58:41.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2023-03-14T16:37:13.000Z (about 3 years ago)
- Last Synced: 2025-04-03T22:39:53.067Z (about 1 year ago)
- Topics: copy, move, sqs, sqs-queue
- Language: Scala
- Homepage:
- Size: 104 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# SQSMove
[](https://github.com/gchudnov/sqsmove/actions/workflows/ci.yml)
Moves or copies messages between AWS SQS queues with the desired parallelism, preserving attributes.
## Usage
Download the binary executable or build it.
```bash
$ sqsmove --help
```
```text
sqsmove 1.2.4
Usage: sqsmove [options]
-s, --src-queue source queue name
-d, --dst-queue destination queue name
--src-dir source directory path
--dst-dir destination directory path
-p, --parallelism
parallelism (default: 16)
-c, --count number of messages to process (default: no limit)
--visibility-timeout
visibility timeout (default: 30s). Format: 1d12h35m16s
--no-delete do not delete messages after processing
--yes do not ask for confirmation
-v, --verbose verbose output
-h, --help prints this usage text
--version prints the version
Examples:
- Move messages from queue A to queue B:
sqsmove -s A -d B
- Move messages from queue A to queue B with parallelism 1:
sqsmove -s A -d B -p 1
- Copy messages from queue A to queue B with visibility timeout 5m:
sqsmove -s A -d B --no-delete --visibility-timeout=5m
- Download messages to directory D:
sqsmove -s A --dst-dir D
- Download N messages to directory D:
sqsmove -s A --dst-dir D -c N
- Download N messages to directory D without deletion:
sqsmove -s A --dst-dir D -c N --no-delete
- Upload messages from directory D:
sqsmove --src-dir D -d B
```
## Examples
**Move messages from queue A to queue B:**
```bash
sqsmove -s A -d B
```
**Move messages from queue A to queue B with parallelism 1:**
```bash
sqsmove -s A -d B -p 1
```
**Copy messages from queue A to queue B:**
```bash
sqsmove -s A -d B --no-delete
```
The flag `--no-delete` prevents the deletion of messages in the source queue. If there are *a lot* of messages in the source queue, it might be worth to increase the visibility timeout, e.g. `--visibility-timeout=5m` (the value depends on the expected time to copy all messages) to make sure that the utility is not copying the same message again when visibility timeout expires (default: 30s).
```bash
sqsmove -s A -d B --no-delete --visibility-timeout=5m
```
**Download messages to directory D:**
```bash
sqsmove -s A --dst-dir D
```
It downloads both body of the message and message attributes as two separate files and saves them to the specified directory.
If the message has id `X`, body will be saved to the file `X`, message attributes to the file `X.meta`.
Attributes are stored as a CSV with 3 columns: `name`, `type` and `value` where each line is a separate message attribute:
```text
name,type,value
strAttr,String,str
numAttr,Number,1
binAttr,Binary,QUJD
```
The supported types are `String`, `Number` and `Binary` where binary data is encoded as a Base64 string.
**Download N messages to directory D:**
```bash
sqsmove -s A --dst-dir D -c N
```
**Upload messages from directory D:**
```bash
sqsmove --src-dir D -d B
```
When uploading messages from a local directory, files are not deleted. If there is a corresponding `.meta` file is present for a file, it will be used as a source of message attributes.
**Confirm the action:**
When `sqsmove` is executed, it asks a user to confirm the action.
For example, `sqsmove --src-queue=A --dst-dir=D` generates the following output:
```text
Going to MOVE messages 'A' -> 'D'
[parallelism: 16; visibility-timeout: 30s; no-delete: false]
Are you sure? (y|N)
```
To avoid the question, provide `--yes` parameter:
```bash
sqsmove -s A -d D --yes
```
## Build
Follow the instructions, listed in [BUILD](res/graalvm/BUILD.md) readme.
## Contact
[Grigorii Chudnov](mailto:g.chudnov@gmail.com)
## License
Distributed under the [The MIT License (MIT)](LICENSE).