https://github.com/emc-mongoose/mongoose
Mongoose is a storage performance testing tool
https://github.com/emc-mongoose/mongoose
atmos bundle docker-image ecs filesystem openstack-swift s3
Last synced: about 1 year ago
JSON representation
Mongoose is a storage performance testing tool
- Host: GitHub
- URL: https://github.com/emc-mongoose/mongoose
- Owner: emc-mongoose
- License: mit
- Created: 2016-03-23T10:03:53.000Z (about 10 years ago)
- Default Branch: master
- Last Pushed: 2025-03-13T10:22:31.000Z (about 1 year ago)
- Last Synced: 2025-03-30T01:11:18.423Z (about 1 year ago)
- Topics: atmos, bundle, docker-image, ecs, filesystem, openstack-swift, s3
- Language: Dockerfile
- Homepage:
- Size: 786 MB
- Stars: 49
- Watchers: 13
- Forks: 10
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://gitlab.com/emc-mongoose/mongoose/commits/master)
[](https://hub.docker.com/r/emcmongoose/mongoose/)
[](https://gitter.im/emc-mongoose)
# Mongoose Bundle
The repo contains the automation scripts to build/test/deploy the Mongoose backward compatibility bundle. Previously the
repo contained the Mongoose sources for the basic functionality and some commonly used extensions. Currently it was
split into the independent repos and the corresponding components. Each component has its own documentation, CI versioning. For the
mongoose documentation refer this [link](https://github.com/emc-mongoose/mongoose-base/tree/master/doc).
# Bundle Contents
The components listed below are included in this backward compatibility bundle.
| Repo | Description | Latest Release | Integration Status | Issue Tracker Link |
|------|-------------|---------|-------------------------------|--------|
| [mongoose-**base**](https://github.com/emc-mongoose/mongoose-base) | Mongoose storage performance testing tool - base functionality |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-load-step-**pipeline**](https://github.com/emc-mongoose/mongoose-load-step-pipeline) | Load operations pipeline (create,delay,read-then-update, for example), extension |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-load-step-**weighted**](https://github.com/emc-mongoose/mongoose-load-step-weighted) | Weighted load extension, allowing to generate 20% write and 80% read operations, for example |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**coop**](https://github.com/emc-mongoose/mongoose-storage-driver-coop) | Cooperative multitasking storage driver primitive, utilizing [fibers](https://github.com/akurilov/fiber4j) |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**netty**](https://github.com/emc-mongoose/mongoose-storage-driver-netty) | [Netty](https://netty.io/)-storage-driver-nettyd storage driver primitive, extends the cooperative multitasking storage driver primitive |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**nio**](https://github.com/emc-mongoose/mongoose-storage-driver-nio) | Non-blocking I/O storage driver primitive, extends the cooperative multitasking storage driver primitive |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**http**](https://github.com/emc-mongoose/mongoose-storage-driver-http) | HTTP storage driver primitive, extends the Netty-storage-driver-httpd storage driver primitive |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**fs**](https://github.com/emc-mongoose/mongoose-storage-driver-fs) | [VFS](https://www.oreilly.com/library/view/understanding-the-linux/0596005652/ch12s01.html) storage driver, extends the NIO storage driver primitive |  |  | [FS](https://mongoose-issues.atlassian.net/projects/FS)
| [mongoose-storage-driver-**atmos**](https://github.com/emc-mongoose/mongoose-storage-driver-atmos) | [Dell EMC Atmos](https://poland.emc.com/collateral/software/data-sheet/h5770-atmos-ds.pdf) storage driver, extends the HTTP storage driver primitive |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**s3**](https://github.com/emc-mongoose/mongoose-storage-driver-s3) | [Amazon S3](https://docs.aws.amazon.com/en_us/AmazonS3/latest/API/Welcome.html) storage driver, extends the HTTP storage driver primitive |  |  | [S3](https://mongoose-issues.atlassian.net/projects/S3)
| [mongoose-storage-driver-**swift**](https://github.com/emc-mongoose/mongoose-storage-driver-swift) | [OpenStack Swift](https://wiki.openstack.org/wiki/Swift) storage driver, extends the HTTP storage driver primitive |  |  | [SWIFT](https://mongoose-issues.atlassian.net/projects/SWIFT)
# Additional Extensions
The *additional extension* are not included in this bundle.
| Repo | Description | Latest Release | Integration Status | Issue Tracker Link |
|------|-------------|---------|-------------------------------|--------|
| [mongoose-storage-driver-**preempt**](https://github.com/emc-mongoose/mongoose-storage-driver-preempt) | Preemptive multitasking storage driver primitive, using thread-per-task approach for the I/O |  |  | [BASE](https://mongoose-issues.atlassian.net/projects/BASE)
| [mongoose-storage-driver-**hdfs**](https://github.com/emc-mongoose/mongoose-storage-driver-hdfs) | [Apache HDFS](http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) storage driver, extends the NIO storage driver primitive |  |  | [HDFS](https://mongoose-issues.atlassian.net/projects/HDFS)
| [mongoose-storage-driver-**pravega**](https://github.com/emc-mongoose/mongoose-storage-driver-pravega) | [Pravega](http://pravega.io) storage driver, extends the preemptive multitasking storage driver primitive |  |  | [PRAVEGA](https://mongoose-issues.atlassian.net/projects/PRAVEGA)
| [mongoose-storage-driver-**kafka**](https://github.com/emc-mongoose/mongoose-storage-driver-kafka) | [Apache Kafka](https://kafka.apache.org/) storage driver, extends the preemptive multitasking storage driver primitive |  |  | [KAFKA](https://mongoose-issues.atlassian.net/projects/KAFKA)
| [mongoose-storage-driver-**pulsar**](https://github.com/emc-mongoose/mongoose-storage-driver-pulsar) | [Apache Pulsar](https://pulsar.apache.org/) storage driver, extends the cooperative multitasking storage driver primitive |  |  | [PULSAR](https://mongoose-issues.atlassian.net/projects/PULSAR)
# Auxiliary Tools
| Repo | Description | Latest Release | Integration Status | Issue Tracker Link |
|------|-------------|---------|-------------------------------|--------|
| [darzee](https://github.com/emc-mongoose/darzee) | Mongoose GUI web application | TBD | TBD | [GUI](https://mongoose-issues.atlassian.net/browse/GUI)
| [mongoose-helm-charts](https://github.com/emc-mongoose/mongoose-helm-charts) | [Helm](https://helm.sh/) charts to easily deploy Mongoose in [K8s](https://kubernetes.io/) environment | TBD | TBD | [HELM](https://mongoose-issues.atlassian.net/projects/HELM/issues)
| [e2e-latency-generator](https://github.com/emc-mongoose/e2e-latency-generator) | The tool consuming the Mongoose's operations trace output data and producing the raw end-to-end latency data and heatmap chart | - | - | -
| [scenario-converter-3to4](https://github.com/emc-mongoose/scenario-converter-3to4) | This tool converts the Json-scenarios used in the Mongoose v3.* into new JavaScript-scenarios. | - | - | -
# Backward Compatibility Notes
* The extensions are not overriding the base default options when launched from the jar file. E.g. the default storage
port is 7 and the default storage driver is "dummy-mock". Override the defaults explicitly or consider using the
Docker image.
* The base Mongoose version and this bundle version may differ. The base version is used to determine the logs output
path.
Example:
```bash
java -jar mongoose-bundle-.jar \
--storage-driver-type=s3 \
--storage-net-node-port=9020
```
# Build bundle
```bash
./gradlew clean jar
ls -l build/libs
```
# Deploy
## Bare Jar Download
https://repo.maven.apache.org/maven2/com/github/emc-mongoose/mongoose-bundle/
## Docker
```bash
docker run ... emcmongoose/mongoose[:] ...
```
# Dependency
The following dependency graph should be considered when running Mongoose w/o Docker and using some specific extension.
For example, using the `mongoose-storage-driver-hdfs` extension will require to have the `mongoose-storage-driver-coop`
and `mongoose-storage-driver-nio` extensions in the `~/.mongoose//ext` directory (with proper versions).
```
mongoose-base
|___ mongoose-load-step-pipeline
|___ mongoose-load-step-weighted
|___ mongoose-storage-driver-coop
| |___ mongoose-storage-driver-netty
| | |___ mongoose-storage-driver-http
| | |___ mongoose-storage-driver-atmos
| | |___ mongoose-storage-driver-s3
| | |___ mongoose-storage-driver-swift
| |___ mongoose-storage-driver-nio
| | |___ mongoose-storage-driver-fs
| | |___ mongoose-storage-driver-hdfs
| |___ mongoose-storage-driver-pulsar
|___ mongoose-storage-driver-preempt
|___ mongoose-storage-driver-kafka
|___ mongoose-storage-driver-pravega
```