com.github.emc-mongoose:mongoose-storage-driver-net-http-swift

Mongoose is a high-load storage performance testing tool


Keywords
atmos, bundle, docker-image, ecs, filesystem, openstack-swift, s3
License
MIT

Documentation

Maven metadata URL CI status Docker Pulls Gitter chat

Mongoose Bundle

The repo contains the automation scripts to build/test/deploy the Mongoose backward compatibility bundle. Previously the repo contained the Mongoose sources for the basic functionality and some commonly used extensions. Currently it was split into the independent repos and the corresponding components. Each component has its own documentation, CI versioning. For the mongoose documentation refer this link.

Bundle Contents

The components listed below are included in this backward compatibility bundle.

Repo Description Latest Release Integration Status Issue Tracker Link
mongoose-base Mongoose storage performance testing tool - base functionality Maven metadata URL Gitlab pipeline status BASE
mongoose-load-step-pipeline Load operations pipeline (create,delay,read-then-update, for example), extension Maven metadata URL Gitlab pipeline status BASE
mongoose-load-step-weighted Weighted load extension, allowing to generate 20% write and 80% read operations, for example Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-coop Cooperative multitasking storage driver primitive, utilizing fibers Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-netty Netty-storage-driver-nettyd storage driver primitive, extends the cooperative multitasking storage driver primitive Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-nio Non-blocking I/O storage driver primitive, extends the cooperative multitasking storage driver primitive Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-http HTTP storage driver primitive, extends the Netty-storage-driver-httpd storage driver primitive Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-fs VFS storage driver, extends the NIO storage driver primitive Maven metadata URL Gitlab pipeline status FS
mongoose-storage-driver-atmos Dell EMC Atmos storage driver, extends the HTTP storage driver primitive Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-s3 Amazon S3 storage driver, extends the HTTP storage driver primitive Maven metadata URL Gitlab pipeline status S3
mongoose-storage-driver-swift OpenStack Swift storage driver, extends the HTTP storage driver primitive Maven metadata URL Gitlab pipeline status SWIFT

Additional Extensions

The additional extension are not included in this bundle.

Repo Description Latest Release Integration Status Issue Tracker Link
mongoose-storage-driver-preempt Preemptive multitasking storage driver primitive, using thread-per-task approach for the I/O Maven metadata URL Gitlab pipeline status BASE
mongoose-storage-driver-hdfs Apache HDFS storage driver, extends the NIO storage driver primitive Maven metadata URL Gitlab pipeline status HDFS
mongoose-storage-driver-pravega Pravega storage driver, extends the preemptive multitasking storage driver primitive Maven metadata URL Gitlab pipeline status PRAVEGA
mongoose-storage-driver-kafka Apache Kafka storage driver, extends the preemptive multitasking storage driver primitive Maven metadata URL Gitlab pipeline status KAFKA
mongoose-storage-driver-pulsar Apache Pulsar storage driver, extends the cooperative multitasking storage driver primitive Maven metadata URL Gitlab pipeline status PULSAR

Auxiliary Tools

Repo Description Latest Release Integration Status Issue Tracker Link
darzee Mongoose GUI web application TBD TBD GUI
mongoose-helm-charts Helm charts to easily deploy Mongoose in K8s environment TBD TBD HELM
e2e-latency-generator The tool consuming the Mongoose's operations trace output data and producing the raw end-to-end latency data and heatmap chart - - -
scenario-converter-3to4 This tool converts the Json-scenarios used in the Mongoose v3.* into new JavaScript-scenarios. - - -

Backward Compatibility Notes

  • The extensions are not overriding the base default options when launched from the jar file. E.g. the default storage port is 7 and the default storage driver is "dummy-mock". Override the defaults explicitly or consider using the Docker image.

  • The base Mongoose version and this bundle version may differ. The base version is used to determine the logs output path.

Example:

java -jar mongoose-bundle-<BUNDLE_VERSION>.jar \
    --storage-driver-type=s3 \
    --storage-net-node-port=9020

Build bundle

./gradlew clean jar
ls -l build/libs

Deploy

Bare Jar Download

https://repo.maven.apache.org/maven2/com/github/emc-mongoose/mongoose-bundle/

Docker

docker run ... emcmongoose/mongoose[:<VERSION>] ...

Dependency

The following dependency graph should be considered when running Mongoose w/o Docker and using some specific extension. For example, using the mongoose-storage-driver-hdfs extension will require to have the mongoose-storage-driver-coop and mongoose-storage-driver-nio extensions in the ~/.mongoose/<BASE_VERSION>/ext directory (with proper versions).

mongoose-base
    |___ mongoose-load-step-pipeline
    |___ mongoose-load-step-weighted
    |___ mongoose-storage-driver-coop
    |       |___ mongoose-storage-driver-netty
    |       |       |___ mongoose-storage-driver-http
    |       |               |___ mongoose-storage-driver-atmos
    |       |               |___ mongoose-storage-driver-s3
    |       |               |___ mongoose-storage-driver-swift
    |       |___ mongoose-storage-driver-nio
    |       |       |___ mongoose-storage-driver-fs
    |       |       |___ mongoose-storage-driver-hdfs
    |       |___ mongoose-storage-driver-pulsar
    |___ mongoose-storage-driver-preempt
            |___ mongoose-storage-driver-kafka
            |___ mongoose-storage-driver-pravega