io.mantisrx:mantis-source-job-publish

mantis-source-job-publish


License
Apache-2.0

Documentation

Mantis logo

Mantis Documentation

Build Status OSS Lifecycle License

Official Website


Development

Setting up Intellij

Run ./gradlew idea to (re-) generate IntelliJ project and module files from the templates in .baseline. The generated project is pre-configured with Baseline code style settings and support for the CheckStyle-IDEA plugin.

Install Docker

Install and run Docker to support local containers.

Building

$ ./gradlew clean build

Testing

$ ./gradlew clean test

Formatting the code

Run ./gradlew format task which autoformats all Java files using Spotless.

Building deployment into local Maven cache

$ ./gradlew clean publishNebulaPublicationToMavenLocal

Releasing

We release by tagging which kicks off a CI build. The CI build will run tests, integration tests, static analysis, checkstyle, build, and then publish to the public Bintray repo to be synced into Maven Central.

Tag format:

vMajor.Minor.Patch

You can tag via git or through Github's Release UI.

Contributing

Mantis is interested in building the community. We welcome any forms of contributions through discussions on any of our mailing lists or through patches.

For more information on contribution, check out the contributions file here.

Module Structure

This excludes all connectors and examples as they are mostly leaf nodes in the dependency graph.

Module Purpose Example Classes Package Prefixes
mantis-common-serde Support for serializing and deserializing POJOs using Json
  • JsonSerializer
  • io.mantisrx.common.*
  • mantis-discovery-proto todo: need to fill this in
  • JobDiscoveryProto
  • AppJobClustersMap
  • StageWorkers
  • com.netflix.mantis.discovery.proto.*
  • mantis-common Mantis common code shared across modules
  • MantisJobDefinition
  • MachineDefinition
  • MantisJobState
  • io.mantisrx.common.*
  • io.mantisrx.runtime.*
  • io.mantisrx.server.code.*
  • io.reactivx.mantis.operators.*
  • com.mantisrx.common.utils.*
  • mantis-remote-observable connection to other job,stage related code
  • ConnectToObservable
  • ConnectToGroupedObservable
  • RemoteObservable
  • io.reactivex.mantis.remote.observable.*
  • io.reactivex.netty.codec.*
  • mantis-control-plane-core common code between mantis-control-plane-server, mantis-control-plane-client, mantis-server-worker, mantis-server-agent
  • TaskExecutorID
  • ClusterID
  • ExecuteStageRequest
  • JobAssignmentResult
  • Status
  • io.mantisrx.server.core.*
  • io.mantisrx.server.master.resourcecluster.*
  • io.mantisrx.server.worker.*
  • mantis-control-plane-client API to talk to the mantis control plane server
  • MasterClientWrapper
  • MantisMasterGateway
  • MantisMasterClientApi
  • io.mantisrx.server.master.client.*
  • io.mantisrx.server.master.resourcecluster.*
  • mantis-network todo: need to fill this in
    mantis-publish-core todo: need to fill this in
    mantis-server-worker-client API to interact with workers
  • MetricsClient
  • MetricsClientImpl
  • WorkerConnection
  • WorkerMetricsClient
  • io.mantisrx.server.worker.client.*
  • mantis-runtime Runtime that the jobs need to depend upon. Job DSL should go in here along with how to talk to other jobs
  • KeyToKey
  • GroupToGroup
  • Source
  • Sink
  • io.mantisrx.runtime.*
  • mantis-publish-netty todo: need to fill this in
    mantis-client client to interact with mantis control plane
  • MantisClient::submitJob
  • MantisClient::killJob
  • io.mantisrx.client.*
  • mantis-publish-netty-guice todo: need to fill this in
    mantis-control-plane-server Actual server that runs the mantis master code
  • MasterMain
  • SchedulingService
  • ServiceLifecycle
  • io.mantisrx.master.*
  • io.mantisrx.server.master.*
  • mantis-server-agent Contains mantis-runtime agnostic code to start the task executor that runs on the agent
  • TaskExecutor
  • TaskExecutorStarter
  • BlobStore
  • BlobStoreAwareClassLoaderHandle
  • io.mantisrx.server.agent.*
  • mantis-server-worker One implementation of Mantis Worker that depends on the master runtime
  • MantisWorker
  • Task
  • ExecuteStageRequestService
  • JobAutoScaler
  • io.mantisrx.server.worker.config.*
  • io.mantisrx.server.worker.jobmaster.*
  • Dependency Graph

    graph TD;
        A[mantis-common-serde]-->B[mantis-common];
        B-->C[mantis-control-plane-core];
        B-->D[mantis-runtime];
        C-->E[mantis-control-plane-client];
        E-->F[mantis-server-worker-client];
        F-->G[mantis-client];
        E-->G;
        C-->H[mantis-control-plane-server];
        C-->I[mantis-server-worker];
        F-->I;
        D-->I;
        F-->J[mantis-server-agent];
        E-->J;