Branch	Builds
Develop
Master

The Red Hat Marketplace Operators are a collection of operators for reporting workload metrics and integrating product install lifecycle through https://marketplace.redhat.com.

The IBM Metrics Operator is used to meter workloads and report usage from Prometheus on an OpenShift cluster, and reports it to Red Hat Marketplace.

The IBM Data Reporter Operator is used as a push event interface into the IBM Metrics Operator's data-service, for reporting to Red Hat Marketplace.

The Red Hat Marketplace Deployment Operator by IBM is used for integrating cluster and operator subscription management on an OpenShift cluster via https://marketplace.redhat.com.

The Red Hat Marketplace Operator metering and deployment functionalities have been separated into two operators.

• The metering functionality is included in the IBM Metrics Operator
  • Admin level functionality and permissions are removed from the IBM Metrics Operator
  • ClusterServiceVersion/ibm-metrics-operator
• The deployment functionality remains as part of the Red Hat Marketplace Deployment Operator
  • The Red Hat Marketplace Deployment Operator prerequisites the IBM Metrics Operator
  • Some admin level RBAC permissions are required for deployment functionality
  • ClusterServiceVersion/redhat-marketplace-operator

Full registration and visibility of usage metrics on https://marketplace.redhat.com requires both IBM Metrics Operator and Red Hat Marketplace Deployment Operator.

The operator releases adhere to semantic versioning and provides a seamless upgrade path for minor and patch releases within the current stable channel.

• User with Cluster Admin role
• OpenShift Container Platform, major version 4 with any available supported minor version
• It is required to enable monitoring for user-defined projects as the Prometheus provider.
  • A minimum retention time of 168h and minimum storage capacity of 40Gi per volume.

Minimum system resources required:

Multiple nodes are required to provide pod scheduling for high availability for Red Hat Marketplace Data Service and Prometheus.

The IBM Metrics Operator automatically creates 3 x 1Gi PersistentVolumeClaims to store reports as part of the data service, with ReadWriteOnce access mode. Te PersistentVolumeClaims are automatically created by the ibm-metrics-operator after creating a redhat-marketplace-pull-secret and accepting the license in marketplaceconfig.

• OpenShift Container Storage / OpenShift Data Foundation version 4.x, from version 4.2 or higher
• IBM Cloud Block storage and IBM Cloud File storage
• IBM Storage Suite for IBM Cloud Paks:
  • File storage from IBM Spectrum Fusion/Scale
  • Block storage from IBM Spectrum Virtualize, FlashSystem or DS8K
• Portworx Storage, version 2.5.5 or above
• Amazon Elastic File Storage

• ReadWriteOnce (RWO)

Choose one of the following options to provision storage for the ibm-metrics-operator data-service

• A StorageClass is defined with a metadata.annotations: storageclass.kubernetes.io/is-default-class: "true"
• PersistentVolumes will be provisioned automatically for the generated PersistentVolumeClaims

• Must be performed before creating a redhat-marketplace-pull-secret or accepting the license in marketplaceconfig. Otherwise, the automatically generated PersistentVolumeClaims are immutable.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: rhm-data-service
  name: rhm-data-service-rhm-data-service-0
  namespace: redhat-marketplace
spec:
  storageClassName: rook-cephfs
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

• May be performed before or after creating a redhat-marketplace-pull-secret or accepting the license in marketplaceconfig.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: rhm-data-service-rhm-data-service-0
spec:
  csi:
    driver: rook-ceph.cephfs.csi.ceph.com
    volumeHandle: rhm-data-service-rhm-data-service-0
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: rook-cephfs
  volumeMode: Filesystem
  claimRef:
    kind: PersistentVolumeClaim
    namespace: redhat-marketplace
    name: rhm-data-service-rhm-data-service-0

For installation and configuration see the Red Hat Marketplace documentation.

The Operators and their components support running under the OpenShift Container Platform default restricted and restricted-v2 security context constraints.

The IBM Metrics Operator components require specific ClusterRoleBindings.

• The metric-state component requires a ClusterRoleBinding for the the view ClusterRole.
• The reporter component requires a ClusterRoleBinding for the the cluster-monitoring-view ClusterRole.

Due to limitations of Operator Lifecycle Manager (OLM), this ClusterRoleBinding can not be provided automatically for arbitrary installation target namespaces.

A ClusterRoleBinding is included for installation to the default namespace of redhat-marketplace, and namespaces openshift-redhat-marketplace, ibm-common-services.

To update the ClusterRoleBindings for installation to an alternate namespace

oc patch clusterrolebinding ibm-metrics-operator-metric-state-view-binding --type='json' -p='[{"op": "add", "path": "/subjects/1", "value": {"kind": "ServiceAccount", "name": "ibm-metrics-operator-metric-state","namespace": "NAMESPACE" }}]'

oc patch clusterrolebinding ibm-metrics-operator-reporter-cluster-monitoring-binding --type='json' -p='[{"op": "add", "path": "/subjects/1", "value": {"kind": "ServiceAccount", "name": "ibm-metrics-operator-reporter","namespace": "NAMESPACE" }}]'

The metric-state Deployment obtains get/list/watch access to metered resources via the view ClusterRole. For operators deployed using Operator Lifecycle Manager (OLM), permissions are added to clusterrole/view dynamically via a generated and annotated -view ClusterRole. If you wish to meter an operator, and its Custom Resource Definitions (CRDs) are not deployed through OLM, one of two options are required

1. Add the following label to a clusterrole that has get/list/watch access to your CRD: rbac.authorization.k8s.io/aggregate-to-view: "true", thereby dynamically adding it to clusterrole/view
2. Create a ClusterRole that has get/list/watch access to your CRD, and create a ClusterRoleBinding for the metric-state ServiceAccount

Attempting to meter a resource with a MeterDefinition without the required permissions will log an AccessDeniedError in metric-state.

To plan for disaster recovery, note the PhysicalVolumeClaims rhm-data-service-rhm-data-service-N.

• In connected environments, MeterReport data upload attempts occur hourly, and are then removed from data-service. There is a low risk of losing much unreported data.
• In an airgap environment, MeterReport data must be pulled from data-service and uploaded manually using datactl. To prevent data loss in a disaster scenario, the data-service volumes should be considered in a recovery plan.

It is possible to configure how OLM deploys an Operator via the config field in the Subscription object.

The IBM Metrics Operator and Red Hat Marketplace Deployment Operator will also read the config and append it to the operands. The primary use case is to control scheduling using Tolerations and NodeSelectors.

A limitation is that the config elements are only appended to the operands. The elements in the operands are not removed if the config is removed from the Subscripion. The operand must be modified manually, or deleted and recreated by the controller.

This is a high level visual representation of how the operators interact with Prometheus, CustomResources, with each other, and with Red Hat Marketplace.

Red Hat Marketplace

Wiki