Shipping an operator that includes Webhooks

The WebhookDefinition ClusterServiceVersion Object

Operators may define validating, mutating, and conversion webhooks within its ClusterServiceVersion resource (CSV)

These webhooks are defined with the CSV’s WebhookDefinition array. The WebhookDescription object contains a union of the fields defined in the ValidatingWebhookConfiguration and MutatingWebhookConfiguration type with the exception of the NamespaceSelector, which is generated by OLM to match namespaces scoped by the OperatorGroup that the operator is deployed in.

In addition to these fields, OLM requires that you define:

The Type of the Webhook, which must be set to ValidatingAdmissionWebhook, MutatingAdmissionWebhook, or ConversionWebhook
The DeploymentName that OLM must mount the CA Cert information into, this should match the name of a deployment defined in the CSV

Creating an Operator with a Webhook

This document will not cover the steps required to create an operator that includes an admission or conversion webhook. If you are interested in creating such an operator, please refer to the following documentation provided by the Operator-SDK and Kubebuilder projects:

Deploying an operator with webhooks using OLM

OLM has a few constraints that must be considered when developing an operator featuring validating, mutating, or conversion webhooks.

Certificate Authority Requirements

OLM will create and mount a self-signed Certificate Authority (CA) to each deployment that contains a webhook as defined in the CSV. The certificates generated by OLM will expire after 2 years, at which point OLM will generate new certificates and redeploy the operator. The logic that generates and mounts the CA into the deployment was originally developed for the API Service lifecycle logic and has been expanded for use with webhooks.

OLM will mount the certificates at the default location that operators built with Kubebuilder and the Operator-SDK expect the certificates, specifically

The TLS Cert file will be mounted to the deployment at /tmp/k8s-webhook-server/serving-certs/tls.cert
The TLS Key file will be mounted to the deployment at /tmp/k8s-webhook-server/serving-certs/tls.key

Note: When OLM was released, the CAs meant for Webhooks were mounted to the same location as defined by the existing API Service Lifecycle code. In an effort to remain backwards compatible, the certificates are also mounted to the following locations:

The TLS Cert file will also be mounted to the deployment at /apiserver.local.config/certificates/apiserver.crt
The TLS Key file will also be mounted to the deployment at /apiserver.local.config/certificates/apiserver.key

OLM does not allow users to specify a mount location or name for the certificates. This issue should be addressed once OLM makes the decision to offload the management of CAs to projects such as Cert-Manager and the Service-CA Operator, this work can be tracked here.

Admission Webhook Rule Requirements

When developing an admission webhook please be aware that in an attempt to prevent operator from configuring the cluster into an unrecoverable state, OLM will place the CSV in the failed phase if the Rules defined in an admission webhook:

Intercept requests that target all groups
Intercept requests that target the operators.coreos.com group
Intercept requests that target the ValidatingWebhookConfigurations or MutatingWebhookConfigurations resources

Conversion Webhook Rules Requirements

CSVs featuring a conversion webhook may only support the AllNamespaces installMode
The CRD targeted by the Conversion webhook must have its spec.preserveUnknownFields field set to false or nil
The Conversion Webhook defined in the CSV must target an owned CRD

Example: Installing a operator featuring webhooks with OLM

This example will focus on installing an operator that the OLM team built for its e2e test portfolio, the webhook-operator. The webhook-operator implements a validating, mutating, and conversion webhook for the WebhookTest CRD.

Defining Conversion Webhooks in the ClusterServiceVersion

The full webhook operator CSV can be seen here, but we will highlight the important parts of the CSV below:

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  name: webhook-operator.v0.0.1
spec:
  customresourcedefinitions:
    owned:
    - kind: WebhookTest
      name: webhooktests.webhook.operators.coreos.io # The CRDs target by the conversion webhook must exist here
      version: v1
  install:
    spec:
      deployments:
      - name: webhook-operator-webhook
        ...
        ...
        ...
    strategy: deployment
  installModes:
  - supported: false
    type: OwnNamespace
  - supported: false
    type: SingleNamespace
  - supported: false
    type: MultiNamespace
  - supported: true
    type: AllNamespaces
  webhookdefinitions:
  - type: ValidatingAdmissionWebhook
    admissionReviewVersions:
    - v1beta1
    - v1
    containerPort: 443
    targetPort: 4343
    deploymentName: webhook-operator-webhook
    failurePolicy: Fail
    generateName: vwebhooktest.kb.io
    rules:
    - apiGroups:
      - webhook.operators.coreos.io
      apiVersions:
      - v1
      operations:
      - CREATE
      - UPDATE
      resources:
      - webhooktests
    sideEffects: None
    webhookPath: /validate-webhook-operators-coreos-io-v1-webhooktest
  - type: MutatingAdmissionWebhook
    admissionReviewVersions:
    - v1beta1
    - v1
    containerPort: 443
    targetPort: 4343
    deploymentName: webhook-operator-webhook
    failurePolicy: Fail
    generateName: mwebhooktest.kb.io
    rules:
    - apiGroups:
      - webhook.operators.coreos.io
      apiVersions:
      - v1
      operations:
      - CREATE
      - UPDATE
      resources:
      - webhooktests
    sideEffects: None
    webhookPath: /mutate-webhook-operators-coreos-io-v1-webhooktest
  - type: ConversionWebhook
    admissionReviewVersions:
    - v1beta1
    - v1
    containerPort: 443
    targetPort: 4343
    deploymentName: webhook-operator-webhook
    generateName: cwebhooktest.kb.io
    sideEffects: None
    webhookPath: /convert
    conversionCRDs:
    - webhooktests.webhook.operators.coreos.io # Each CRD must have spec.PreserveUnknownFields property set to `false` or `nil`