Configuration¶

This page covers the manager's own configuration, the credential model, the add-on selector labels, and the cluster-wide policy knobs. The field-by-field CRD reference is auto-generated — see CRD API; every template variable is in Templates & variables.

Input formats: YAML and JSON¶

Every CAPVCO manifest can be written as either YAML or JSON. The Kubernetes API server is JSON-native and YAML 1.2 is a superset of JSON, so both go through the same decode, the same defaulting and the same validation — there is no second code path and no format that skips a check.

That is tested rather than assumed: every object in every shipped template is decoded from both forms and compared, on every pipeline. It was also confirmed against a live API server — a VCOCluster applied as JSON and as YAML stores an identical spec, and the same bad value is rejected with a byte-identical message either way.

YAMLJSON

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: VCOCluster
metadata:
  name: demo
  namespace: default
spec:
  location: qt-gen-dc01-003
  credentialSecretRef: demo-vco-credential
  cloudspace:
    privateNetwork: 192.168.150.0/24

{
  "apiVersion": "infrastructure.cluster.x-k8s.io/v1alpha1",
  "kind": "VCOCluster",
  "metadata": {
    "name": "demo",
    "namespace": "default"
  },
  "spec": {
    "location": "qt-gen-dc01-003",
    "credentialSecretRef": "demo-vco-credential",
    "cloudspace": {
      "privateNetwork": "192.168.150.0/24"
    }
  }
}

kubectl apply -f manifest.json works exactly as -f manifest.yaml does.

Multi-object files and clusterctl templates¶

JSON has no --- document separator, so a file holding several objects needs a kind: List wrapper. A bare top-level JSON array does not work — clusterctl rejects it with cannot unmarshal array into Go value of type map[string]interface {}.

With the wrapper, clusterctl generate cluster --from template.json does work: variables are substituted normally, and the generated output applies with kubectl.

{
  "apiVersion": "v1",
  "kind": "List",
  "items": [
    { "apiVersion": "cluster.x-k8s.io/v1beta2", "kind": "Cluster", "metadata": { "name": "${CLUSTER_NAME}" } },
    { "apiVersion": "infrastructure.cluster.x-k8s.io/v1alpha1", "kind": "VCOCluster", "metadata": { "name": "${CLUSTER_NAME}" } }
  ]
}

The templates CAPVCO ships are multi-document YAML, and stay that way — clusterctl generate cluster --flavor <x> reads them from the release. Converting one is a local choice, not something you need from us.

Schema validation without a cluster¶

Each release ships standalone JSON Schemas for the CRDs as crd-schemas.tar.gz, extracted from the CRDs' own openAPIV3Schema. This is the piece that makes generated configuration checkable, and it helps YAML and JSON authors equally.

curl -sSLO <release-url>/crd-schemas.tar.gz && tar -xzf crd-schemas.tar.gz   # → schemas/

# validate a manifest offline, in either format
kubeconform -schema-location default \
  -schema-location 'schemas/{{ .ResourceKind }}_{{ .ResourceAPIVersion }}.json' \
  demo.yaml

For editor completion and inline errors, point the YAML language server at the same file:

# yaml-language-server: $schema=./schemas/vcocluster_v1alpha1.json

Generate them from a checkout with make json-schemas. apiVersion and kind are pinned to a single value in each schema, so one cannot be used to validate the wrong object.

XML is not supported, and will not be

Kubernetes has no XML representation at any layer — not the API server, not kubectl, not the CRD machinery. Supporting it would mean maintaining a translation layer in front of the API for a format no Kubernetes tooling emits or consumes.

Manager instance config¶

The manager reads two instance-level settings from its environment, sourced from the capvco-credentials Secret in its namespace. They are deliberately independent — do not derive one from the other (they differ on white-label deployments, and there's no fixed console./iam. prefix):

Env var	Secret key	Meaning	Example
`VCO_PORTAL_URL`	`portalUrl`	whitesky.cloud portal base URL (without `/api/1`)	`https://<your-whitesky.cloud-portal>`
`VCO_IAM_REFRESH_URL`	`iamRefreshUrl`	IAM JWT-refresh endpoint	`https://<your-iam-host>/v1/oauth/jwt/refresh`

Set these in the capvco-credentials Secret at install time. Everything else is per-cluster. The whitesky.cloud HTTP client uses per-request deadlines (300 s default; up to 60 min for a VM-image import, which the platform serves synchronously).

Nothing brand-specific is compiled in — the portal URL, IAM URL, customer id and credentials are all configuration, so one build serves any whitesky.cloud portal. You run one manager instance per portal.

Credential model¶

CAPVCO mints no credentials. Every VCOCluster references a Secret (spec.credentialSecretRef, in the cluster's namespace) holding a client-supplied whitesky.cloud credential (the Cluster API cluster-identity pattern). The controller uses it for all backend operations for that cluster and injects it into the workload cluster for the add-ons — the manager holds no standing credential.

Expected Secret keys:

Key	Required	Meaning
`jwt`	yes	A self-refreshing whitesky.cloud JWT (carries a `refresh_token` claim).
`customer_id`	yes	The whitesky.cloud customer id.
`portal_url`	no	Per-cluster override of `VCO_PORTAL_URL`.
`iam_refresh_url`	no	Per-cluster override of `VCO_IAM_REFRESH_URL`.

Rotating the Secret is picked up automatically (the client cache invalidates on the Secret's resourceVersion), and the credentials injected into workload clusters are re-pushed on change. The JWT refresh preserves scopes and cannot elevate them, so a narrowly-scoped credential stays narrow.

Scope per cluster

Because one credential drives everything for a cluster (and is readable by that cluster's tenant admin), scope it to just what one cluster needs — see the RBAC caveat.

Operator-privileged operations¶

A few whitesky.cloud APIs are operator-privileged and return 403 for a per-cluster customer credential. They take a separate operator credential (same Secret shape):

Operation	Field
VM-image publishing	`spec.geo.imagePublishCredentialSecretRef`
Registry TLS certificate (air-gapped)	`spec.registry.certCredentialSecretRef`

DNS operations (geo endpoints, ExternalDNS, registry domains) work with the customer credential.

Injected workload Secrets¶

The controller writes these into the workload cluster as add-ons need them (a ClusterResourceSet can't template per-cluster Secret data, so the controller injects them directly):

Secret (namespace)	Keys	For
`csi-driver-secret` (`csi-driver`)	`client_jwt`, `customer_id`	The CSI driver (→ `CSI_DRIVER_JWT`, `CUSTOMER_ID`).
`vco-lb-credential` (`capvco-system`)	`VCO_PORTAL_URL`, `VCO_IAM_REFRESH_URL`, `VCO_CUSTOMER_ID`, `VCO_JWT`	Service-LB / CCM / ExternalDNS webhook.

An idempotent upsert of these Secrets is the rotation mechanism — when the source credential changes, the injected copies are re-pushed.

Add-on selector labels¶

Features are delivered as ClusterResourceSet add-ons selected by labels on the core CAPI Cluster object (not the VCOCluster). All use the prefix capvco.infrastructure.cluster.x-k8s.io/.

Label	Value	Add-on
`…/cni`	`calico`	CNI (Calico; `veth_mtu` pinned to 1350 so any cluster is geo/VPN-safe)
`…/csi`	`vco`	whitesky.cloud CSI driver + StorageClasses
`…/csi-encryption`	`openbao`	Key-backend config for encrypted volumes
`…/ccm`	`enabled`	Cloud-controller-manager (node init + Service-LB)
`…/ccm`	`internal`	The same, with Service LB frontends defaulting to the cloudspace private gateway (private clusters)
`…/service-lb`	`vco`	Standalone Service-LoadBalancer controller
`…/service-lb`	`vco-internal`	Same controller, LBs internal by default (private clusters) — bundle `service-lb-internal`
`…/external-dns`	`enabled`	ExternalDNS + the whitesky.cloud DNS webhook
`…/apiserver-proxy`	`enabled`	Per-node internal-API proxy (geo)

ccm and service-lb are mutually exclusive

Both run the Service-LB controller. Enable one, never both — see Load balancing.

The stock templates set the common labels for you; you only add labels to turn on optional add-ons.

Cluster-wide policy on `VCOCluster.spec`¶

Besides the per-feature subtrees (geo, backup, storage, registry — each covered in its guide), a few cluster-wide knobs:

Field	Default	Meaning
`controlPlaneExposure`	`loadbalancer`	How `:6443` is published: `loadbalancer` (HA), `portforward` (single-CP), or `private` — the private-network gateway, with nothing on the external IP. See Private clusters.
`cloudspace.private`	`false`	Create the cloudspace with no external network. Accepted only with `controlPlaneExposure: private`.
`cloudspace.privateNetwork`	(platform picks)	The cloudspace private CIDR. Load-bearing for a private cluster: its first host is the API endpoint. Must not overlap the pod/service CIDRs or any network you connect it to.
`managementAccess.connectedCloudspace.cloudspaceId`	(empty)	The cloudspace the management cluster runs in. CAPVCO connects every cloudspace it creates for this cluster to it over the site-to-site VPN, both directions, and removes the connection on delete. Empty = build nothing (correct when the management cluster is already in this cluster's cloudspace). Refused together with `cloudspace.private`, which leaves no external address to pair.
`storage.credentialScope`	`location`	How the CSI credential is scoped: `location` (the working default), `perDisk`, or `cloudspace`. See the RBAC caveat.
`driftPolicy`	(disabled)	Drift detection + orphan GC — see below.

Drift policy¶

spec.driftPolicy periodically compares actual whitesky.cloud resources against desired and acts by mode:

`mode`	Behaviour
`Report` (default)	Detect + surface drift/orphans (condition, metric, events). Never changes the backend.
`Reconcile`	Additionally re-create/repair drifted resources still needed (detection + reporting today; repair incremental).
`GC`	Additionally delete orphans (resources whose owner is gone), subject to `resources` opt-in and `dryRun`.

spec:
  driftPolicy:
    mode: GC
    dryRun: true              # log what GC would do, touch nothing
    resources: ["vms"]        # empty = GC nothing; destructive action is always narrow + explicit

dryRun: true and an empty resources list mean even GC mode does nothing destructive until you opt in per resource type — a deliberate safety design.