An open API service indexing awesome lists of open source software.

https://github.com/fluxcd/flux-schema

Flux CLI plugin for Kubernetes schema extraction and manifests validation
https://github.com/fluxcd/flux-schema

fluxcd gitops openapi-validation plugin

Last synced: 24 days ago
JSON representation

Flux CLI plugin for Kubernetes schema extraction and manifests validation

Awesome Lists containing this project

README

          

# flux-schema

[![release](https://img.shields.io/github/release/fluxcd/flux-schema/all.svg)](https://github.com/fluxcd/flux-schema/releases)
[![test](https://github.com/fluxcd/flux-schema/actions/workflows/test.yaml/badge.svg)](https://github.com/fluxcd/flux-schema/actions/workflows/test.yaml)
[![cve-scan](https://github.com/fluxcd/flux-schema/workflows/cve-scan/badge.svg)](https://github.com/fluxcd/flux-schema/actions/workflows/cve-scan.yml)
[![license](https://img.shields.io/github/license/fluxcd/flux-schema.svg)](https://github.com/fluxcd/flux-schema/blob/main/LICENSE)
[![slsa](https://slsa.dev/images/gh-badge-level2.svg)](https://github.com/fluxcd/flux-schema/attestations)

Flux CLI plugin for Kubernetes schema extraction and manifests validation.

> [!NOTE]
> This repository is in early development and the plugin system is not yet available in a stable release of Flux.
> The instructions for installing and using the plugin will be added here when [RFC-0013](https://github.com/fluxcd/flux2/blob/main/rfcs/0013-cli-plugin-system/README.md)
> has been implemented and released in Flux 2.9 or later.

## Available Commands

- `flux-schema validate [paths...]`: Validate Kubernetes manifests against JSON Schemas
- `--schema-location`: URL or file path for schemas (repeatable, tried in order); `default` points at the built-in catalog
- `--skip-missing-schemas`: Skip documents for which no schema can be found
- `--skip-kind`: Skip documents matching `Kind` or `apiVersion/Kind` (repeatable)
- `--fail-fast`: Exit after the first invalid document
- `--concurrent`: Number of concurrent workers (default 8)
- `--insecure-skip-tls-verify`: Disable TLS certificate verification when fetching schemas over HTTPS
- `-v, --verbose`: Print a line for every document, including valid and skipped ones
- `--config`: Path to a YAML file supplying default values for validate flags (env: `FLUX_SCHEMA_CONFIG`)
- `flux-schema extract crd [files...]`: Extract JSON Schema from Kubernetes CRD YAMLs
- `-d, --output-dir`: Directory to write JSON Schema files to (mutually exclusive with `--output-archive`)
- `-a, --output-archive`: Path to write a gzipped tar archive of JSON Schema files to
- `-f, --output-format`: Go template for output file paths (default: `{{ .Group }}/{{ .Kind }}_{{ .Version }}.json`)
- `--strip-description`: Drop `description` fields from the generated schemas to reduce their size
- `flux-schema extract k8s [swagger-file]`: Extract JSON Schema from a Kubernetes OpenAPI v2 swagger document
- `--version X.Y.Z`: Fetch the swagger from `kubernetes/kubernetes` for the given release tag (mutually exclusive with a swagger file)
- `-d, --output-dir`, `-f, --output-format`, `--strip-description`: same as `extract crd`

### Kubernetes Manifests Validation

The validate command reads Kubernetes YAML manifests from one or more files or directories
and validates each document against a JSON Schema resolved from its `apiVersion` and `kind`.

When no `--schema-location` is given, validate uses the [flux-schema catalog](catalog/README.md),
which covers the latest Kubernetes APIs, the stable channel of Gateway API,
and the Flux ecosystem CRDs:

```shell
flux-schema validate ./manifests
```

To validate against your own schemas generated by `flux-schema extract`, pass the
directory as `--schema-location`. Bare paths and URLs are auto-expanded to the
catalog layout `{{.Group}}/{{.Kind}}_{{.Version}}.json`:

```shell
flux-schema validate ./manifests --schema-location ./my-schemas
```

For a different layout, pass a full Go template ending in `.json`:

```shell
flux-schema validate ./manifests \
--skip-missing-schemas \
--schema-location './schemas/{{.Kind}}-{{.GroupPrefix}}-{{.Version}}.json'
```

Template variables are `.Group`, `.GroupPrefix`, `.Kind`, and `.Version`.

The `--schema-location` flag is repeatable and locations are tried in order (the first match wins).
Pass the literal value `default` to include the flux-schema catalog alongside your own schemas:

```shell
flux-schema validate ./manifests \
--schema-location default \
--schema-location https://raw.githubusercontent.com/datreeio/CRDs-catalog/main \
--schema-location './schemas/{{.Kind}}-{{.GroupPrefix}}-{{.Version}}.json'
```

Manifests can also be piped in and certain documents skipped with `--skip-kind`:

```shell
kustomize build . | flux-schema validate \
--skip-kind 'v1/Secret' \
--skip-kind 'source.toolkit.fluxcd.io/v1/ExternalArtifact'
```

Output example with validation errors:

```
manifests/sources.yaml - Bucket/apps/s3-data is invalid: schema validation failed
- /spec: missing property 'bucketName'
- /spec/interval: got number, want string
- /spec/secretRef/name: got object, want string
- /spec: additional properties 'force' not allowed
manifests/sources.yaml - OCIRepository/apps/podinfo is invalid: YAML parse failed
- line 18: key "app.kubernetes.io/name" already set in map
manifests/sources.yaml - HelmChart/apps/redis is valid
manifests/sources.yaml - Secret/apps/auth-sops is skipped: kind skipped
Summary: 4 resources found in 1 file - Valid: 1, Invalid: 2, Skipped: 1
```

A non-zero exit code is returned when any document is invalid or errored.

#### Validation rules

- YAML documents with duplicate keys are rejected matching Flux behavior.
- Documents missing both `metadata.name` and `metadata.generateName` are flagged as invalid
matching Kubernetes API behavior.
- Schemas produced by `flux-schema extract crd` close objects with `additionalProperties: false`,
so undocumented fields under `spec` fail validation.
- String formats `duration`, `date`, `datetime`/`date-time`, and `time` are validated
matching Kubernetes API conventions.

#### Config file

Flag values can be pre-set in a YAML config file and referenced with `--config`
or with the `FLUX_SCHEMA_CONFIG` environment variable.
This keeps long invocations out of CI scripts and makes validation reproducible
across developers:

```shell
flux-schema validate ./manifests --config .flux-schema.yaml
```

The file has a `version` (required, must be `"1"`) and a `validate` section
whose keys mirror the CLI flag names:

```yaml
version: "1"
validate:
schema-location:
- default
- https://raw.githubusercontent.com/datreeio/CRDs-catalog/main
skip-kind:
- v1/Secret
- source.toolkit.fluxcd.io/v1/ExternalArtifact
skip-missing-schemas: false
verbose: true
fail-fast: false
concurrent: 8
insecure-skip-tls-verify: false
```

Rules:

- CLI flags override config values. Setting `--verbose=false` wins over `verbose: true` in the file.
- Setting `--config` overrides `FLUX_SCHEMA_CONFIG`. When both are set, the flag wins and the env var is ignored.
- Manifest paths stay positional. The config file configures how to validate; paths are given on the command line.

### Kubernetes CRD Extraction

The `extract crd` command reads Kubernetes CustomResourceDefinition YAML
and writes one JSON Schema file per CRD version.
The input can be a bare CRD, a `List` of CRDs, or a multi-document YAML stream.

Generate schemas for every CRD installed in a cluster, using the
per-group-directory layout:

```shell
kubectl get crds -o yaml | flux-schema extract crd -d ./schemas
```

You can supply `-f, --output-format` with a Go template to change the layout, e.g. the
`kubeconform`/`kubeval` flat layout:

```shell
flux-schema extract crd crds.yaml -f '{{ .Kind }}-{{ .GroupPrefix }}-{{ .Version }}.json'
```

> The output is compatible with `kubeconform` and `kubeval`, making this command a drop-in replacement for kubeconform's
> [openapi2jsonschema.py](https://github.com/yannh/kubeconform/blob/master/scripts/openapi2jsonschema.py) script.

To bundle the schemas into a gzipped tar archive instead of writing to a directory,
use `-a, --output-archive`:

```shell
kustomize build config/crd | flux-schema extract crd -a dist/crd-schemas.tar.gz
```

The archive path must end in `.tar.gz` or `.tgz`, and its parent directory is created if missing.
`-f, --output-format` still controls the entry paths inside the archive, so a nested template produces
subdirectories within the tarball.

Supported template variables (all lowercased at render time):

| Variable | Example |
|----------------|----------------------------|
| `.Group` | `source.toolkit.fluxcd.io` |
| `.GroupPrefix` | `source` |
| `.Kind` | `gitrepository` |
| `.Version` | `v1` |

An empty `.Group` (Kubernetes core API, e.g. `apiVersion: v1`) is normalized to `core`, and `.GroupPrefix`
is derived from `.Group` when unset. So a core `Pod` renders as `core/pod_v1.json` with the default template
and resolves to the same path when `validate` looks up its schema.

Note that the generated schemas apply the following OpenAPI → JSON Schema transformations:

- Objects with `properties` are closed with `additionalProperties: false`, except under nodes
marked with `x-kubernetes-preserve-unknown-fields: true`, which stay open so free-form maps validate correctly.
- Integer-or-string fields are rewritten to `oneOf: [{type: string}, {type: integer}]`. Both the
legacy `format: int-or-string` and the structural `x-kubernetes-int-or-string: true` forms are recognized.

### Kubernetes OpenAPI Extraction

The `extract k8s` command reads a Kubernetes OpenAPI v2 swagger document and writes
one JSON Schema file per kind listed under `x-kubernetes-group-version-kind`. Helper
types (e.g. `PodSpec`, `ObjectMeta`) are not emitted as standalone files — they are
inlined into the kinds that reference them.

Fetch the upstream swagger for a Kubernetes release and generate schemas:

```shell
flux-schema extract k8s --version 1.35.0 -d ./schemas
```

Supply the swagger file from the cluster:

```shell
kubectl get --raw /openapi/v2 | flux-schema extract k8s -d ./schemas
```

The same `-f, --output-format` template used by `extract crd` works here, so the
generated files remain resolvable by `validate` with a single `--schema-location`
for both built-ins and CRDs. Core API kinds (`apiVersion: v1`) render under the
`core/` group directory.

The emitted schemas are the standalone-strict variant:

- Every `$ref` is inlined so schemas have no cross-file dependencies.
- Objects with `properties` are closed with `additionalProperties: false`, except
under nodes marked `x-kubernetes-preserve-unknown-fields: true`.
- Integer-or-string fields are rewritten to `oneOf: [{type: string}, {type: integer}]`.
- Optional fields are marked nullable (`type: [, "null"]`), matching the
Kubernetes API server's behavior of accepting `null` for unset optional values.
- `apiVersion` and `kind` are injected into every kind's properties and required list.