go-yamlvalidator

About go-yamlvalidator

This large README is shown as a plain-text preview. Read the full README at the source

# yamlvalidator

<p align="center">
  <a href="https://github.com/Yakwilik/go-yamlvalidator/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/Yakwilik/go-yamlvalidator/actions/workflows/ci.yml/badge.svg"></a>
  <a href="https://github.com/Yakwilik/go-yamlvalidator/tree/coverage"><img alt="Measured statement coverage" src="https://raw.githubusercontent.com/Yakwilik/go-yamlvalidator/coverage/coverage.svg"></a>
  <a href="https://pkg.go.dev/github.com/Yakwilik/go-yamlvalidator"><img alt="Go Reference" src="https://pkg.go.dev/badge/github.com/Yakwilik/go-yamlvalidator.svg"></a>
  <a href="https://github.com/Yakwilik/go-yamlvalidator/releases/latest"><img alt="Release" src="https://img.shields.io/github/v/release/Yakwilik/go-yamlvalidator?sort=semver"></a>
  <a href="https://github.com/Yakwilik/go-yamlvalidator/blob/master/LICENSE"><img alt="License" src="https://img.shields.io/github/license/Yakwilik/go-yamlvalidator"></a>
  <a href="https://github.com/Yakwilik/go-yamlvalidator/blob/master/go.mod"><img alt="Go version" src="https://img.shields.io/github/go-mod/go-version/Yakwilik/go-yamlvalidator?logo=go&logoColor=white"></a>
  <a href="https://github.com/avelino/awesome-go"><img alt="Mentioned in Awesome Go" src="https://awesome.re/mentioned-badge-flat.svg"></a>
</p>

**Source-aware YAML validation and typed YAML codecs for Go.** Validate Go struct tags, native Go schemas, or full JSON Schema while preserving paths, lines, and columns in diagnostics.

```text
[ERROR] line 3:1: unknown key "plugin"; did you mean "plugins"? (path: generate.plugin)
>    3 | plugin:
       | ^
```

For typed application configuration, put rules next to Go fields with `yamlvalidate` and use `yamlvalidator.Unmarshal` / `yamlvalidator.Marshal`. Use `FieldSchema` when you need a programmatic native contract, or bring an existing JSON Schema when portability matters. Validation runs against the YAML representation before application decoding.

### Why this library?

The comparison below covers the libraries' documented APIs, not every integration that could be built on top of them. The JSON Schema column refers to the Go library, not its separate CLI.

| Need | [yaml.v3](https://github.com/go-yaml/yaml/tree/v3.0.1) | [goccy/go-yaml](https://github.com/goccy/go-yaml) | [go-playground/validator](https://github.com/go-playground/validator) | [santhosh-tekuri/jsonschema](https://github.com/santhosh-tekuri/jsonschema) | [go-yamlvalidator](https://github.com/Yakwilik/go-yamlvalidator) |
|---|---|---|---|---|---|
| YAML parsing and Marshal/Unmarshal | Yes | Yes | No; validates Go values | No YAML codec in library API | Yes; uses yaml.v3 |
| Validation diagnostics with YAML source locations | Manual via <code>yaml.Node</code> | Validator integration and YAML Path annotations | No YAML source mapping | JSON instance paths; needs YAML source mapping | Paths, lines, columns and source snippets |
| Go struct-tag validation | Not built in | Via <code>StructValidator</code> on decoded values | Yes, on Go values | No | Native tags checked on YAML before decoding |
| JSON Schema validation | Not built in | Not built in | No | Yes | Yes, with YAML source diagnostics |
| Custom Go validation | Manual hooks | <code>StructValidator</code> integration | Field and struct callbacks | Formats and vocabularies | Value/key validators and registry |
| Unknown struct/object keys | <code>KnownFields(true)</code> | <code>DisallowUnknownField()</code> | Not recoverable once discarded by struct decoding | <code>additionalProperties</code> | Error/warn/ignore policies and key suggestions |

[goccy/go-yaml](https://github.com/goccy/go-yaml) already provides source-aware errors: its [validator integration](https://pkg.go.dev/github.com/goccy/go-yaml#Validator) can annotate field-validation failures, [YAML Path](https://pkg.go.dev/github.com/goccy/go-yaml#Path.AnnotateSource) can annotate custom errors, and [unknown-field rejection](https://pkg.go.dev/github.com/goccy/go-yaml#example-Decoder.Decode-DisallowUnknownField) includes source context. It should not be treated as a decoder without validation support. The distinction here is validation of the original YAML tree before decoding into an application struct, with native schemas, struct tags, JSON Schema and optional typed code generation in one library.

Originally built to validate configuration files in [EasyP](https://github.com/easyp-tech/easyp). The development story is in [this Habr article (Russian)](https://habr.com/ru/articles/1086386/).

## Installation

Requires Go 1.24 or newer. CI verifies the minimum supported Go 1.24 line and the current stable Go release. Security-fixed IDNA/Unicode code is kept in an internal third-party snapshot so full `idn-hostname`/`idn-email` support does not force consumers onto Go 1.25.

```bash
go get github.com/Yakwilik/[email protected]
```

## Quick Start

### Validated Go values with struct tags

For typed configuration, the shortest path is a normal Go struct with `yaml` and `yamlvalidate` tags:

~~~go
type Config struct {
    Mode  string   `yaml:"mode" yamlvalidate:"required,enum=[dev,prod]"`
    File  string   `yaml:"file,omitempty" yamlvalidate:"exactlyOneOf=[file,url],nonempty"`
    URL   string   `yaml:"url,omitempty"`
    Hosts []string `yaml:"hosts" yamlvalidate:"required,minItems=1,items={nonempty,format=hostname}"`
}

var config Config
if err := yamlvalidator.Unmarshal(data, &config); err != nil {
    return err
}

encoded, err := yamlvalidator.Marshal(config)
~~~

`Unmarshal` parses one YAML document, validates the source tree, and only then decodes into the destination. A validation failure does not mutate the destination. Unknown struct fields are errors by default. `Marshal` validates the YAML representation it actually emits, so `omitempty`, inline fields, and custom codecs affect validation exactly as they affect output.

Object rules have explicit scopes. A bare group such as `exactlyOneOf=[file,url]` on a real struct field constrains sibling keys in the containing mapping, even when the carrier field is absent. The `Keys` form, such as `exactlyOneOfKeys=[file,url]`, constrains the mapping value of the current field.

See [the high-level API guide](docs/high-level-api.md) for the complete tag language, registry bindings, diagnostics, limits, nested `items`/`values`, and recursive types. The [runnable example](examples/highlevel/main.go) defines a complete configuration.

### Optional typed code generation

Generate methods for existing Go types without changing their definitions:

~~~sh
go run github.com/Yakwilik/go-yamlvalidator/cmd/yamlvalidator-gen -all -output=zz_yamlvalidator_generated.go
~~~

Generated types support ordinary `yaml.v3` Marshal/Unmarshal through standard YAML hooks, as well as `yamlvalidator` Options. Use `-all` to generate every eligible named struct in the package without maintaining a type list; nested statically known types are followed transitively.

The generated file contains typed field/collection mapping and a normalized native schema graph. It does not inspect generated structs with reflection or parse `yamlvalidate` tags at runtime. Generated hook mechanics live in the dedicated `genruntime` package; the application-facing root package exposes no generator bridge functions. Reflection remains only in fallback paths for non-generated or genuinely dynamic values.

See [code generation](docs/code-generation.md), [generated runtime architecture](docs/generated-runtime-architecture.md), the [benchmark suite](benchmarks), the [runnable generated example](examples/codegen), and [cross-field rule examples in Russian](docs/tag-rules-examples.ru.md).

### Native schema validation

```go
package main

import (
    "fmt"

    v "github.com/Yakwilik/go-yamlvalidator"
)

func main() {
    schema := &v.FieldSchema{
        Type:             v.TypeMap,
        UnknownKeyPolicy: v.UnknownKeyError,
        AllowedKeys: map[string]*v.FieldSchema{
            "name": {Type: v.TypeString, Required: true},
            "port": {Type: v.TypeInt},
        },
    }

    data := []byte("name: api\nport: 8080\n")
    result := v.NewValidator(schema).ValidateBytes(data)

    if result.HasErrors() {
        fmt.Print(result.FormatAll(true))
    }
}
```

## Schema Definition

### Field Schema

```go
type FieldSchema struct {
    // Basic properties
    Type         NodeType   // Expected type (TypeString, TypeInt, etc.)
    AllowedTypes []NodeType // Accept any listed type; takes precedence over Type
    Required     bool       // Field must be present
    Nullable    bool        // Allow null values
    Deprecated  string      // Deprecation message (empty = not deprecated)
    Description string      // Human-readable description
    Default     interface{} // Suggested default; emits warning if omitted, does not mutate input

    // Map-specific
    AllowedKeys          map[string]*FieldSchema // Known keys
    AdditionalProperties *FieldSchema            // Schema for unknown keys
    UnknownKeyPolicy     UnknownKeyPolicy        // How to handle unknown keys
    KeyValidators        []KeyValidator          // Key name validators

    // Sequence-specific
    ItemSchema *FieldSchema // Schema for items
    MinItems   *int
    MaxItems   *int

    // Value validators
    Validators []ValueValidator

    // Inter-field logic
    AnyOf             [][]string        // At least one group must be present
    ExactlyOneOf      []string          // Exactly one field must be present
    MutuallyExclusive []string          // At most one field can be present
    Conditions        []ConditionalRule // Conditional validation
    OneOfRequired     [][]string        // Exactly one field group must be present
    ForbiddenTogether [][]string        // A complete field group is forbidden
    DependentRequired map[string][]string

    // Schema composition
    OneOfSchemas []*FieldSchema // Exactly one schema must match
    AnyOfSchemas []*FieldSchema // At least one schema must match
}
```

### Native schema compilation

For schemas assembled dynamically, prefer <code>CompileFieldSchema</code> over constructing a validator directly. It rejects invalid schema graphs before any YAML is validated: recursive graphs, incompatible map/sequence constraints, invalid min/max bounds, nil validators, bad regexp definitions, invalid inter-field references, and other definition errors.

~~~go
validator, err := v.CompileFieldSchema(schema)
if err != nil {
    return err
}
~~~

`CompileFieldSchema` also snapshots the `FieldSchema` graph (maps, slices, nested schemas, item limits, conditions, and ordinary JSON/YAML container defaults). Later mutations of the source schema therefore do not change the compiled validator. Opaque application-defined objects stored in `Default`, plus custom `ValueValidator` / `KeyValidator` instances, are copied by value and remain the caller's concurrency responsibility. A compiled validator can otherwise be reused concurrently.

`NewValidator` remains available for backwards compatibility when the schema is already trusted; unlike `CompileFieldSchema`, it retains the supplied schema pointer directly.

### Node Types

| Type | Description |
|------|-------------|
| `TypeAny` | Any type; bare maps/sequences are not recursively constrained |
| `TypeNull` | Null values only |
| `TypeString` | String values |
| `TypeInt` | Integer values |
| `TypeFloat` | Float values (also accepts int) |
| `TypeBool` | Boolean values |
| `TypeMap` | Mapping nodes |
| `TypeSequence` | Sequence/array nodes |

### Unknown Key Policy

| Policy | Behavior |
|--------|----------|
| `UnknownKeyInherit` | Uses `ctx.StrictKeys` (default) |
| `UnknownKeyError` | Unknown keys are errors |
| `UnknownKeyWarn` | Unknown keys are warnings |
| `UnknownKeyIgnore` | Unknown keys are ignored |

## JSON Schema Compilation

<code>CompileJSONSchema</code> compiles a standards-compliant JSON Schema and returns a <code>FieldSchema</code> adapter. Validation is delegated to a full JSON Schema engine; the YAML validator remains responsible for parsing YAML, preserving source positions, resolving YAML aliases/merge keys, converting the YAML value to the JSON data model, and mapping JSON Schema errors back to YAML diagnostics.

Supported dialects:

- JSON Schema draft-04
- JSON Schema draft-06
- JSON Schema draft-07
- JSON Schema 2019-09
- JSON Schema 2020-12

An explicit <code>$schema</code> selects the dialect. Without <code>$schema</code>, draft 2020-12 is used by default and can be changed with <code>DefaultDraft</code>.

All keywords implemented by the engine are available, including references and modern applicators such as <code>$ref</code>, <code>$dynamicRef</code>, <code>$defs</code>, <code>$anchor</code>, <code>allOf</code>, <code>anyOf</code>, <code>oneOf</code>, <code>if/then/else</code>, <code>dependentSchemas</code>, <code>unevaluatedProperties</code>, <code>unevaluatedItems</code>, <code>prefixItems</code>, <code>contains</code>, numeric/string/object/array constraints, boolean schemas, annotations, and vocabulary declarations. Unknown extension keywords follow JSON Schema semantics instead of being rejected merely because yamlvalidator does not know them.

~~~go
schemaJSON, err := os.ReadFile("config.schema.json")
if err != nil {
    return err
}

schema, err := v.CompileJSONSchema(schemaJSON)
if err != nil {
    return err
}

result := v.NewValidator(schema).ValidateBytes(yamlData)
~~~

### Compilation options

~~~go
policy := v.UnknownKeyWarn
schema, err := v.CompileJSONSchemaWithOptions(schemaJSON, v.JSONSchemaCompileOptions{
    SchemaURL:    "https://schemas.example.com/config.json",
    DefaultDraft: v.JSONSchemaDraft2020,

    AssertFormat:  true,
    AssertContent: true,
    AssertVocabs:  true,
    RegexpTimeout: 500 * time.Millisecond, // Optional protection for untrusted schemas

    Resources: map[string][]byte{
        "https://schemas.example.com/common.json": commonSchema,
    },

    Resolver: v.NewJSONSchemaCachingResolver(
        v.JSONSchemaResolverFunc(func(ctx context.Context, url string) ([]byte, error) {
            return loadSchemaResource(ctx, url)
        }),
    ),

    AdditionalPropertiesFalsePolicy: &policy,
})
~~~

<code>Resources</code> preloads a closed schema graph without I/O. Compilation is closed-world by default: unresolved external references do not read the filesystem or network. Set <code>Resolver</code> explicitly to resolve them. Built-in resolver helpers include <code>JSONSchemaResourceMap</code>, <code>JSONSchemaResolverChain</code>, <code>JSONSchemaCachingResolver</code>, and the root-constrained <code>JSONSchemaFileResolver</code>.

For cancellable resolution/compilation use <code>CompileJSONSchemaContext</code> or <code>CompileJSONSchemaContextWithOptions</code>. Cancellation is checked around library-controlled work and is propagated into resolvers; the underlying JSON Schema engine does not expose an interrupt hook in the middle of one synchronous compile/validate call. HTTP(S) fetching is deliberately not enabled implicitly.

<code>RegexpTimeout</code> limits one ECMAScript regexp match. The default value <code>0</code> preserves unlimited matching for full compatibility; set a positive duration when validating against untrusted schemas.

Custom formats, content encodings, content media types, and vocabularies are registered through yamlvalidator-owned types, so callers do not need to import the underlying JSON Schema engine:

~~~go
schema, err := v.CompileJSONSchemaWithOptions(schemaJSON, v.JSONSchemaCompileOptions{
    AssertContent: true,
    ContentEncodings: []v.JSONSchemaContentEncoding{
        {
            Name:   "my-encoding",
            Decode: decodeMyEncoding,
        },
    },
    ContentMediaTypes: []v.JSONSchemaContentMediaType{
        {
            Name:     "application/x-example",
            Validate: validateExampleMediaType,
        },
    },
})
~~~

### Functional JSON Schema extensions

For common extension cases, yamlvalidator exposes its own API so callers do not need to depend directly on the underlying JSON Schema engine.

A custom format is useful for scalar validation and can also validate mapping keys through `propertyNames`:

```go
schema, err := v.CompileJSONSchemaWithOptions(schemaJSON, v.JSONSchemaCompileOptions{
    AssertFormat: true,
    Formats: []v.JSONSchemaFormat{
        {
            Name: "module-name",
            Validate: func(value any) error {
                name, ok := value.(string)
                if ok && !validModuleName(name) {
                    return fmt.Errorf("invalid module name")
                }
                return nil
            },
        },
    },
})
```

The schema can then use `{"format":"module-name"}` or `{"propertyNames":{"format":"module-name"}}`.

For arbitrary business logic, register a functional custom keyword. The simplest form is literally a validation function receiving the keyword value from the schema and the current instance value:

```go
keyword := v.JSONSchemaKeyword{
    Name: "x-require-property",
    Validate: func(raw, instance any, ctx *v.JSONSchemaKeywordValidationContext) {
        property, ok := raw.(string)
        if !ok {
            ctx.AddError("x-require-property expects a string")
            return
        }
        object, ok := instance.(map[string]any)
        if !ok {
            return
        }

        ctx.MarkPropertyEvaluated(property)
        if _, ok := object[property]; !ok {
            ctx.AddErrorAt([]string{property}, "required by x-require-property")
        }
    },
}

schema, err := v.CompileJSONSchemaWithOptions(schemaJSON, v.JSONSchemaCompileOptions{
    Keywords: []v.JSONSchemaKeyword{keyword},
})
```

If keyword configuration needs parsing or validation, use `Compile` instead of `Validate`; it receives the keyword value once at schema compilation time and returns a `JSONSchemaKeywordValidateFunc` for subsequent instance validation.


Keywords that contain nested schemas can use `CompileWithContext` instead of importing the underlying engine. Declare where nested schemas may occur, compile exact branches with `Subschema` or `Reference`, then validate them through `ValidateSubschema`:

```go
keyword := v.JSONSchemaKeyword{
    Name: "x-dispatch",
    Subschemas: []v.JSONSchemaSubschemaPath{
        {v.JSONSchemaSubschemaAllProperties()},
    },
    CompileWithContext: func(
        ctx *v.JSONSchemaKeywordCompileContext,
        raw any,
    ) (v.JSONSchemaKeywordValidateFunc, error) {
        config := raw.(map[string]any)
        branches := make(map[string]*v.JSONSchemaSubschema, len(config))
        for name := range config {
            branches[name] = ctx.Subschema(name)
        }

        return func(instance any, validation *v.JSONSchemaKeywordValidationContext) {
            object, _ := instance.(map[string]any)
            kind, _ := object["kind"].(string)
            if branch := branches[kind]; branch != nil {
                validation.ValidateSubschema(branch, instance, nil)
            } else {
                validation.AddErrorAt([]string{"kind"}, "unknown kind")
            }
        }, nil
    },
}
```

Subschema location helpers cover an exact property, all property values, an exact array item, or all array items. Nested schemas keep normal JSON Schema reference semantics, including `$ref`.

`JSONSchemaKeywordValidationContext` supports multiple diagnostics through `AddError` / `AddErrorAt` and integrates with `unevaluatedProperties` / `unevaluatedItems` through `MarkPropertyEvaluated` / `MarkItemEvaluated`. Relative issue paths are mapped back to the corresponding YAML line and column, and the keyword name becomes the diagnostic `Code`.

Frequently Asked Questions