protovalidate-go

protovalidate-go is the Go language implementation of protovalidate designed to validate Protobuf messages at runtime based on user-defined validation constraints. Powered by Google's Common Expression Language (CEL), it provides a flexible and efficient foundation for defining and evaluating custom validation rules. The primary goal of protovalidate is to help developers ensure data consistency and integrity across the network without requiring generated code.

The protovalidate project

Head over to the core protovalidate repository for:

• The API definition: used to describe validation constraints
• Documentation: how to apply protovalidate effectively
• Migration tooling: incrementally migrate from protoc-gen-validate
• Conformance testing utilities: for acceptance testing of protovalidate implementations

Other protovalidate runtime implementations:

• C++: protovalidate-cc
• Java: protovalidate-java
• Python: protovalidate-python

And others coming soon:

• TypeScript: protovalidate-ts

Installation

Requires the go toolchain (≥ v1.18)

To install the package, use the go get command from within your Go module:

go get github.com/bufbuild/protovalidate-go

Import the package into your Go project:

import "github.com/bufbuild/protovalidate-go"

Remember to always check for the latest version of protovalidate-go on the project's GitHub releases page to ensure you're using the most up-to-date version.

Usage

Implementing validation constraints

Validation constraints are defined directly within .proto files. Documentation for adding constraints can be found in the protovalidate project README and its comprehensive docs.

syntax = "proto3";

package my.package;

import "google/protobuf/timestamp.proto";
import "buf/validate/validate.proto";

message Transaction {
  uint64 id = 1 [(buf.validate.field).uint64.gt = 999];
  google.protobuf.Timestamp purchase_date = 2;
  google.protobuf.Timestamp delivery_date = 3;
  
  string price = 4 [(buf.validate.field).cel = {
    id: "transaction.price",
    message: "price must be positive and include a valid currency symbol ($ or £)",
    expression: "(this.startsWith('$') || this.startsWith('£')) && double(this.substring(1)) > 0"
  }];

  option (buf.validate.message).cel = {
    id: "transaction.delivery_date",
    message: "delivery date must be after purchase date",
    expression: "this.delivery_date > this.purchase_date"
  };
}

Buf managed mode

protovalidate-go assumes the constraint extensions are imported into the generated code via buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go.

If you are using Buf managed mode to augment Go code generation, ensure that the protovalidate module is excluded in your buf.gen.yaml:

version: v1
# <snip>
managed:
  enabled: true
  go_package_prefix:
    except:
      - buf.build/bufbuild/protovalidate
# <snip>

Example

package main

import (
	"fmt"
	"time"
	
	pb "github.com/path/to/generated/protos"
	"github.com/bufbuild/protovalidate-go"
	"google.golang.org/protobuf/types/known/timestamppb"
)

func main() {
	msg := &pb.Transaction{
		Id:           1234,
		Price:        "$5.67",
		PurchaseDate: timestamppb.New(time.Now()),
		DeliveryDate: timestamppb.New(time.Now().Add(time.Hour)),
	}

	v, err := protovalidate.New()
	if err != nil {
		fmt.Println("failed to initialize validator:", err)
	}

	if err = v.Validate(msg); err != nil {
		fmt.Println("validation failed:", err)
	} else {
		fmt.Println("validation succeeded")
	}
}

Lazy mode

protovalidate-go defaults to lazily construct validation logic for Protobuf message types the first time they are encountered. A validator's internal cache can be pre-warmed with the WithMessages or WithDescriptors options during initialization:

validator, err := protovalidate.New(
  protovalidate.WithMessages(
    &pb.MyFoo{}, 
    &pb.MyBar{}, 
  ),
)

Lazy mode requires usage of a mutex to keep the validator thread-safe, which results in about 50% of CPU time spent obtaining a read lock. While performance is sub-microsecond, the mutex overhead can be further reduced by disabling lazy mode with the WithDisableLazy option. Note that all expected messages must be provided during initialization of the validator:

validator, err := protovalidate.New(
  protovalidate.WithDisableLazy(true),
  protovalidate.WithMessages(
    &pb.MyFoo{},
    &pb.MyBar{},
  ),
)

Support legacy protoc-gen-validate constraints

The protovalidate-go module comes with a legacy package which adds opt-in support for existing protoc-gen-validate constraints. Provide thelegacy.WithLegacySupport option when initializing the validator:

validator, err := protovalidate.New(
  legacy.WithLegacySupport(legacy.ModeMerge),
)

protoc-gen-validate code generation is not used by protovalidate-go. The legacy package assumes the protoc-gen-validate extensions are imported into the generated code via github.com/envoyproxy/protoc-gen-validate/validate.

A migration tool is also available to incrementally upgrade legacy constraints in .proto files.

Performance

Benchmarks are provided to test a variety of use-cases. Generally, after the initial cold start, validation on a message is sub-microsecond and only allocates in the event of a validation error.

[circa 15 May 2023]
goos: darwin
goarch: arm64
pkg: github.com/bufbuild/protovalidate-go
BenchmarkValidator
BenchmarkValidator/ColdStart
BenchmarkValidator/ColdStart-10         	    4372	    276457 ns/op	  470780 B/op	    9255 allocs/op
BenchmarkValidator/Lazy/Valid
BenchmarkValidator/Lazy/Valid-10        	 9022392	     134.1 ns/op	       0 B/op	       0 allocs/op
BenchmarkValidator/Lazy/Invalid
BenchmarkValidator/Lazy/Invalid-10      	 3416996	     355.9 ns/op	     632 B/op	      14 allocs/op
BenchmarkValidator/Lazy/FailFast
BenchmarkValidator/Lazy/FailFast-10     	 6751131	     172.6 ns/op	     168 B/op	       3 allocs/op
BenchmarkValidator/PreWarmed/Valid
BenchmarkValidator/PreWarmed/Valid-10   	17557560	     69.10 ns/op	       0 B/op	       0 allocs/op
BenchmarkValidator/PreWarmed/Invalid
BenchmarkValidator/PreWarmed/Invalid-10 	 3621961	     332.9 ns/op	     632 B/op	      14 allocs/op
BenchmarkValidator/PreWarmed/FailFast
BenchmarkValidator/PreWarmed/FailFast-10	13960359	     92.22 ns/op	     168 B/op	       3 allocs/op
PASS

Ecosystem

• protovalidate core repository
• Buf
• CEL Go
• CEL Spec

Legal

Offered under the Apache 2 license.