Building Kubernetes-Style APIs: A Practical Walkthrough

If you've spent any time with Kubernetes, you've probably absorbed its API design patterns without thinking much about them. You create a Deployment, and the cluster just... makes it happen. You don't tell Kubernetes how to roll out three replicas — you tell it what you want, and something in the background keeps nudging reality toward that goal.

That pattern — declarative desired state, continuously reconciled by a controller — is worth stealing even outside an actual Kubernetes cluster. This post walks through it end to end: an OpenAPI-first API server, a Postgres-backed store, and a reconciler — using a BackupPolicy resource as the running example.

What makes an API "Kubernetes-style"

Three ideas define the pattern:

1. Declarative, not imperative. You don't say "back this up now." You say "backups should happen every 6 hours, retain 7." The system figures out the individual steps.
2. Spec and status are separate. spec is what the user wants; status is what the controller observes. Resources typically carry a status.conditions array — a standard Kubernetes convention worth copying, since tooling already knows how to render it.
3. Reconciliation is level-triggered, not edge-triggered. The service doesn't react to "the spec changed" as a one-off event — it continuously evaluates "given the current spec, is the current state correct?" This is what makes the system self-healing: drift gets repaired on the next pass without anyone needing to notice and re-trigger anything.

OpenAPI-first, not code-first

A common pattern in these systems: an api/ directory holds hand-maintained OpenAPI YAML, a make generate step produces the Go types and client code from it, and generated *.gen.go files are never edited by hand. This matters more than it sounds — it means the wire contract is the source of truth, not a Go struct that happens to serialize a particular way. It's what lets independent tooling (a CLI, an automation collection, an integration server) build clients against the same API without guessing at the shape.

We'll do the same: the spec lives in its own file, checked into the repo, and both the server-side interface and the client are generated from it — nobody hand-writes request/response structs.

The spec file

# api/v1alpha1/openapi.yaml
openapi: 3.0.3
info:
  title: Platform API
  version: v1alpha1
paths:
  /apis/platform.example.com/v1alpha1/backuppolicies:
    get:
      operationId: listBackupPolicies
      parameters:
        - name: labelSelector
          in: query
          schema: { type: string }
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/BackupPolicyList"
    post:
      operationId: createBackupPolicy
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/BackupPolicy"
      responses:
        "201":
          description: Created
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/BackupPolicy"
        "409":
          description: Already exists

  /apis/platform.example.com/v1alpha1/backuppolicies/{name}:
    get:
      operationId: getBackupPolicy
      parameters:
        - name: name
          in: path
          required: true
          schema: { type: string }
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/BackupPolicy"
        "404":
          description: Not found
    put:
      operationId: updateBackupPolicy
      parameters:
        - name: name
          in: path
          required: true
          schema: { type: string }
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/BackupPolicy"
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/BackupPolicy"
        "409":
          description: resourceVersion conflict
    delete:
      operationId: deleteBackupPolicy
      parameters:
        - name: name
          in: path
          required: true
          schema: { type: string }
      responses:
        "204":
          description: Deleted

components:
  schemas:
    BackupPolicy:
      type: object
      required: [apiVersion, kind, metadata, spec]
      properties:
        apiVersion: { type: string }
        kind: { type: string }
        metadata:
          $ref: "#/components/schemas/ObjectMeta"
        spec:
          $ref: "#/components/schemas/BackupPolicySpec"
        status:
          $ref: "#/components/schemas/BackupPolicyStatus"

    BackupPolicyList:
      type: object
      properties:
        items:
          type: array
          items:
            $ref: "#/components/schemas/BackupPolicy"

    ObjectMeta:
      type: object
      required: [name]
      properties:
        name: { type: string }
        labels:
          type: object
          additionalProperties: { type: string }
        resourceVersion: { type: string }

    BackupPolicySpec:
      type: object
      required: [target, schedule, retention, destination]
      properties:
        target:
          type: object
          properties:
            kind: { type: string }
            name: { type: string }
        schedule:
          type: string
          description: Cron expression for backup frequency
        retention:
          type: object
          properties:
            count: { type: integer, minimum: 1 }
        destination:
          type: object
          properties:
            bucket: { type: string }

    BackupPolicyStatus:
      type: object
      properties:
        lastBackupTime: { type: string, format: date-time }
        lastBackupStatus: { type: string, enum: [Succeeded, Failed, Pending] }
        backupsRetained: { type: integer }
        conditions:
          type: array
          items:
            type: object
            properties:
              type: { type: string }
              status: { type: string, enum: ["True", "False", "Unknown"] }
              reason: { type: string }
              message: { type: string }
              lastTransitionTime: { type: string, format: date-time }

Generating server and client from it

oapi-codegen is the standard tool for this in Go. A config file tells it what to emit:

# api/v1alpha1/codegen-server.yaml
package: apiv1alpha1
generate:
  chi-server: true
  models: true
  embedded-spec: true
output: internal/api/v1alpha1/server.gen.go

# api/v1alpha1/codegen-client.yaml
package: apiv1alpha1client
generate:
  client: true
  models: true
output: client/v1alpha1/client.gen.go

Wired to Makefile target

.PHONY: generate
generate:
	go run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen \
		-config api/v1alpha1/codegen-server.yaml api/v1alpha1/openapi.yaml
	go run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen \
		-config api/v1alpha1/codegen-client.yaml api/v1alpha1/openapi.yaml

make generate produces two things:

1. A server interface the handlers must implement — the router, request parsing, and response marshaling are all generated, so a handler can't drift from the spec without a compile error:

// generated in internal/api/v1alpha1/server.gen.go — do not edit
type ServerInterface interface {
    ListBackupPolicies(w http.ResponseWriter, r *http.Request, params ListBackupPoliciesParams)
    CreateBackupPolicy(w http.ResponseWriter, r *http.Request)
    GetBackupPolicy(w http.ResponseWriter, r *http.Request, name string)
    UpdateBackupPolicy(w http.ResponseWriter, r *http.Request, name string)
    DeleteBackupPolicy(w http.ResponseWriter, r *http.Request, name string)
}

Our hand-written implementation is a thin type satisfying that interface — this is what replaces the manually wired chi.Router from earlier:

type BackupPolicyAPI struct {
    db *pgxpool.Pool
}

func (a *BackupPolicyAPI) CreateBackupPolicy(w http.ResponseWriter, r *http.Request) {
    var policy apiv1alpha1.BackupPolicy
    if err := json.NewDecoder(r.Body).Decode(&policy); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    // ... same transactional insert + event-log write as before,
    // but now `policy.Spec` and `policy.Metadata` are generated,
    // typed structs instead of hand-decoded json.RawMessage.
}

func (a *BackupPolicyAPI) GetBackupPolicy(w http.ResponseWriter, r *http.Request, name string) {
    var policy apiv1alpha1.BackupPolicy
    err := a.db.QueryRow(r.Context(), `
        SELECT spec, status, resource_version FROM objects
        WHERE kind = 'BackupPolicy' AND name = $1 AND deleted_at IS NULL
    `, name).Scan(&policy.Spec, &policy.Status, &policy.Metadata.ResourceVersion)
    if err != nil {
        http.Error(w, "not found", http.StatusNotFound)
        return
    }
    json.NewEncoder(w).Encode(policy)
}

// Mount it:
r := chi.NewRouter()
apiv1alpha1.HandlerFromMux(&BackupPolicyAPI{db: pool}, r)

2. A typed client — this is what the reconciler, the CLI, and any other integration should use instead of hand-rolling HTTP calls:

// generated in client/v1alpha1/client.gen.go — do not edit
client, err := apiv1alpha1client.NewClientWithResponses("https://platform.example.com")
if err != nil {
    return err
}

resp, err := client.GetBackupPolicyWithResponse(ctx, "orders-service-backup")
if err != nil {
    return err
}
if resp.StatusCode() != http.StatusOK {
    return fmt.Errorf("unexpected status: %d", resp.StatusCode())
}
policy := resp.JSON200 // fully typed *BackupPolicy, no manual unmarshaling

The reconciler from earlier in this post can now use client.UpdateBackupPolicyWithResponse(...) instead of talking to the database directly — which also means the reconciler could run as a completely separate service, talking to the API over HTTP rather than sharing a database connection pool with the API server.

This is the benefit an OpenAPI-first setup gets you generally: the spec file is small enough to review in a pull request, make generate catches drift immediately (a handler that doesn't match the interface fails to compile), and every consumer — CLI, worker, third-party integrations — gets an identical, typed view of the API for free.

The resource, as a user would write it

Following typical Kubernetes-style YAML conventions (apiVersion, kind, metadata, spec) and a selector pattern:

apiVersion: platform.example.com/v1alpha1
kind: BackupPolicy
metadata:
  name: orders-service-backup
  labels:
    team: orders
    env: prod
spec:
  target:
    kind: Database
    name: orders-db
  schedule: "0 */6 * * *"
  retention:
    count: 7
  destination:
    bucket: s3://backups-prod/orders-db
status:
  lastBackupTime: "2026-08-12T04:00:00Z"
  lastBackupStatus: Succeeded
  backupsRetained: 7
  conditions:
    - type: Ready
      status: "True"
      reason: BackupsHealthy
      lastTransitionTime: "2026-08-12T04:00:12Z"

A user manages this the way kubectl apply -f manages a Kubernetes resource: myctl apply -f backuppolicy.yaml, myctl get backuppolicies, myctl edit backuppolicy orders-service-backup.

The store: PostgreSQL

In systems like this, integration tests typically spin up an ephemeral Postgres instance via testcontainers — Postgres is the actual store, not a stand-in for etcd used only in tests. The schema needs to support optimistic concurrency (a resourceVersion-equivalent) and a way to stream changes to watchers like the CLI and UI:

CREATE TABLE objects (
    id               UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    kind             TEXT NOT NULL,
    name             TEXT NOT NULL,
    labels           JSONB NOT NULL DEFAULT '{}',
    resource_version BIGINT NOT NULL DEFAULT 1,
    spec             JSONB NOT NULL,
    status           JSONB NOT NULL DEFAULT '{}',
    deleted_at       TIMESTAMPTZ,
    created_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
    UNIQUE (kind, name)
);

CREATE TABLE object_events (
    id               BIGSERIAL PRIMARY KEY,
    object_id        UUID NOT NULL REFERENCES objects(id),
    event_type       TEXT NOT NULL CHECK (event_type IN ('ADDED','MODIFIED','DELETED')),
    resource_version BIGINT NOT NULL,
    payload          JSONB NOT NULL,
    created_at       TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE OR REPLACE FUNCTION notify_object_event() RETURNS trigger AS $$
BEGIN
    PERFORM pg_notify('object_events', NEW.id::text);
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER object_events_notify
AFTER INSERT ON object_events
FOR EACH ROW EXECUTE FUNCTION notify_object_event();

labels as JSONB is what makes label-selector-style querying possible later — WHERE labels @> '{"team":"orders"}' gives you Kubernetes-style label matching without extra tables.

The Store interface

Rather than scattering SQL across HTTP handlers, define a Store interface that's the only thing allowed to know about the schema. Everything above this line — SQL, transactions, LISTEN/NOTIFY — stays behind it:

type Store interface {
    Create(ctx context.Context, kind, name string, labels map[string]string, spec json.RawMessage) (*Object, error)
    Get(ctx context.Context, kind, name string) (*Object, error)
    List(ctx context.Context, kind string, labelSelector map[string]string) ([]*Object, error)
    UpdateSpec(ctx context.Context, kind, name string, spec json.RawMessage, expectedRV int64) (*Object, error)
    UpdateStatus(ctx context.Context, kind, name string, status json.RawMessage) (*Object, error)
    Delete(ctx context.Context, kind, name string) error
    Watch(ctx context.Context, kind string, sinceRV int64) (<-chan Event, error)
}

type Object struct {
    ID              string
    Kind            string
    Name            string
    Labels          map[string]string
    ResourceVersion int64
    Spec            json.RawMessage
    Status          json.RawMessage
}

type Event struct {
    Type   string // ADDED, MODIFIED, DELETED
    Object *Object
}

The Postgres implementation is where the transactional insert-plus-event-log write from before lives — now isolated in one place instead of inline in a handler:

type pgStore struct {
    db *pgxpool.Pool
}

func NewPostgresStore(db *pgxpool.Pool) Store {
    return &pgStore{db: db}
}

func (s *pgStore) Create(ctx context.Context, kind, name string, labels map[string]string, spec json.RawMessage) (*Object, error) {
    tx, err := s.db.Begin(ctx)
    if err != nil {
        return nil, err
    }
    defer tx.Rollback(ctx)

    obj := &Object{Kind: kind, Name: name, Labels: labels, Spec: spec}
    err = tx.QueryRow(ctx, `
        INSERT INTO objects (kind, name, labels, spec, status)
        VALUES ($1, $2, $3, $4, '{}')
        RETURNING id, resource_version
    `, kind, name, labels, spec).Scan(&obj.ID, &obj.ResourceVersion)
    if err != nil {
        return nil, err
    }

    _, err = tx.Exec(ctx, `
        INSERT INTO object_events (object_id, event_type, resource_version, payload)
        VALUES ($1, 'ADDED', $2, (SELECT to_jsonb(o) FROM objects o WHERE o.id = $1))
    `, obj.ID, obj.ResourceVersion)
    if err != nil {
        return nil, err
    }

    if err := tx.Commit(ctx); err != nil {
        return nil, err
    }
    return obj, nil
}

func (s *pgStore) UpdateSpec(ctx context.Context, kind, name string, spec json.RawMessage, expectedRV int64) (*Object, error) {
    var obj Object
    err := s.db.QueryRow(ctx, `
        UPDATE objects
        SET spec = $1, resource_version = resource_version + 1, updated_at = now()
        WHERE kind = $2 AND name = $3 AND resource_version = $4 AND deleted_at IS NULL
        RETURNING id, resource_version
    `, spec, kind, name, expectedRV).Scan(&obj.ID, &obj.ResourceVersion)
    if err != nil {
        return nil, ErrResourceVersionConflict
    }
    return &obj, nil
}

func (s *pgStore) Watch(ctx context.Context, kind string, sinceRV int64) (<-chan Event, error) {
    events := make(chan Event)
    go func() {
        defer close(events)

        rows, _ := s.db.Query(ctx, `
            SELECT event_type, payload FROM object_events
            WHERE resource_version > $1 ORDER BY resource_version ASC
        `, sinceRV)
        for rows.Next() {
            var e Event
            var payload json.RawMessage
            rows.Scan(&e.Type, &payload)
            json.Unmarshal(payload, &e.Object)
            events <- e
        }

        conn, _ := s.db.Acquire(ctx)
        defer conn.Release()
        conn.Exec(ctx, "LISTEN object_events")
        for {
            n, err := conn.Conn().WaitForNotification(ctx)
            if err != nil {
                return
            }
            var e Event
            var payload json.RawMessage
            s.db.QueryRow(ctx, `
                SELECT event_type, payload FROM object_events WHERE id = $1
            `, n.Payload).Scan(&e.Type, &payload)
            json.Unmarshal(payload, &e.Object)
            events <- e
        }
    }()
    return events, nil
}

// Get, List, UpdateStatus, Delete follow the same shape — straight queries
// against `objects`, with UpdateStatus and Delete also writing to object_events.

Nothing outside this file knows the table names, the JSONB columns, or that LISTEN/NOTIFY is involved at all.

The service layer

The Store is pure persistence — it has no opinion about cron expressions, retention counts, or what "Ready" means. That logic belongs in a BackupPolicyService, which wraps a Store and is what the HTTP handlers and the reconciler both actually talk to:

type BackupPolicyService struct {
    store Store
}

func NewBackupPolicyService(store Store) *BackupPolicyService {
    return &BackupPolicyService{store: store}
}

func (s *BackupPolicyService) Create(ctx context.Context, name string, labels map[string]string, spec BackupPolicySpec) (*BackupPolicy, error) {
    if _, err := cron.ParseStandard(spec.Schedule); err != nil {
        return nil, fmt.Errorf("invalid schedule: %w", err)
    }
    if spec.Retention.Count < 1 {
        return nil, fmt.Errorf("retention.count must be at least 1")
    }
    raw, _ := json.Marshal(spec)
    obj, err := s.store.Create(ctx, "BackupPolicy", name, labels, raw)
    if err != nil {
        return nil, err
    }
    return toBackupPolicy(obj), nil
}

func (s *BackupPolicyService) UpdateSpec(ctx context.Context, name string, spec BackupPolicySpec, expectedRV int64) (*BackupPolicy, error) {
    if _, err := cron.ParseStandard(spec.Schedule); err != nil {
        return nil, fmt.Errorf("invalid schedule: %w", err)
    }
    raw, _ := json.Marshal(spec)
    obj, err := s.store.UpdateSpec(ctx, "BackupPolicy", name, raw, expectedRV)
    if err != nil {
        return nil, err
    }
    return toBackupPolicy(obj), nil
}

func (s *BackupPolicyService) Get(ctx context.Context, name string) (*BackupPolicy, error) {
    obj, err := s.store.Get(ctx, "BackupPolicy", name)
    if err != nil {
        return nil, err
    }
    return toBackupPolicy(obj), nil
}

func (s *BackupPolicyService) Watch(ctx context.Context, sinceRV int64) (<-chan Event, error) {
    return s.store.Watch(ctx, "BackupPolicy", sinceRV)
}

Validation (a bad cron expression, a nonsensical retention count) is rejected here, before it ever reaches the store — the store would happily persist garbage, because persistence isn't where correctness belongs.

The API server

Handlers become thin: decode the request, call the service, encode the response. No SQL, no transactions, nothing store-specific in sight:

type API struct {
    svc *BackupPolicyService
}

func (a *API) routes() chi.Router {
    r := chi.NewRouter()
    r.Use(jwtAuthMiddleware)
    r.Route("/apis/platform.example.com/v1alpha1/backuppolicies", func(r chi.Router) {
        r.Post("/", a.handleCreate)
        r.Get("/{name}", a.handleGet)
        r.Put("/{name}", a.handleUpdateSpec)
        r.Get("/{name}/watch", a.handleWatch)
    })
    return r
}

func (a *API) handleCreate(w http.ResponseWriter, r *http.Request) {
    var body struct {
        Metadata struct {
            Name   string            `json:"name"`
            Labels map[string]string `json:"labels"`
        } `json:"metadata"`
        Spec BackupPolicySpec `json:"spec"`
    }
    if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    policy, err := a.svc.Create(r.Context(), body.Metadata.Name, body.Metadata.Labels, body.Spec)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(policy)
}

func (a *API) handleUpdateSpec(w http.ResponseWriter, r *http.Request) {
    name := chi.URLParam(r, "name")
    var body struct {
        ResourceVersion int64            `json:"resourceVersion"`
        Spec            BackupPolicySpec `json:"spec"`
    }
    json.NewDecoder(r.Body).Decode(&body)

    policy, err := a.svc.UpdateSpec(r.Context(), name, body.Spec, body.ResourceVersion)
    if errors.Is(err, ErrResourceVersionConflict) {
        http.Error(w, "resourceVersion conflict", http.StatusConflict)
        return
    }
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    json.NewEncoder(w).Encode(policy)
}

func (a *API) handleWatch(w http.ResponseWriter, r *http.Request) {
    sinceRV, _ := strconv.ParseInt(r.URL.Query().Get("resourceVersion"), 10, 64)
    flusher, _ := w.(http.Flusher)

    events, err := a.svc.Watch(r.Context(), sinceRV)
    if err != nil {
        http.Error(w, "internal error", http.StatusInternalServerError)
        return
    }
    for e := range events {
        json.NewEncoder(w).Encode(e)
        flusher.Flush()
    }
}

Everything the generated ServerInterface requires is satisfied by *API — the interface enforces the shape, the service enforces the rules, and the store enforces persistence. Three layers, each replaceable independently: swap Postgres for something else without touching the service or the handlers; change a validation rule without touching SQL.

The reconciler / worker

The reconciler is just another client of the service — it doesn't reach into the store directly, and it doesn't need to run inside the API process:

type Reconciler struct {
    svc *BackupPolicyService
}

func (c *Reconciler) Run(ctx context.Context) {
    var lastRV int64
    for {
        events, err := c.svc.Watch(ctx, lastRV)
        if err != nil {
            time.Sleep(time.Second)
            continue
        }
        for ev := range events {
            var policy BackupPolicy
            json.Unmarshal(ev.Object.Spec, &policy.Spec)
            c.reconcile(ctx, ev.Object.Name, &policy)
            lastRV = ev.Object.ResourceVersion
        }
    }
}

func (c *Reconciler) reconcile(ctx context.Context, name string, policy *BackupPolicy) {
    due, err := nextScheduledBackup(policy.Spec.Schedule, policy.Status.LastBackupTime)
    if err != nil {
        c.setCondition(ctx, name, "Ready", "False", "InvalidSchedule")
        return
    }
    if time.Now().Before(due) {
        return
    }
    if err := c.runBackup(ctx, policy); err != nil {
        c.setCondition(ctx, name, "Ready", "False", "BackupFailed")
        return
    }
    c.setCondition(ctx, name, "Ready", "True", "BackupsHealthy")
}

Like client-go informers, this should also run a periodic full resync on top of the watch, so a missed event or a restart during downtime doesn't leave a policy silently stuck. Because it goes through the service rather than the store, the reconciler could run in-process today and move to a fully separate binary later without a rewrite — only how it's wired up changes.

Where people go wrong

• Skipping the OpenAPI-first step. Hand-writing Go structs and letting JSON serialization be the de facto contract works until a second client (a CLI, an automation collection, an integration server) needs to talk to the same API and starts guessing at the shape.
• Editing generated code by hand. The moment someone patches a .gen.go file to fix a bug, the next make generate silently reverts it. Fix the spec, regenerate — never the other way around.
• Letting SQL leak into handlers. The moment a handler queries the database directly, you've lost the one place validation and business rules were supposed to live, and every new handler has to re-derive the rules on its own.
• Treating status like a cache the client can write to. If clients can set status fields directly, you've lost the single source of truth. Route status writes through the service, not the store, and don't expose a status-setting method to the user-facing API at all.
• Trusting LISTEN/NOTIFY as durable. It isn't — the object_events table is the durable record; NOTIFY is only a wake-up signal for watchers already listening.
• No periodic resync in the worker. Watch-only reconciliation looks correct until a connection blips and nobody notices a stuck policy.

Why this pattern generalizes

None of this machinery is Kubernetes-specific — it's a general recipe for building declarative, self-healing systems on infrastructure you already trust, like Postgres instead of etcd. Once spec/status separation, resourceVersion-based concurrency, and a durable watch log are in place, you can bolt reconcilers for anything onto the same server: provisioning, certificate rotation, backup enforcement, whatever your domain needs. The service/store split is what keeps that generic: the store never changes shape no matter how many resource types you add, and each new resource just gets its own thin service on top.