Platforms

Kubernetes

Deploy Rivet Engine to Kubernetes with PostgreSQL storage.

Quick Start

1. Create Namespace

Save as namespace.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: rivet-engine

Apply:

kubectl apply -f namespace.yaml

2. Deploy PostgreSQL

Save as postgres.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
  namespace: rivet-engine
data:
  POSTGRES_DB: rivet
  POSTGRES_USER: postgres
---
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
  namespace: rivet-engine
type: Opaque
stringData:
  # IMPORTANT: Change this password in production!
  POSTGRES_PASSWORD: "postgres"
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: rivet-engine
spec:
  type: ClusterIP
  ports:
  - port: 5432
    targetPort: 5432
    protocol: TCP
    name: postgres
  selector:
    app: postgres
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: rivet-engine
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:17
        ports:
        - containerPort: 5432
          name: postgres
        envFrom:
        - configMapRef:
            name: postgres-config
        - secretRef:
            name: postgres-secret
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
          subPath: pgdata
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
          limits:
            cpu: 1000m
            memory: 2Gi
        livenessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - pg_isready -U postgres
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - pg_isready -U postgres
          initialDelaySeconds: 5
          periodSeconds: 5
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

Apply and wait for PostgreSQL to be ready:

kubectl apply -f postgres.yaml
kubectl -n rivet-engine wait --for=condition=ready pod -l app=postgres --timeout=300s

3. Deploy Rivet Engine

The Rivet Engine deployment consists of two components:

Main Engine Deployment: Runs all services except singleton services. Configured with Horizontal Pod Autoscaling (HPA) to automatically scale between 2-10 replicas based on CPU and memory utilization.
Singleton Engine Deployment: Runs singleton services that must have exactly 1 replica (e.g., schedulers, coordinators).

Save as rivet-engine.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: engine-config
  namespace: rivet-engine
data:
  config.jsonc: |
    {
      "postgres": {
        "url": "postgresql://postgres:postgres@postgres:5432/rivet"
      },
      "topology": {
        "datacenter_label": 1,
        "datacenters": [
          {
            "name": "local",
            "datacenter_label": 1,
            "is_leader": true,
            "public_url": "http://localhost:6420",
            "peer_url": "http://rivet-engine.rivet-engine.svc.cluster.local:6421",
            "proxy_url": "http://rivet-engine.rivet-engine.svc.cluster.local:6420"
          }
        ]
      }
    }
---
apiVersion: v1
kind: Service
metadata:
  name: rivet-engine
  namespace: rivet-engine
spec:
  type: LoadBalancer
  ports:
  - name: guard
    port: 6420
    targetPort: 6420
    protocol: TCP
  - name: api-peer
    port: 6421
    targetPort: 6421
    protocol: TCP
  selector:
    app: rivet-engine
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rivet-engine
  namespace: rivet-engine
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rivet-engine
  template:
    metadata:
      labels:
        app: rivet-engine
    spec:
      containers:
      - name: rivet-engine
        image: rivetdev/engine:latest
        args:
        - start
        - --except-services
        - singleton
        env:
        - name: RIVET_CONFIG_PATH
          value: /etc/rivet/config.jsonc
        ports:
        - containerPort: 6420
          name: guard
        - containerPort: 6421
          name: api-peer
        volumeMounts:
        - name: config
          mountPath: /etc/rivet
          readOnly: true
        resources:
          requests:
            cpu: 2000m
            memory: 4Gi
          limits:
            cpu: 4000m
            memory: 8Gi
        startupProbe:
          httpGet:
            path: /health
            port: 6421
          initialDelaySeconds: 30
          periodSeconds: 10
          failureThreshold: 30
        readinessProbe:
          httpGet:
            path: /health
            port: 6421
          periodSeconds: 5
          failureThreshold: 2
        livenessProbe:
          httpGet:
            path: /health
            port: 6421
          periodSeconds: 10
          failureThreshold: 3
      volumes:
      - name: config
        configMap:
          name: engine-config
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: rivet-engine
  namespace: rivet-engine
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: rivet-engine
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rivet-engine-singleton
  namespace: rivet-engine
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rivet-engine-singleton
  template:
    metadata:
      labels:
        app: rivet-engine-singleton
    spec:
      containers:
      - name: rivet-engine
        image: rivetdev/engine:latest
        args:
        - start
        - --services
        - singleton
        - --services
        - api-peer
        env:
        - name: RIVET_CONFIG_PATH
          value: /etc/rivet/config.jsonc
        ports:
        - containerPort: 6421
          name: api-peer
        volumeMounts:
        - name: config
          mountPath: /etc/rivet
          readOnly: true
        resources:
          requests:
            cpu: 2000m
            memory: 4Gi
          limits:
            cpu: 4000m
            memory: 8Gi
        startupProbe:
          httpGet:
            path: /health
            port: 6421
          initialDelaySeconds: 30
          periodSeconds: 10
          failureThreshold: 30
        readinessProbe:
          httpGet:
            path: /health
            port: 6421
          periodSeconds: 5
          failureThreshold: 2
        livenessProbe:
          httpGet:
            path: /health
            port: 6421
          periodSeconds: 10
          failureThreshold: 3
      volumes:
      - name: config
        configMap:
          name: engine-config

Apply and wait for the engine to be ready:

kubectl apply -f rivet-engine.yaml
kubectl -n rivet-engine wait --for=condition=ready pod -l app=rivet-engine --timeout=300s
kubectl -n rivet-engine wait --for=condition=ready pod -l app=rivet-engine-singleton --timeout=300s

Note: The HPA requires a metrics server to be running in your cluster. Most Kubernetes distributions (including k3d, GKE, EKS, AKS) include this by default.

4. Verify Deployment

Check that all pods are running (you should see 2+ engine pods and 1 singleton pod):

kubectl -n rivet-engine get pods
kubectl -n rivet-engine get hpa

5. Access the Engine

Get the service URL:

# For LoadBalancer
kubectl -n rivet-engine get service rivet-engine

# For port forwarding (local development)
kubectl -n rivet-engine port-forward service/rivet-engine 6420:6420 6421:6421

Test the health endpoint:

curl http://localhost:6420/health

Expected response:

{"runtime":"engine","status":"ok","version":"..."}

Local Development with k3d

For local Kubernetes testing with k3d:

# Create k3d cluster
k3d cluster create rivet \
  --api-port 6550 \
  -p "6420:30420@loadbalancer" \
  -p "6421:30421@loadbalancer" \
  --agents 2

# Apply manifests (use NodePort service type for k3d)
kubectl apply -f namespace.yaml
kubectl apply -f postgres.yaml
kubectl -n rivet-engine wait --for=condition=ready pod -l app=postgres --timeout=300s

# Modify rivet-engine.yaml service to use NodePort before applying:
# Change `type: LoadBalancer` to `type: NodePort`
# Add nodePort fields:
#   - name: guard
#     port: 6420
#     targetPort: 6420
#     nodePort: 30420
#   - name: api-peer
#     port: 6421
#     targetPort: 6421
#     nodePort: 30421

kubectl apply -f rivet-engine.yaml
kubectl -n rivet-engine wait --for=condition=ready pod -l app=rivet-engine --timeout=300s

# Access at http://localhost:6420 and http://localhost:6421

Cleanup:

k3d cluster delete rivet

Production Setup

Security

Change PostgreSQL password in postgres-secret
Use TLS for external access (configure ingress controller)

Set admin token via environment variable:

env:
- name: RIVET__AUTH__ADMIN_TOKEN
  valueFrom:
    secretKeyRef:
      name: rivet-secrets
      key: admin-token

Scaling

The engine is configured with Horizontal Pod Autoscaling (HPA) by default, automatically scaling between 2-10 replicas based on CPU (60%) and memory (80%) utilization.

To adjust the scaling parameters, modify the HPA configuration:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: rivet-engine
  namespace: rivet-engine
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: rivet-engine
  minReplicas: 2  # Adjust minimum replicas
  maxReplicas: 20 # Adjust maximum replicas
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70  # Adjust CPU threshold
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80  # Adjust memory threshold

Monitor HPA status:

kubectl -n rivet-engine get hpa
kubectl -n rivet-engine describe hpa rivet-engine

Next Steps

See Configuration for all options
For advanced multi-datacenter setup, see the GitHub repository