k8s-infra

This file provides guidance to Claude Code when working in 03-k8s-infra. See the repo-root CLAUDE.md for shared conventions, deployment order, and CI/CD.

Cluster-level infrastructure for the k3s homelab cluster, deployed via Helm and @pulumi/kubernetes. Reads kubeconfig from k8s-bootstrap via StackReference.

Common Commands

npm install
pulumi refresh --yes && pulumi up --yes

CI installs the helm CLI (azure/setup-helm) before running Pulumi.

Architecture

Component	What it does
NFS CSI Driver	`csi-driver-nfs` Helm chart — enables NFS-backed PVCs
`truenas-nfs` StorageClass	Backed by an NFS share on TrueNAS (`Retain` reclaim policy)
cert-manager	Certificate lifecycle management
`letsencrypt-prod` ClusterIssuer	DNS-01 via Cloudflare, issues Let's Encrypt TLS certs
Longhorn	Default block storage (`longhorn` StorageClass, 2 replicas) — used by app/monitoring PVCs. Worker nodes get an extra `longhorn-extra` disk at `/mnt/longhorn-extra` (a dedicated mount set up by `k8s-bootstrap`, patched onto the Longhorn `Node` CR here) so Longhorn data stays off the root disk and doesn't cause `DiskPressure`. `nodeDownPodDeletionPolicy` deletes both the StatefulSet and Deployment pod on node-down so stateful workloads reschedule automatically
kube-vip	DaemonSet that ARP-announces a virtual IP for LoadBalancer Services
kube-vip-cloud-provider	Allocates LoadBalancer IPs from a single-address pool (`kubeVipAddress`) — k3s's built-in ServiceLB is not used
`traefik` ServicePatch	Pins Traefik's LoadBalancer IP to `kubeVipAddress`

Export: nfsStorageClass = "truenas-nfs".

Required Pulumi config

pulumi config set --secret truenasHost      <truenas-ip>
pulumi config set --secret truenasNfsPath   /mnt/pool/k8s
pulumi config set --secret cloudflareApiToken <cf-token>
pulumi config set --secret letsencryptEmail   <your-email>
pulumi config set --secret kubeVipAddress   <vip-ip>
pulumi config set kubeVipInterface          <e.g. eth0>

kubeconfig is read automatically via StackReference — do not set it here.

TrueNAS one-time setup (before deploying)

Create dataset pool/k8s
Add an NFS share for that dataset:
- ACL type: POSIX/generic Unix permissions (not SMB/NFSv4 ACLs) on TrueNAS SCALE
- Maproot User / Group: root / root
- Allowed Hosts/Networks: restrict to the k3s node IPs/CIDR
In Network → Allowed Networks, permit your LAN subnet (e.g. 192.168.1.0/24)

CI/CD

Workflow: ../.gitea/workflows/03-deploy-k8s-infra.yaml. Secret: K8S_INFRA_PULUMI_DEV_YAML.

2.6 KiB Raw Blame History