56 lines
2.6 KiB
Markdown
56 lines
2.6 KiB
Markdown
# k8s-infra
|
|
|
|
This file provides guidance to Claude Code when working in `03-k8s-infra`. See the [repo-root CLAUDE.md](../CLAUDE.md) for shared conventions, deployment order, and CI/CD.
|
|
|
|
Cluster-level infrastructure for the k3s homelab cluster, deployed via Helm and `@pulumi/kubernetes`. Reads `kubeconfig` from `k8s-bootstrap` via StackReference.
|
|
|
|
## Common Commands
|
|
|
|
```bash
|
|
npm install
|
|
pulumi refresh --yes && pulumi up --yes
|
|
```
|
|
|
|
CI installs the `helm` CLI (`azure/setup-helm`) before running Pulumi.
|
|
|
|
## Architecture
|
|
|
|
| Component | What it does |
|
|
| --- | --- |
|
|
| NFS CSI Driver | `csi-driver-nfs` Helm chart — enables NFS-backed PVCs |
|
|
| `truenas-nfs` StorageClass | Backed by an NFS share on TrueNAS (`Retain` reclaim policy) |
|
|
| cert-manager | Certificate lifecycle management |
|
|
| `letsencrypt-prod` ClusterIssuer | DNS-01 via Cloudflare, issues Let's Encrypt TLS certs |
|
|
| Longhorn | Default block storage (`longhorn` StorageClass, 2 replicas) — used by app/monitoring PVCs. Worker nodes get an extra `longhorn-extra` disk at `/mnt/longhorn-extra` (a dedicated mount set up by `k8s-bootstrap`, patched onto the Longhorn `Node` CR here) so Longhorn data stays off the root disk and doesn't cause `DiskPressure`. `nodeDownPodDeletionPolicy` deletes both the StatefulSet and Deployment pod on node-down so stateful workloads reschedule automatically |
|
|
| kube-vip | DaemonSet that ARP-announces a virtual IP for LoadBalancer Services |
|
|
| kube-vip-cloud-provider | Allocates LoadBalancer IPs from a single-address pool (`kubeVipAddress`) — k3s's built-in ServiceLB is not used |
|
|
| `traefik` ServicePatch | Pins Traefik's LoadBalancer IP to `kubeVipAddress` |
|
|
|
|
**Export**: `nfsStorageClass = "truenas-nfs"`.
|
|
|
|
## Required Pulumi config
|
|
|
|
```bash
|
|
pulumi config set --secret truenasHost <truenas-ip>
|
|
pulumi config set --secret truenasNfsPath /mnt/pool/k8s
|
|
pulumi config set --secret cloudflareApiToken <cf-token>
|
|
pulumi config set --secret letsencryptEmail <your-email>
|
|
pulumi config set --secret kubeVipAddress <vip-ip>
|
|
pulumi config set kubeVipInterface <e.g. eth0>
|
|
```
|
|
|
|
`kubeconfig` is read automatically via StackReference — do not set it here.
|
|
|
|
## TrueNAS one-time setup (before deploying)
|
|
|
|
1. Create dataset `pool/k8s`
|
|
2. Add an NFS share for that dataset:
|
|
- ACL type: POSIX/generic Unix permissions (not SMB/NFSv4 ACLs) on TrueNAS SCALE
|
|
- Maproot User / Group: `root` / `root`
|
|
- Allowed Hosts/Networks: restrict to the k3s node IPs/CIDR
|
|
3. In Network → Allowed Networks, permit your LAN subnet (e.g. `192.168.1.0/24`)
|
|
|
|
## CI/CD
|
|
|
|
Workflow: `../.gitea/workflows/03-deploy-k8s-infra.yaml`. Secret: `K8S_INFRA_PULUMI_DEV_YAML`.
|