# Homelab Infrastructure as Code A Pulumi-based IaC template for managing a Proxmox homelab. The goal is to replace manual GUI configuration and ad-hoc YAML stacks with version-controlled, reproducible infrastructure — a highly available k3s cluster across multiple Proxmox nodes, bootstrapped and configured end-to-end from code. This repo is intentionally abstract: credentials are never hardcoded, making it easy to fork and adapt as a template for your own homelab. ## Why IaC for a homelab? - **Reproducibility** — rebuild your entire environment from scratch with a single command - **Version history** — every change is tracked in git; roll back at any time - **Auditability** — diff infrastructure changes before applying them (`pulumi preview`) - **Automation** — CI/CD handles deploys; no manual SSH into nodes - **Portability** — swap node names, datastores, or credentials without touching logic ## Repository layout ``` . ├── proxmox-infra/ # Stack 1 — VMs & DHCP on Proxmox + pfSense │ ├── index.ts │ ├── Pulumi.yaml │ └── sdks/pfsense/ # Locally bundled @pulumi/pfsense SDK ├── k8s-bootstrap/ # Stack 2 — Install k3s on the VMs over SSH │ ├── index.ts │ └── Pulumi.yaml ├── k8s-infra/ # Stack 3 — Cluster-level infrastructure via Helm │ ├── index.ts │ └── Pulumi.yaml └── .gitea/workflows/ # Gitea Actions — one workflow per stack ``` ## Stack overview The three stacks are deployed in order and linked via Pulumi StackReferences — outputs from one stack flow automatically into the next. ``` proxmox-infra → k8s-bootstrap → k8s-infra ``` ### Stack 1: `proxmox-infra` Provisions a 5-node k3s cluster spread across two Proxmox hosts: | VM name | Role | Proxmox node | | ------------ | ------ | ------------ | | k3s-master-1 | master | pve | | k3s-master-2 | master | pve | | k3s-worker-1 | worker | pve | | k3s-master-3 | master | pve-bckp | | k3s-worker-2 | worker | pve-bckp | Each VM is a full clone of an Ubuntu Noble (24.04) cloud-image template. Cloud-init injects the hostname, user credentials, and SSH keys at boot. An ED25519 SSH keypair is generated once and stored in Pulumi state — the public key goes into every VM, the private key is exported as a stack output for `k8s-bootstrap` to consume via StackReference. Each VM's MAC address is registered as a DHCPv4 static mapping in pfSense so nodes always receive their designated IPs. **Providers:** `@muhlba91/pulumi-proxmoxve` v8.x, `@pulumi/pfsense` (locally bundled), `@pulumi/tls` **Exports:** `vmIds`, `ciRunnerPrivateKey`, `pve1Endpoint`, `pve1ApiToken`, `pve2Endpoint`, `pve2ApiToken`, `master1Ip` … `worker2Ip` ### Stack 2: `k8s-bootstrap` Installs k3s on the Proxmox VMs over SSH using `@pulumi/command`. Reads VM IDs, Proxmox credentials, and the SSH private key from `proxmox-infra` via StackReference — no manual key distribution needed. Deployment sequence: 1. Start all 5 VMs via the Proxmox REST API 2. Wait for port 22 to open on each node 3. Install k3s on `k3s-master-1` with `--cluster-init` (embedded etcd) 4. Join `k3s-master-2` and `k3s-master-3` as additional etcd nodes 5. Join `k3s-worker-1` and `k3s-worker-2` as agent nodes 6. Read `/etc/rancher/k3s/k3s.yaml` from master-1, patch the server URL, export as a secret stack output **Exports:** `kubeconfig` (secret) ### Stack 3: `k8s-infra` Deploys cluster-level infrastructure via Helm and the `@pulumi/kubernetes` provider. Reads `kubeconfig` from `k8s-bootstrap` via StackReference. | Component | What it does | | -------------------- | -------------------------------------------------------------------------- | | NFS CSI Driver | `csi-driver-nfs` Helm chart (v4.12.0) — enables dynamic NFS-backed PVCs | | TrueNAS StorageClass | `truenas-nfs` StorageClass backed by an NFS share on TrueNAS | | cert-manager | `cert-manager` Helm chart — certificate lifecycle management | | ClusterIssuer | `letsencrypt-prod` — DNS-01 via Cloudflare, issues Let's Encrypt TLS certs | ## Prerequisites - [Pulumi CLI](https://www.pulumi.com/docs/install/) installed - Node.js 18+ and npm - Two Proxmox nodes with API tokens - pfSense with REST API credentials (used for DHCPv4 static mapping) - TrueNAS with an NFS share (no API key needed — CSI driver connects via NFS protocol directly) - A Cloudflare account with an API token scoped to DNS edit on your zone - A self-hosted Pulumi state backend (PostgreSQL connection string) - Gitea instance for CI/CD (optional for local use) ## Getting started Deploy the stacks **in order**. Each stack must be fully deployed before the next one runs. ### Stack 1 — `proxmox-infra` ```bash cd proxmox-infra npm install # also compiles the bundled pfSense SDK pulumi stack init dev ``` Set secrets: ```bash # Proxmox API credentials pulumi config set --secret pve1Endpoint https://:8006 pulumi config set --secret pve1ApiToken @pam!= pulumi config set --secret pve2Endpoint https://:8006 pulumi config set --secret pve2ApiToken @pam!= # VM credentials pulumi config set --secret k3sVmPassword pulumi config set --secret sshPvePublicKey "ssh-ed25519 AAAA..." # pfSense credentials (used for DHCPv4 static mappings) pulumi config set --secret pfSenseUrl https:// pulumi config set --secret pfSenseUser pulumi config set --secret pfSensePassword # Static IP addresses assigned to each k3s node pulumi config set --secret master1Ip pulumi config set --secret master2Ip pulumi config set --secret worker1Ip pulumi config set --secret master3Ip pulumi config set --secret worker2Ip ``` Deploy: ```bash export PULUMI_BACKEND_URL=postgresql://:@/ export PULUMI_CONFIG_PASSPHRASE= pulumi preview # inspect before touching anything pulumi up --yes ``` ### Stack 2 — `k8s-bootstrap` ```bash cd ../k8s-bootstrap npm install pulumi stack init dev pulumi config set --secret k3sToken "$(openssl rand -hex 32)" ``` Node IPs and Proxmox credentials are read automatically from `proxmox-infra` outputs via StackReference — do not set them here. ```bash pulumi up --yes ``` ### Stack 3 — `k8s-infra` **TrueNAS one-time setup** (before deploying this stack): 1. Create dataset `tank/k8s` 2. Add an NFS share for that dataset - Acl Type: Set this to POSIX (or generic Unix permissions) rather than SMB/NFSv4 ACLs if you are on TrueNAS SCALE, as Kubernetes handles basic Unix permissions natively. - Maproot User / Maproot Group: Set this to root and root - Allowed Hosts/Networks: Restrict this share specifically to the IP addresses or the CIDR block of your Proxmox K3s nodes (e.g., 192.168.1.50) 3. In Network → Allowed Networks, permit your LAN subnet (e.g. `192.168.1.0/24`) ```bash cd ../k8s-infra npm install pulumi stack init dev # kubeconfig from k8s-bootstrap KUBECONFIG=$(cd ../k8s-bootstrap && pulumi stack output kubeconfig --show-secrets) pulumi config set --secret kubeconfig "$KUBECONFIG" # TrueNAS NFS pulumi config set --secret truenasHost pulumi config set --secret truenasNfsPath /mnt/tank/k8s # cert-manager + Cloudflare DNS-01 pulumi config set --secret cloudflareApiToken pulumi config set --secret letsencryptEmail pulumi up --yes ``` ## CI/CD (Gitea Actions) One workflow per stack under `.gitea/workflows/`. Each triggers on changes to its own stack directory. | Event | Action | | --------------------- | ------------------------------ | | Pull request → `main` | `pulumi preview` (read-only) | | Push to `main` | `pulumi refresh` + `pulumi up` | | Manual trigger | `pulumi refresh` + `pulumi up` | ### Required Gitea secrets Configure under **Settings → Actions → Secrets**: | Secret | Used by | Description | | ------------------------------- | --------------------------- | ---------------------------------------------- | | `PULUMI_BACKEND_URL` | all workflows | PostgreSQL connection string for state backend | | `PULUMI_CONFIG_PASSPHRASE` | all workflows | Passphrase to decrypt secrets | | `PROXMOX_INFRA_PULUMI_DEV_YAML` | `deploy-proxmox-infra.yaml` | Base64-encoded `proxmox-infra/Pulumi.dev.yaml` | | `K8S_BOOTSTRAP_PULUMI_DEV_YAML` | `deploy-k8s-bootstrap.yaml` | Base64-encoded `k8s-bootstrap/Pulumi.dev.yaml` | | `K8S_INFRA_PULUMI_DEV_YAML` | `deploy-k8s-infra.yaml` | Base64-encoded `k8s-infra/Pulumi.dev.yaml` | `Pulumi.dev.yaml` files are gitignored (they contain your encryption salt). Whenever one changes, re-encode and paste into the corresponding Gitea secret: ```bash base64 -w 0 proxmox-infra/Pulumi.dev.yaml base64 -w 0 k8s-bootstrap/Pulumi.dev.yaml base64 -w 0 k8s-infra/Pulumi.dev.yaml ``` ## Adapting this as a template 1. Fork or copy the repo 2. Update Proxmox node names (`pve`, `pve-bckp`) and datastore IDs in `proxmox-infra/index.ts` 3. Add or remove VMs from the `nodeConfigs` array 4. Set your own secrets with `pulumi config set --secret` 5. Point the CI/CD workflows at your own Gitea instance ## Roadmap - LXC container management - `k8s-apps` stack — application deployments on the cluster - Docker / Compose stack provisioning - Migrate secrets management to [OpenBao](https://openbao.org/) — replace `PULUMI_CONFIG_PASSPHRASE` and manual `Pulumi.dev.yaml` encoding with a self-hosted vault - Add a third bare metal Proxmox node for true 3-node HA parity