kasun 8b913e921f
Deploy k8s Infra / Pulumi Preview (pull_request) Failing after 15m19s
Deploy k8s Infra / Pulumi Deploy (pull_request) Has been skipped
added longhorn pod deletion policy for stateful apps
2026-06-20 02:51:09 +02:00
2026-06-12 18:33:09 +02:00
2026-06-12 18:33:09 +02:00
2026-06-12 18:33:09 +02:00

Homelab Infrastructure as Code

A Pulumi-based IaC project for a Proxmox homelab: a highly-available k3s cluster, its cluster-level infrastructure, and the applications running on top of it — all provisioned and configured from code.

This repo is intentionally abstract: credentials are never hardcoded, making it easy to fork and adapt as a template for your own homelab.

Why IaC for a homelab?

  • Reproducibility — rebuild the entire environment from scratch from code
  • Version history — every change is tracked in git; roll back at any time
  • Auditabilitypulumi preview diffs infrastructure changes before they're applied
  • Automation — CI/CD handles deploys; no manual SSH into nodes for routine changes
  • Portability — swap node names, datastores, domains, or credentials without touching logic

Repository layout

.
├── 01-proxmox-infra/   # Stack 1 — k3s VMs + DHCP on Proxmox/pfSense
├── 02-k8s-bootstrap/   # Stack 2 — install k3s on the VMs over SSH
├── 03-k8s-infra/       # Stack 3 — storage, cert-manager, kube-vip
├── 04-k8s-apps/        # Stack 4 — application workloads
├── 05-monitoring/      # Stack 5 — kube-prometheus-stack (Prometheus + Grafana)
├── .gitea/workflows/   # CI/CD — one workflow per stack
└── CLAUDE.md           # Conventions & architecture notes (root + one per stack)

Each stack is an independent Pulumi project (own Pulumi.yaml/package.json), all using the Pulumi stack name dev.

Stacks

Stacks are deployed in order and linked via pulumi.StackReference — outputs from one stack flow automatically into the next, so a stack must be deployed before anything depending on it will work.

proxmox-infra → k8s-bootstrap → k8s-infra
                              ↘ k8s-apps → monitoring

1. proxmox-infra

Provisions a 5-node k3s cluster across two Proxmox hosts, each VM a clone of an Ubuntu Noble (24.04) cloud-image template:

VM Role
k3s-master-1 master
k3s-master-2 master
k3s-worker-1 worker
k3s-master-3 master
k3s-worker-2 worker

It also generates an ED25519 SSH keypair (stored in Pulumi state, public key injected via cloud-init) and registers each VM's MAC address as a DHCPv4 static mapping in pfSense.

Providers: @muhlba91/pulumi-proxmoxve, @pulumi/pfsense (locally bundled), @pulumi/tls

2. k8s-bootstrap

Starts the VMs via the Proxmox API and installs k3s over SSH: k3s-master-1 initializes the cluster (--cluster-init, embedded etcd), k3s-master-2/k3s-master-3 join as additional etcd nodes, and the two workers join as agents. Exports the cluster's kubeconfig as a secret for downstream stacks.

3. k8s-infra

Cluster-level infrastructure via Helm and @pulumi/kubernetes:

Component What it does
NFS CSI Driver + truenas-nfs StorageClass Dynamic NFS-backed PVCs from a TrueNAS share
Longhorn Default block storage (longhorn StorageClass, 3 replicas)
cert-manager + letsencrypt-prod ClusterIssuer TLS certs via Let's Encrypt, DNS-01 through Cloudflare
kube-vip + kube-vip-cloud-provider Announces a virtual IP and allocates it to LoadBalancer Services (e.g. Traefik)

4. k8s-apps

Application workloads exposed at app.<domain> via Traefik with a Let's Encrypt cert.

5. monitoring

kube-prometheus-stack Helm release — Prometheus (20Gi Longhorn PVC) and Grafana (2Gi PVC, ingress at grafana.<domain>).

Prerequisites

  • Pulumi CLI, Node.js 24+ and npm
  • One or more Proxmox nodes with API tokens
  • pfSense (DHCPv4 static mappings)
  • TrueNAS with an NFS share (connected directly — no API key needed)
  • A domain with DNS managed by Cloudflare (cert-manager DNS-01 + ingress hostnames)
  • A self-hosted Pulumi state backend (PostgreSQL connection string)
  • Gitea instance for CI/CD (optional for local-only use)

Getting started

Deploy stacks in order — each must be fully pulumi up'd before the next stack's StackReference resolves. Set the shared backend env vars once per shell:

export PULUMI_BACKEND_URL=postgresql://<user>:<pass>@<host>/<db>
export PULUMI_CONFIG_PASSPHRASE=<your-passphrase>

Then for each stack, in order:

cd 0N-<stack-dir>
npm install
pulumi stack init dev
pulumi config set ...   # required keys and secrets
pulumi up --yes

Stack-specific notes:

  1. proxmox-infra — set Proxmox API tokens, pfSense credentials, VM credentials/SSH public key, and a static IP per node.
  2. k8s-bootstrap — set k3sToken (e.g. openssl rand -hex 32) and a static IP per node. Proxmox credentials and the SSH key come automatically from proxmox-infra.
  3. k8s-infra — before deploying, create an NFS dataset/share on TrueNAS (POSIX ACLs, root/root maproot, restricted to your k3s nodes). Set the TrueNAS, Cloudflare, and kubeVipAddress/kubeVipInterface config.
  4. k8s-apps — set domain, the base domain used for ingress hostnames.
  5. monitoring — set grafanaAdminPassword.

CI/CD (Gitea Actions)

One workflow per stack, .gitea/workflows/0N-deploy-<stack>.yaml, scoped to changes under that stack's directory.

Trigger Stacks Action
PR → main all pulumi preview (read-only)
Push to main 0305 pulumi refresh + pulumi up
Manual dispatch all pulumi refresh + pulumi up

Stacks 01/02 (VM provisioning, cluster bootstrap) deploy only via manual dispatch — they're riskier to automate.

Required secrets

Secret Used by
PULUMI_BACKEND_URL, PULUMI_CONFIG_PASSPHRASE all workflows
PROXMOX_INFRA_PULUMI_DEV_YAML 01-deploy-proxmox-infra
K8S_BOOTSTRAP_PULUMI_DEV_YAML 02-deploy-k8s-bootstrap
K8S_INFRA_PULUMI_DEV_YAML 03-deploy-k8s-infra
K8S_APPS_PULUMI_DEV_YAML 04-deploy-k8s-apps
MONITORING_PULUMI_DEV_YAML 05-deploy-monitoring

Each <STACK>_PULUMI_DEV_YAML secret is a base64-encoded copy of that stack's gitignored Pulumi.dev.yaml. After changing local config, re-encode and update the secret:

base64 -w 0 0N-<stack>/Pulumi.dev.yaml

Using this as a template

  1. Fork or copy the repo
  2. Update Proxmox node names/datastores and the nodeConfigs array in 01-proxmox-infra/index.ts for your VM topology
  3. Set your own config/secrets per stack with pulumi config set [--secret] — see each stack's CLAUDE.md for the full list of required keys
  4. Point the CI/CD workflows at your own Gitea/GitOps instance and update the secrets above

Roadmap

  • Migrate secrets management to OpenBao — replace PULUMI_CONFIG_PASSPHRASE and manual Pulumi.dev.yaml encoding with a self-hosted vault
  • Add a third bare-metal Proxmox node for true 3-node HA parity
S
Description
I set up my Proxmox homelab with Pulumi and Gitea Actions to utilize IaC.
Readme
576 KiB
Languages
TypeScript 100%