# Homelab Infrastructure as Code

A Pulumi-based IaC project for a Proxmox homelab: a highly-available k3s cluster, its cluster-level infrastructure, and the applications running on top of it — all provisioned and configured from code.

This repo is intentionally abstract: credentials are never hardcoded, making it easy to fork and adapt as a template for your own homelab.

## Why IaC for a homelab?

- **Reproducibility** — rebuild the entire environment from scratch from code
- **Version history** — every change is tracked in git; roll back at any time
- **Auditability** — `pulumi preview` diffs infrastructure changes before they're applied
- **Automation** — CI/CD handles deploys; no manual SSH into nodes for routine changes
- **Portability** — swap node names, datastores, domains, or credentials without touching logic

## Repository layout

```
.
├── 01-proxmox-infra/   # Stack 1 — k3s VMs + DHCP on Proxmox/pfSense
├── 02-k8s-bootstrap/   # Stack 2 — install k3s on the VMs over SSH
├── 03-k8s-infra/       # Stack 3 — storage, cert-manager, kube-vip
├── 04-k8s-apps/        # Stack 4 — application workloads
├── 05-monitoring/      # Stack 5 — kube-prometheus-stack (Prometheus + Grafana)
├── .gitea/workflows/   # CI/CD — one workflow per stack
└── CLAUDE.md           # Conventions & architecture notes (root + one per stack)
```

Each stack is an independent Pulumi project (own `Pulumi.yaml`/`package.json`), all using the Pulumi stack name `dev`.

## Stacks

Stacks are deployed in order and linked via `pulumi.StackReference` — outputs from one stack flow automatically into the next, so a stack must be deployed before anything depending on it will work.

```
proxmox-infra → k8s-bootstrap → k8s-infra
                              ↘ k8s-apps → monitoring
```

### 1. proxmox-infra

Provisions a 5-node k3s cluster across two Proxmox hosts, each VM a clone of an Ubuntu Noble (24.04) cloud-image template:

| VM | Role |
| --- | --- |
| k3s-master-1 | master |
| k3s-master-2 | master |
| k3s-worker-1 | worker |
| k3s-master-3 | master |
| k3s-worker-2 | worker |

It also generates an ED25519 SSH keypair (stored in Pulumi state, public key injected via cloud-init) and registers each VM's MAC address as a DHCPv4 static mapping in pfSense.

**Providers:** `@muhlba91/pulumi-proxmoxve`, `@pulumi/pfsense` (locally bundled), `@pulumi/tls`

### 2. k8s-bootstrap

Starts the VMs via the Proxmox API and installs k3s over SSH: `k3s-master-1` initializes the cluster (`--cluster-init`, embedded etcd), `k3s-master-2`/`k3s-master-3` join as additional etcd nodes, and the two workers join as agents. Exports the cluster's `kubeconfig` as a secret for downstream stacks.

### 3. k8s-infra

Cluster-level infrastructure via Helm and `@pulumi/kubernetes`:

| Component | What it does |
| --- | --- |
| NFS CSI Driver + `truenas-nfs` StorageClass | Dynamic NFS-backed PVCs from a TrueNAS share |
| Longhorn | Default block storage (`longhorn` StorageClass, 3 replicas) |
| cert-manager + `letsencrypt-prod` ClusterIssuer | TLS certs via Let's Encrypt, DNS-01 through Cloudflare |
| kube-vip + kube-vip-cloud-provider | Announces a virtual IP and allocates it to LoadBalancer Services (e.g. Traefik) |

### 4. k8s-apps

Application workloads exposed at `app.<domain>` via Traefik with a Let's Encrypt cert.

### 5. monitoring

`kube-prometheus-stack` Helm release — Prometheus (20Gi Longhorn PVC) and Grafana (2Gi PVC, ingress at `grafana.<domain>`).

## Prerequisites

- [Pulumi CLI](https://www.pulumi.com/docs/install/), Node.js 24+ and npm
- One or more Proxmox nodes with API tokens
- pfSense (DHCPv4 static mappings)
- TrueNAS with an NFS share (connected directly — no API key needed)
- A domain with DNS managed by Cloudflare (cert-manager DNS-01 + ingress hostnames)
- A self-hosted Pulumi state backend (PostgreSQL connection string)
- Gitea instance for CI/CD (optional for local-only use)

## Getting started

Deploy stacks **in order** — each must be fully `pulumi up`'d before the next stack's `StackReference` resolves. Set the shared backend env vars once per shell:

```bash
export PULUMI_BACKEND_URL=postgresql://<user>:<pass>@<host>/<db>
export PULUMI_CONFIG_PASSPHRASE=<your-passphrase>
```

Then for each stack, in order:

```bash
cd 0N-<stack-dir>
npm install
pulumi stack init dev
pulumi config set ...   # required keys and secrets
pulumi up --yes
```

Stack-specific notes:

1. **proxmox-infra** — set Proxmox API tokens, pfSense credentials, VM credentials/SSH public key, and a static IP per node.
2. **k8s-bootstrap** — set `k3sToken` (e.g. `openssl rand -hex 32`) and a static IP per node. Proxmox credentials and the SSH key come automatically from `proxmox-infra`.
3. **k8s-infra** — before deploying, create an NFS dataset/share on TrueNAS (POSIX ACLs, `root`/`root` maproot, restricted to your k3s nodes). Set the TrueNAS, Cloudflare, and `kubeVipAddress`/`kubeVipInterface` config.
4. **k8s-apps** — set `domain`, the base domain used for ingress hostnames.
5. **monitoring** — set `grafanaAdminPassword`.

## CI/CD (Gitea Actions)

One workflow per stack, `.gitea/workflows/0N-deploy-<stack>.yaml`, scoped to changes under that stack's directory.

| Trigger | Stacks | Action |
| --- | --- | --- |
| PR → `main` | all | `pulumi preview` (read-only) |
| Push to `main` | 03–05 | `pulumi refresh` + `pulumi up` |
| Manual dispatch | all | `pulumi refresh` + `pulumi up` |

Stacks 01/02 (VM provisioning, cluster bootstrap) deploy **only** via manual dispatch — they're riskier to automate.

### Required secrets

| Secret | Used by |
| --- | --- |
| `PULUMI_BACKEND_URL`, `PULUMI_CONFIG_PASSPHRASE` | all workflows |
| `PROXMOX_INFRA_PULUMI_DEV_YAML` | 01-deploy-proxmox-infra |
| `K8S_BOOTSTRAP_PULUMI_DEV_YAML` | 02-deploy-k8s-bootstrap |
| `K8S_INFRA_PULUMI_DEV_YAML` | 03-deploy-k8s-infra |
| `K8S_APPS_PULUMI_DEV_YAML` | 04-deploy-k8s-apps |
| `MONITORING_PULUMI_DEV_YAML` | 05-deploy-monitoring |

Each `<STACK>_PULUMI_DEV_YAML` secret is a base64-encoded copy of that stack's gitignored `Pulumi.dev.yaml`. After changing local config, re-encode and update the secret:

```bash
base64 -w 0 0N-<stack>/Pulumi.dev.yaml
```

## Using this as a template

1. Fork or copy the repo
2. Update Proxmox node names/datastores and the `nodeConfigs` array in `01-proxmox-infra/index.ts` for your VM topology
3. Set your own config/secrets per stack with `pulumi config set [--secret]` — see each stack's `CLAUDE.md` for the full list of required keys
4. Point the CI/CD workflows at your own Gitea/GitOps instance and update the secrets above

## Roadmap

- Migrate secrets management to [OpenBao](https://openbao.org/) — replace `PULUMI_CONFIG_PASSPHRASE` and manual `Pulumi.dev.yaml` encoding with a self-hosted vault
- Add a third bare-metal Proxmox node for true 3-node HA parity