42 lines
2.5 KiB
Markdown
42 lines
2.5 KiB
Markdown
# k8s-bootstrap
|
|
|
|
This file provides guidance to Claude Code when working in `02-k8s-bootstrap`. See the [repo-root CLAUDE.md](../CLAUDE.md) for shared conventions, deployment order, and CI/CD.
|
|
|
|
Bootstraps a k3s cluster on the Proxmox VMs created by `proxmox-infra`. Starts VMs via the Proxmox REST API, then provisions k3s over SSH using `@pulumi/command` `remote.Command`.
|
|
|
|
## How it works
|
|
|
|
1. Starts all VMs via `POST /api2/json/nodes/{node}/qemu/{vmid}/status/start`
|
|
2. Waits for port 22 to open on each VM (bash `/dev/tcp`)
|
|
3. Installs k3s on `k3s-master-1` with `--cluster-init --tls-san <master1Ip>`
|
|
4. Joins `k3s-master-2` and `k3s-master-3` as embedded etcd nodes
|
|
5. Joins `k3s-worker-1` and `k3s-worker-2` as agent nodes
|
|
6. Formats the workers' dedicated `scsi1` disk (ext4, if not already formatted) and mounts it at `/mnt/longhorn-extra` (via `/etc/fstab`) — this is the disk `03-k8s-infra` later registers as an extra Longhorn disk, to keep Longhorn off the root filesystem and avoid `DiskPressure`
|
|
7. Reads `/etc/rancher/k3s/k3s.yaml` from master-1 via SSH, patches the server URL, and exports it as the secret stack output `kubeconfig`
|
|
|
|
VM IDs and the CI runner SSH private key are read automatically from the `proxmox-infra` stack output via StackReference — no manual setup needed for those.
|
|
|
|
## Required Pulumi config
|
|
|
|
```bash
|
|
# Pre-shared k3s token — any strong random string
|
|
pulumi config set --secret k3sToken "$(openssl rand -hex 32)"
|
|
|
|
# Node IPs — static DHCP leases from the router (not secrets)
|
|
pulumi config set master1Ip "192.168.1.x"
|
|
pulumi config set master2Ip "192.168.1.x"
|
|
pulumi config set master3Ip "192.168.1.x"
|
|
pulumi config set worker1Ip "192.168.1.x"
|
|
pulumi config set worker2Ip "192.168.1.x"
|
|
```
|
|
|
|
Proxmox credentials (`pve1Endpoint`, `pve1ApiToken`, `pve2Endpoint`, `pve2ApiToken`) are read automatically from `proxmox-infra` via StackReference — do **not** set them here.
|
|
|
|
## Prerequisites
|
|
|
|
`proxmox-infra` must be deployed first. On its first `pulumi up` it generates an ed25519 SSH keypair via `@pulumi/tls`, writes the public key into every VM's cloud-init, and exports the private key as a secret stack output. VMs must be re-provisioned (or have the public key manually added to `~/.ssh/authorized_keys`) before this stack can SSH in.
|
|
|
|
## CI/CD
|
|
|
|
Workflow: `../.gitea/workflows/02-deploy-k8s-bootstrap.yaml`. Secret: `K8S_BOOTSTRAP_PULUMI_DEV_YAML`. Deploy is manual (`workflow_dispatch`) — re-running this stack restarts the bootstrap sequence against live VMs.
|