# Testbook: Provision Pipeline

Verifies the full provision pipeline: `tap.hetzner.provision` event through to a
running forest with grove dashboard on `{slug}.mynimsforest.com`.

Scope starts at the NATS tap event. Stripe checkout flow is tested separately
in the nimsforestecommerce testbook (which should end by verifying the tap event
is dropped).

## Prerequisites

- SSH access to land server (46.225.164.179)
- `hcloud` CLI with `nimsforest` context
- NATS CLI on land server

## Test Slug

Use `acme` / "Acme Corp" throughout.

---

## Steps

### 1. Pre-flight cleanup

```bash
export HCLOUD_CONTEXT=nimsforest

# Check for leftover test server
hcloud server list --selector org=acme
# If exists, tear down first:
ssh root@46.225.164.179 "nats pub tap.hetzner.teardown '{\"org_slug\":\"acme\"}'"
# Wait for teardown to complete, then verify:
hcloud server list --selector org=acme
# Expected: empty
```

### 2. Verify treehouse is running

```bash
ssh root@178.104.70.180 "docker logs hetznertreehouse --tail 5"
# Expected: "hetznertreehouse v0.2.0 ready (catching tap.hetzner.*)"
```

### 3. Trigger provisioning

```bash
ssh root@46.225.164.179 "nats pub tap.hetzner.provision '{\"org_slug\":\"acme\",\"org_name\":\"Acme Corp\"}'"
```

### 4. Watch provisioning (~2-3 minutes)

```bash
ssh root@178.104.70.180 "docker logs -f hetznertreehouse"
# Expected log sequence:
#   [Provision] starting for acme
#   [Provision] server land-acme created: ip=X.X.X.X id=NNNNN
#   [DNS] created acme.mynimsforest.com → X.X.X.X
#   [DNS] created *.acme.mynimsforest.com → X.X.X.X
#   [Provision] land-acme is healthy!
```

### 5. Verify Hetzner server

```bash
export HCLOUD_CONTEXT=nimsforest
hcloud server list --selector org=acme
# Expected: land-acme, running, with IP
```

### 6. Verify DNS records

```bash
export HCLOUD_CONTEXT=nimsforest
hcloud zone rrset list mynimsforest.com --type A | grep acme
# Expected:
#   acme    A    <server-ip>
#   *.acme  A    <server-ip>

# Verify resolution (may take a few minutes for propagation)
dig +short acme.mynimsforest.com
# Expected: <server-ip>
```

### 7. Verify cloud-init config files

```bash
SERVER_IP=$(hcloud server list --selector org=acme -o columns=ipv4 -o noheader)

ssh root@$SERVER_IP "cat /etc/land.yaml"
# Expected:
#   registry: registry.nimsforest.com
#   proxy.base_domain: mynimsforest.com
#   containers: nimsforest + nimsforestgrove
#   nimsforestgrove domains: acme.mynimsforest.com

ssh root@$SERVER_IP "cat /opt/nimsforest/config/forest.yaml"
# Expected:
#   organization.slug: acme
#   organization.name: "Acme Corp"

ssh root@$SERVER_IP "cat /opt/nimsforestgrove/config.yaml"
# Expected:
#   organization.name: "Acme Corp"
#   server.listen: ":8091"
#   nats.url: "nats://127.0.0.1:4222"
```

### 8. Verify Land is running

```bash
ssh root@$SERVER_IP "systemctl status land"
# Expected: active (running), ExecStart includes --config /etc/land.yaml

ssh root@$SERVER_IP "curl -s http://localhost:8080/health"
# Expected: healthy response
```

### 9. Verify containers are running

```bash
ssh root@$SERVER_IP "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Image}}'"
# Expected: nimsforest and nimsforestgrove containers running
```

### 10. Verify grove dashboard via HTTPS

```bash
# TLS cert provisioning may take up to 1 minute
curl -s https://acme.mynimsforest.com/ | head -20
# Expected: HTML with grove dashboard

curl -s https://acme.mynimsforest.com/api/v1/health
# Expected: {"status":"ok"}
```

### 11. Verify org name in grove

```bash
curl -s https://acme.mynimsforest.com/ | grep -i "Acme Corp"
# Expected: org name appears in dashboard HTML
```

### 12. Verify landregistry entry

```bash
ssh root@46.225.164.179 "curl -s -H 'Authorization: Bearer bce55fc30d525368050c0da813e32a2ab04d79baeb3fb7618f6cbff198b0373a' http://localhost:8096/api/v1/lands" | python3 -m json.tool
# Expected: entry for acme with status "provisioned", correct IP
```

### 13. Teardown

```bash
ssh root@46.225.164.179 "nats pub tap.hetzner.teardown '{\"org_slug\":\"acme\"}'"

# Watch logs:
ssh root@178.104.70.180 "docker logs -f hetznertreehouse"
# Expected:
#   [Teardown] starting for acme
#   [DNS] deleted acme.mynimsforest.com
#   [DNS] deleted *.acme.mynimsforest.com
#   [Teardown] server land-acme deleted
```

### 14. Verify cleanup

```bash
export HCLOUD_CONTEXT=nimsforest
hcloud server list --selector org=acme
# Expected: empty

hcloud zone rrset list mynimsforest.com --type A | grep acme
# Expected: empty

dig +short acme.mynimsforest.com
# Expected: empty (after TTL expires)
```

---

## Pass Criteria

- [ ] Server `land-acme` created with `org=acme` label
- [ ] DNS A records created on `mynimsforest.com` zone (acme + *.acme)
- [ ] `/etc/land.yaml` has proxy config with `base_domain: mynimsforest.com`
- [ ] `/etc/land.yaml` has nimsforest + nimsforestgrove containers
- [ ] nimsforestgrove domain is `acme.mynimsforest.com`
- [ ] `/opt/nimsforest/config/forest.yaml` has org slug and name
- [ ] `/opt/nimsforestgrove/config.yaml` has org name and local NATS
- [ ] Land running with `--config /etc/land.yaml`
- [ ] Both containers pulled from registry and running
- [ ] `https://acme.mynimsforest.com/` returns grove HTML with "Acme Corp"
- [ ] `https://acme.mynimsforest.com/api/v1/health` returns `{"status":"ok"}`
- [ ] Landregistry has entry for acme
- [ ] Teardown deletes server, DNS records, and updates landregistry