docs(fleet): add PegaProx, iVentoy remastering procedures, update admin cheat sheet
- fleet/admin-cheat-sheet.md: Added PegaProx section, updated MK33/MK34/MK39 statuses to Online (PVE), added iVentoy remastering notes, iVentoy Pro upgrade pending marker. - procedures/pega-prox-deploy.md: New procedure for deploying PegaProx on Docker Swarm (host mode, CSRF, API gotchas). - procedures/iventoy-remaster-procedure.md: New procedure for remastering Proxmox ISOs with embedded answer URLs and locked gfxmode. - changelog/2026-05-31-pxe-pegaprox-deployment.md: Changelog entry for todays fleet work. - 04-service-catalog.md: Added PegaProx to Management / Dashboard section.
This commit is contained in:
F.R.I.D.A.Y.
238
procedures/iventoy-remaster-procedure.md
Normal file
238
procedures/iventoy-remaster-procedure.md
Normal file
@@ -0,0 +1,238 @@
|
||||
# Procedure: Remaster Proxmox VE ISOs for iVentoy Auto-Install
|
||||
|
||||
**Scope:** Remaster stock Proxmox VE ISOs with embedded auto-install answer URLs and locked UEFI gfxmode for PXE boot via iVentoy.
|
||||
**Author:** F.R.I.D.A.Y.
|
||||
**Date:** 2026-05-31
|
||||
**Prerequisites:** Stock Proxmox VE ISO, `xorriso`, Python 3, iVentoy PXE server running.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
iVentoy Free does NOT support per-MAC ISO binding. To provision each node with its own network config (IP, gateway, etc.), we remaster the stock Proxmox ISO:
|
||||
|
||||
1. Embed an `auto-installer-mode.toml` file pointing to a per-node answer file
|
||||
2. Lock UEFI `gfxmode` to `1024x768` (remove `640x480` fallback)
|
||||
3. Each ISO points to its own answer URL: `http://192.168.10.15:8080/pve/answers/mkNN.toml`
|
||||
|
||||
---
|
||||
|
||||
## 2. Answer File Structure
|
||||
|
||||
### iVentoy Answer Server
|
||||
|
||||
iVentoy runs a built-in HTTP server on `192.168.10.15:8080`. Answer files live in:
|
||||
```
|
||||
/opt/iventoy/user/answers/
|
||||
├── mk33.toml
|
||||
├── mk34.toml
|
||||
├── mk39.toml
|
||||
└── mk42.toml
|
||||
```
|
||||
|
||||
### Per-Node Answer File Example (`mk33.toml`)
|
||||
|
||||
```toml
|
||||
[target]
|
||||
source = "from-dhcp" # Node IP assigned by iVentoy DHCP, NOT hardcoded
|
||||
|
||||
global]
|
||||
keyboard = "en-us"
|
||||
timezone = "America/Toronto"
|
||||
|
||||
[network]
|
||||
iface = "eno1"
|
||||
address = "192.168.7.33/18" # Static after install
|
||||
gateway = "192.168.18.1"
|
||||
dns = "192.168.7.7"
|
||||
|
||||
[root-password]
|
||||
pwhash = "$y$j9T$YOUR_HASH_HERE" # Pre-hashed password
|
||||
```
|
||||
|
||||
> **Important:** The `answer_url` in the embedded `auto-installer-mode.toml` points to the **server** (`192.168.10.15:8080`), not the node IP. The node IP comes from DHCP during PXE boot (`source = "from-dhcp"`).
|
||||
|
||||
---
|
||||
|
||||
## 3. Remaster Script
|
||||
|
||||
Save as `/tmp/remaster_pve_iso.py`:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Remaster Proxmox VE ISO with embedded auto-install answer URL.
|
||||
Locks UEFI gfxmode to 1024x768 (removes 640x480 fallback).
|
||||
"""
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import os
|
||||
import shutil
|
||||
|
||||
# Node-specific config
|
||||
NODE = sys.argv[1] # e.g., mk33
|
||||
SRC_ISO = sys.argv[2] # e.g., proxmox-ve_9.2-1.iso
|
||||
DST_ISO = f"proxmox-{NODE}-auto.iso"
|
||||
ANSWER_URL = f"http://192.168.10.15:8080/pve/answers/{NODE}.toml"
|
||||
|
||||
# Create auto-installer-mode.toml
|
||||
auto_installer_toml = f"""[proxmox-auto-installer]
|
||||
answer_url = "{ANSWER_URL}"
|
||||
"""
|
||||
|
||||
# Work in temp dir
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
# Extract ISO contents
|
||||
subprocess.run(["xorriso", "-osirrox", "on", "-indev", SRC_ISO,
|
||||
"-extract", "/", tmpdir], check=True)
|
||||
|
||||
# Write auto-installer-mode.toml into ISO root
|
||||
ai_path = os.path.join(tmpdir, "auto-installer-mode.toml")
|
||||
with open(ai_path, "w") as f:
|
||||
f.write(auto_installer_toml)
|
||||
|
||||
# Patch grub.cfg: lock gfxmode to 1024x768 only
|
||||
grub_path = os.path.join(tmpdir, "boot", "grub", "grub.cfg")
|
||||
if os.path.exists(grub_path):
|
||||
with open(grub_path, "r") as f:
|
||||
content = f.read()
|
||||
# Remove 640x480 fallback
|
||||
content = content.replace("set gfxmode=1024x768,640x480",
|
||||
"set gfxmode=1024x768")
|
||||
with open(grub_path, "w") as f:
|
||||
f.write(content)
|
||||
print("Patched grub.cfg: gfxmode locked to 1024x768")
|
||||
|
||||
# Rebuild ISO with same boot properties
|
||||
subprocess.run([
|
||||
"xorriso", "-as", "mkisofs",
|
||||
"-o", DST_ISO,
|
||||
"-isohybrid-mbr", os.path.join(tmpdir, "usr", "lib", "ISOLINUX", "isohdpfx.bin"),
|
||||
"-c", "boot.cat",
|
||||
"-b", "isolinux/isolinux.bin",
|
||||
"-no-emul-boot", "-boot-load-size", "4", "-boot-info-table",
|
||||
"-eltorito-alt-boot",
|
||||
"-e", "EFI/BOOT/BOOTX64.EFI",
|
||||
"-no-emul-boot", "-isohybrid-gpt-basdat",
|
||||
"-r", "-V", f"Proxmox-VE-Auto-{NODE}",
|
||||
tmpdir
|
||||
], check=True)
|
||||
|
||||
print(f"Created: {DST_ISO}")
|
||||
print(f"Answer URL embedded: {ANSWER_URL}")
|
||||
```
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# On Shield (iVentoy server)
|
||||
python3 /tmp/remaster_pve_iso.py mk33 /opt/iventoy/iso/proxmox-ve_9.2-1.iso
|
||||
python3 /tmp/remaster_pve_iso.py mk34 /opt/iventoy/iso/proxmox-ve_9.2-1.iso
|
||||
python3 /tmp/remaster_pve_iso.py mk39 /opt/iventoy/iso/proxmox-ve_9.2-1.iso
|
||||
python3 /tmp/remaster_pve_iso.py mk42 /opt/iventoy/iso/proxmox-ve_9.2-1.iso
|
||||
|
||||
# Move to iVentoy ISO directory
|
||||
mv proxmox-mk33-auto.iso /opt/iventoy/iso/
|
||||
mv proxmox-mk34-auto.iso /opt/iventoy/iso/
|
||||
mv proxmox-mk39-auto.iso /opt/iventoy/iso/
|
||||
mv proxmox-mk42-auto.iso /opt/iventoy/iso/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. In-Place ISO Patching (gfxmode only)
|
||||
|
||||
If you already have remastered ISOs and only need to patch gfxmode:
|
||||
|
||||
```bash
|
||||
# Extract grub.cfg from ISO, patch, replace in-place
|
||||
ISO=/opt/iventoy/iso/proxmox-mk33-auto.iso
|
||||
xorriso -cpx /boot/grub/grub.cfg /tmp/grub.cfg -< /dev/null -- "$ISO"
|
||||
sed -i 's/set gfxmode=1024x768,640x480/set gfxmode=1024x768/' /tmp/grub.cfg
|
||||
xorriso -boot_image any replay -map /tmp/grub.cfg /boot/grub/grub.cfg -- "$ISO"
|
||||
```
|
||||
|
||||
> The `-boot_image any replay` flag preserves boot properties after file replacement.
|
||||
|
||||
---
|
||||
|
||||
## 5. Verification
|
||||
|
||||
```bash
|
||||
# Confirm answer URL is embedded
|
||||
strings /opt/iventoy/iso/proxmox-mk33-auto.iso | grep "192.168.10.15"
|
||||
# Expected: http://192.168.10.15:8080/pve/answers/mk33.toml
|
||||
|
||||
# Confirm gfxmode is locked
|
||||
xorriso -cpx /boot/grub/grub.cfg /tmp/verify.cfg -< /dev/null -- /opt/iventoy/iso/proxmox-mk33-auto.iso
|
||||
grep gfxmode /tmp/verify.cfg
|
||||
# Expected: set gfxmode=1024x768
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. iVentoy Configuration
|
||||
|
||||
### Web UI
|
||||
- URL: `http://192.168.27.205:26000`
|
||||
- Go to **ISO Management** → add remastered ISOs
|
||||
|
||||
### MAC Whitelist (Permit Mode)
|
||||
Add node MACs to iVentoy whitelist:
|
||||
```
|
||||
E0-51-D8-1C-5D-56 (MK-33)
|
||||
E0-51-D8-1C-5C-75 (MK-34)
|
||||
PENDING (MK-39)
|
||||
PENDING (MK-42)
|
||||
```
|
||||
|
||||
Nodes must be in whitelist to PXE boot.
|
||||
|
||||
### DHCP Pool
|
||||
- Subnet: `192.168.10.0/27`
|
||||
- Range: `192.168.10.20` to `192.168.10.30`
|
||||
- Nodes get temporary PXE IPs from this pool during install
|
||||
|
||||
---
|
||||
|
||||
## 7. Post-Install
|
||||
|
||||
After node installs and reboots:
|
||||
1. Remove node MAC from iVentoy whitelist (node boots from local disk)
|
||||
2. Node gets production IP from `/etc/network/interfaces` (set in answer file)
|
||||
3. Verify: `ping 192.168.7.33` (or appropriate node IP)
|
||||
|
||||
---
|
||||
|
||||
## 8. iVentoy Pro Upgrade Notes
|
||||
|
||||
> **Status:** Awaiting private repo link from vendor.
|
||||
|
||||
Expected Pro features (to verify upon upgrade):
|
||||
- Per-MAC ISO binding (may eliminate need for per-node remastered ISOs)
|
||||
- Additional deployment modes
|
||||
- Priority support
|
||||
|
||||
When the private repo link is received:
|
||||
1. Clone the Pro repository
|
||||
2. Review upgrade documentation in the repo
|
||||
3. Backup current `/opt/iventoy/` configuration
|
||||
4. Follow vendor upgrade procedure
|
||||
5. Test with one node before fleet-wide rollout
|
||||
|
||||
---
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
# Remove remastered ISO
|
||||
rm /opt/iventoy/iso/proxmox-mk33-auto.iso
|
||||
|
||||
# Re-add stock ISO in iVentoy Web UI
|
||||
# Node will boot stock ISO -- manual install required
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-05-31*
|
||||
165
procedures/pega-prox-deploy.md
Normal file
165
procedures/pega-prox-deploy.md
Normal file
@@ -0,0 +1,165 @@
|
||||
# Procedure: Deploy PegaProx on Docker Swarm
|
||||
|
||||
**Scope:** Deploy PegaProx (Proxmox VE cluster manager) as a Docker Swarm service on MK7.
|
||||
**Author:** F.R.I.D.A.Y.
|
||||
**Date:** 2026-05-31
|
||||
**Prerequisites:** MK7 Swarm manager active, `traefik-public` overlay network exists.
|
||||
|
||||
---
|
||||
|
||||
## 1. Create Swarm Compose File
|
||||
|
||||
Save as `/tmp/pegaprox_swarm.yml` on MK7:
|
||||
|
||||
```yaml
|
||||
version: "3.8"
|
||||
services:
|
||||
pegaprox:
|
||||
image: pegaprox/pegaprox:latest
|
||||
deploy:
|
||||
mode: replicated
|
||||
replicas: 1
|
||||
placement:
|
||||
constraints:
|
||||
- node.role == manager
|
||||
ports:
|
||||
- target: 5000
|
||||
published: 5000
|
||||
mode: host
|
||||
protocol: tcp
|
||||
- target: 5001
|
||||
published: 5001
|
||||
mode: host
|
||||
protocol: tcp
|
||||
- target: 5002
|
||||
published: 5002
|
||||
mode: host
|
||||
protocol: tcp
|
||||
networks:
|
||||
- traefik-public
|
||||
volumes:
|
||||
- pegaprox-config:/app/config
|
||||
environment:
|
||||
- PEGAPROX_DEBUG=0
|
||||
|
||||
volumes:
|
||||
pegaprox-config:
|
||||
driver: local
|
||||
|
||||
networks:
|
||||
traefik-public:
|
||||
external: true
|
||||
```
|
||||
|
||||
> **Critical:** `mode: host` is required. `ingress` mode breaks WebSocket VNC/SSH consoles because Swarm ingress routing does not support WebSocket upgrade properly.
|
||||
|
||||
---
|
||||
|
||||
## 2. Deploy Stack
|
||||
|
||||
```bash
|
||||
ssh jarvis@mk7.ai.home
|
||||
docker stack deploy -c /tmp/pegaprox_swarm.yml pegaprox
|
||||
```
|
||||
|
||||
Verify:
|
||||
```bash
|
||||
docker service ls | grep pegaprox
|
||||
docker ps | grep pegaprox
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Verify Service Health
|
||||
|
||||
```bash
|
||||
# HTTPS API
|
||||
curl -sk https://192.168.7.7:5000/api/health
|
||||
|
||||
# Check container logs
|
||||
docker logs $(docker ps -q -f name=pegaprox)
|
||||
```
|
||||
|
||||
Expected: `{"status":"ok"}`
|
||||
|
||||
---
|
||||
|
||||
## 4. First Login & Password Change
|
||||
|
||||
1. Open `https://192.168.7.7:5000`
|
||||
2. Login with default credentials:
|
||||
- Username: `pegaprox`
|
||||
- Password: `admin`
|
||||
3. System will force password change on first login
|
||||
4. API returns: `{"security_warning":"DEFAULT_PASSWORD","requires_password_change":true}`
|
||||
|
||||
---
|
||||
|
||||
## 5. API Notes for Automation
|
||||
|
||||
### CSRF Protection
|
||||
All state-changing API calls (POST/PUT/PATCH/DELETE) must include:
|
||||
```
|
||||
X-Requested-With: XMLHttpRequest
|
||||
```
|
||||
|
||||
Exempt paths (no CSRF header needed):
|
||||
- `/api/auth/login`
|
||||
- `/api/auth/setup`
|
||||
- `/api/auth/oidc/*`
|
||||
- `/api/auth/check`
|
||||
- `/api/auth/validate`
|
||||
- `/api/auth/logout`
|
||||
- `/api/health`
|
||||
- `/api/webauthn/auth/begin`
|
||||
|
||||
### Add Cluster
|
||||
```bash
|
||||
curl -sk -X POST https://192.168.7.7:5000/api/clusters \
|
||||
-b cookies.txt \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "X-Requested-With: XMLHttpRequest" \
|
||||
-d '{
|
||||
"name": "MK33",
|
||||
"host": "192.168.7.33",
|
||||
"user": "root@pam",
|
||||
"pass": "YOUR_PVE_PASSWORD"
|
||||
}'
|
||||
```
|
||||
|
||||
> **CRITICAL:** `host` must be **bare IP only**. Do NOT append `:8006`. PegaProx appends the port internally. Supplying `192.168.7.33:8006` causes URL parse failure: `Failed to parse: https://[192.168.7.33:8006]:8006/...`
|
||||
|
||||
---
|
||||
|
||||
## 6. Backup Volume
|
||||
|
||||
```bash
|
||||
# Backup PegaProx config + DB
|
||||
docker run --rm -v pegaprox_pegaprox-config:/src -v /tmp:/dst alpine \
|
||||
tar czf /dst/pegaprox-config-$(date +%Y%m%d).tar.gz -C /src .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Known Issues
|
||||
|
||||
| Issue | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| WebSocket VNC/SSH broken | Swarm `ingress` mode strips upgrade headers | Use `mode: host` |
|
||||
| URL parse error on add-cluster | `:8006` appended to host field | Use bare IP only |
|
||||
| CSRF 403 on API calls | Missing `X-Requested-With` header | Add header to all state-changing calls |
|
||||
| Self-signed cert warning | No CA-signed cert deployed | Accept in browser or deploy custom cert |
|
||||
|
||||
---
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
ssh jarvis@mk7.ai.home
|
||||
docker stack rm pegaprox
|
||||
docker volume rm pegaprox_pegaprox-config # WARNING: destroys all data
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-05-31*
|
||||
Reference in New Issue
Block a user