Files
documentation/fleet/admin-cheat-sheet.md

270 lines
9.0 KiB
Markdown

# Iron Legion Fleet Admin Cheat Sheet
**Generated:** 2026-05-31
**Maintainer:** F.R.I.D.A.Y. (Hermes Agent)
---
## Quick Access Links
| Service | URL / Endpoint | Notes |
|---------|---------------|-------|
| iVentoy PXE Server | http://192.168.27.205:26000 | Shield WiFi fallback |
| PegaProx | https://192.168.7.7:5000 | PVE Cluster Manager (host mode) |
| Portainer | https://portainer.ai.home | Swarm Manager |
| Traefik Dashboard | https://traefik.ai.home:8080 | Proxy/Router |
| Technitium DNS | https://dns.ai.home:5380 | DNS Server |
| Beszel Monitoring | https://beszel.ai.home | Fleet Metrics |
| Dozzle | https://dozzle.ai.home | Container Logs |
| Homepage | https://home.ai.home | Service Portal |
| Prometheus | https://prometheus.ai.home | Metrics DB |
| Authelia | https://auth.ai.home | SSO Portal |
| Trilium (ZimaOS) | https://trilium.nb.mslnath.me | Personal Knowledge Base |
---
## Standalone Nodes (No Ansible)
|| Hostname | LAN IP | Domain | Role | Beszel | NetBird Domain |
||----------|--------|--------|------|--------|
| igor (MK-38) | 192.168.10.211 | — | ZimaOS NAS (Ugreen DXP4800, 30TB) | NetBird: mslnath.me | — |
| MK-46 (Homecoming) | 192.168.26.130 | trilium.nb.mslnath.me | ZimaOS, Trilium, ARR Media Stack | ✅ | mslnath.me |
---
## Fleet Node Inventory
### Swarm Manager
- Hostname: mk7.ai.home
- Armor Code: MK-7
- LAN IP: 192.168.7.7
- Tailscale IP: 100.66.70.51
- Role: Swarm Manager, Technitium DNS, Traefik, Portainer, PegaProx
- CPUs: 18 | RAM: 15 GB | Disk: 916 GB
### Worker Nodes G9 (Proxmox VE)
| Armor | Name | Hostname | LAN IP | Tailscale IP | MAC | Status |
|-------|------|----------|--------|--------------|-----|--------|
| MK-33 | Silver Centurion | mk33.ai.home | 192.168.7.33 | 100.125.155.41 | E0-51-D8-1C-5D-56 | Online (PVE) |
| MK-34 | Southpaw | mk34.ai.home | 192.168.7.34 | 100.94.190.43 | E0-51-D8-1C-5C-75 | Online (PVE) |
| MK-39 | Gemini | mk39.ai.home | 192.168.7.39 | 100.125.155.41 | PENDING | Online (PVE) |
| MK-42 | Extremis | mk42.ai.home | 192.168.7.42 | TBD | PENDING | Offline (not installed) |
### Utility Nodes
| Hostname | LAN IP | Tailscale IP | Role |
|----------|--------|--------------|------|
| nebuchadnezzar.ai.home | 192.168.192.24 | 100.99.123.16 | Nextcloud AIO, Gitea, Git server | NetBird: bobbysh.me |
| mark44.ai.home | 192.168.5.214 | TBD | Ollama GPU |
| mark5.ai.home | 192.168.6.5 | TBD | TBD |
| shield.ai.home | 192.168.10.15 | - | iVentoy PXE Server |
| artemis.ai.home | 192.168.15.182 | 100.100.97.18 | Discord Gateway |
| igor.ai.home | 192.168.10.211 | TBD | ZimaOS NAS (Ugreen DXP4800, 30TB) |
> **Note:** `igor.ai.home` is a separate physical node (ZimaOS NAS). Do NOT confuse with any armor codename.
### Mission Control
- Hostname: mission-control.ai.home
- OS: Windows 11
- Role: Workstation
- Type: Separate physical machine
- Tailscale IP: 100.96.128.121
### Portable Agent Host
- Hostname: cinnamint.ai.home (inferred)
- Role: Hermes Agent USB-portable host
- Tailscale IP: 100.99.65.75
---
## DNS Configuration
**Primary Authoritative DNS:** MK7 (Technitium)
- LAN: 192.168.7.7
- Tailscale: 100.66.70.51
- Web UI: http://dns.ai.home:5380
**Technitium Upstream Forwarder:** tls://1.1.1.1 (Cloudflare DoT)
- Fallback: tls://1.0.0.1
**Fleet Node DNS Fallbacks** (for /etc/resolv.conf when not using DNS proxy):
- Primary: 192.168.7.7 (Technitium)
- Secondary: 192.168.18.1 (Router / Gateway DNS)
- Tertiary: 1.1.1.1 (Cloudflare)
**Internal Domain:** `*.ai.home` — authoritative on Technitium, also via Tailscale MagicDNS split-brain.
---
## PegaProx — Proxmox VE Cluster Manager
| Attribute | Value |
|-----------|-------|
| **Host** | MK7 (192.168.7.7) |
| **Ports** | 5000 (HTTPS UI/API), 5001 (VNC WebSocket), 5002 (SSH WebSocket) |
| **Deploy mode** | Docker Swarm — `host` publish mode |
| **Network** | `traefik-public` overlay |
| **SSL** | Self-signed cert (`CN=PegaProx`, auto-generated) |
| **Default user** | `pegaprox` (password change required on first login) |
| **Cluster IDs** | MK33=`726eb477`, MK34=`df6f5e5d`, MK39=`9711704b` |
**Admin password must be changed on first login.**
**API notes:**
- Add cluster: `host` field must be **bare IP only** (no `:8006` — PegaProx appends port internally)
- CSRF protection requires `X-Requested-With: XMLHttpRequest` on state-changing API calls
- Exempt paths: `/api/auth/login`, `/api/auth/setup`, `/api/health`
---
## iVentoy PXE Configuration
- Server: shield.ai.home — 192.168.10.15/27
- WebUI: http://192.168.27.205:26000
- Subnet: 192.168.10.0/27
- Pool: 192.168.10.20 to 192.168.10.30
- MAC Filter: Permit mode
- Edition: **iVentoy Free** (Pro upgrade pending — private repo link awaited)
### Registered ISOs
| ISO | Node | Purpose |
|-----|------|---------|
| proxmox-mk33-auto.iso | MK-33 | PVE 9.2 Auto-Install |
| proxmox-mk34-auto.iso | MK-34 | PVE 9.2 Auto-Install |
| proxmox-mk39-auto.iso | MK-39 | PVE 9.2 Auto-Install |
| proxmox-mk42-auto.iso | MK-42 | PVE 9.2 Auto-Install |
| proxmox-ve_9.2-1.iso | - | Original PVE ISO |
| ubuntu-24.04.3-live-server-amd64.iso | - | Ubuntu Autoinstall |
### Whitelisted MACs
- E0-51-D8-1C-5D-CA (Legacy)
- E0-51-D8-1C-5D-5C (Legacy)
- E0-51-D8-1C-5D-56 (MK-33)
- E0-51-D8-1C-5C-75 (MK-34)
- PENDING: MK-39
- PENDING: MK-42
Post-Install: Remove MAC from whitelist. Node boots local disk, gets production IP.
### ISO Remastering Notes
All Proxmox auto-install ISOs are **remastered** with:
1. **Embedded answer URL** — each ISO points to `http://192.168.10.15:8080/pve/answers/mkNN.toml` (server URL hardcoded; node IP assigned by DHCP)
2. **UEFI gfxmode locked** — strict `1024x768` (fallback `640x480` removed)
3. **Per-ISO answer files**`mk33.toml`, `mk34.toml`, `mk39.toml`, `mk42.toml` in `/opt/iventoy/user/answers/`
> iVentoy Free does NOT support per-MAC ISO binding. Remastered ISOs achieve per-node provisioning via embedded answer URLs.
---
## DNS Records
### CNAME to traefik.ai.home — A: 192.168.7.7
- artemis.ai.home
- hermes.ai.home
- n8n.ai.home
- pgadmin.ai.home
- portainer.ai.home
- beszel.ai.home
- dozzle.ai.home
- prometheus.ai.home
- homepage.ai.home
- auth.ai.home
- dns.ai.home
### A Records
| Record | IP |
|--------|-----|
| traefik.ai.home | 192.168.7.7 |
| mk7.ai.home | 192.168.7.7 |
| mk33.ai.home | 192.168.7.33 |
| mk34.ai.home | 192.168.7.34 |
| mk39.ai.home | 192.168.7.39 |
| mk42.ai.home | 192.168.7.42 |
| mark44.ai.home | 192.168.5.214 |
| mark5.ai.home | 192.168.6.5 |
| nebuchadnezzar.ai.home | 192.168.192.24 |
| shield.ai.home | 192.168.10.15 |
| artemis.ai.home | 192.168.15.182 |
| igor.ai.home | 192.168.10.211 |
---
## SSH Topology
```
Portable Host (F.R.I.D.A.Y.)
|
+---> artemis.ai.home via id_ed25519
| +---> mk7.ai.home via artemis_key
|
+---> shield via jarvis user
| +---> PXE subnet 192.168.10.0/27
|
+---> nebuchadnezzar via jarvis user
|
+---> mk33-42 via root (key-based, id_ed25519)
```
**Key Files:**
- `~/.ssh/id_ed25519` — bobby@cinnamint, also injected as `friday@hermes` into PVE nodes
- `~/.ssh/artemis_key` — MK7 jump-host
---
## Armor Codenames
| Code | Name | System |
|------|------|--------|
| MK-7 | Mark VII | Swarm Manager |
| MK-33 | Silver Centurion | PVE Worker |
| MK-34 | Southpaw | PVE Worker |
| MK-39 | Gemini | PVE Worker |
| MK-42 | Extremis | PVE Worker (offline) |
| MK-44 | Hulkbuster | GPU/Ollama |
| MK-5 | Mark 5 | TBD |
| MK-38 | Igor | ZimaOS NAS (Ugreen DXP4800, 30TB) |
| MK-46 | Homecoming | ZimaOS, Trilium, ARR Media Stack |
| J.A.R.V.I.S. | Judicious Automated... | Dashboard |
| F.R.I.D.A.Y. | Field-Ready Runtime... | Portable Agent |
| A.R.T.E.M.I.S. | Advanced Real-Time... | Discord Gateway |
| NEO | Nebuchadnezzar | Nextcloud/Gitea |
| SHIELD | - | PXE Server |
> **Note:** `Igor` is **MK-38** (ZimaOS NAS at 192.168.10.211 — Ugreen DXP4800, 30TB). It is NOT MK-34.
---
## Notes
- iVentoy Free does NOT support per-MAC ISO binding.
- Shield PXE subnet isolated via ip_forward=0. Canonical wired IP: 192.168.10.15/27.
- Shield live state may show 192.168.128.33/27 from DHCP/cloud-init drift — canonical config is source-of-truth.
- Mission Control is a separate physical machine — reserved hostname must NOT be used for DNS aliases or services.
- All `*.ai.home` resolve via Technitium DNS (192.168.7.7).
- PegaProx deployed on MK7 Swarm in `host` mode (not routed through Traefik).
- iVentoy Pro upgrade pending — private repo link awaited from vendor.
- Gitea: `gitea.nb.bobbysh.me` (ssh://100.99.123.16:2222).
- Hermes portable sessions on Artemis use `HOME=/home/bobby/1/Hermes-USB-Portable-main/.cache/unix-home`.
- Bobby's SSH config on the portable host lives at `/home/bobby/.ssh/config` and uses `ts-` prefix for Tailscale IP aliases. Fleet aliases are primary LAN, Tailscale fallback.
---
## DNS Reminders
| Context | Primary | Fallback | Notes |
|---------|---------|----------|-------|
| PVE nodes /etc/resolv.conf | 192.168.7.7 | 192.168.18.1, 1.1.1.1 | Technitium internal |
| Technitium forwarder | tls://1.1.1.1 | tls://1.0.0.1 | Cloudflare DoT |
| Router default | Cloudflare 1.1.1.1 | — | For non-fleet devices |
Last updated: 2026-05-31 by F.R.I.D.A.Y.