Add infrastructure packages: terraform-pve, ansible-push, wolfstack-keepalived, terraform-pve-vm
This commit is contained in:
106
artemis-backup-20260614-1920/wolfstack-keepalived/README.md
Normal file
106
artemis-backup-20260614-1920/wolfstack-keepalived/README.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# WolfStack Keepalived HA Setup
|
||||
|
||||
## Files
|
||||
|
||||
| File | Node | Priority | Intended State |
|
||||
|------|------|----------|---------------|
|
||||
| `keepalived-node1-primary.conf` | 192.168.5.214 | 150 | MASTER |
|
||||
| `keepalived-node2-secondary.conf` | `192.168.15.182` | BACKUP | 130 |
|
||||
| `keepalived-node3-tertiary.conf` | 192.168.50.41 | 110 | BACKUP |
|
||||
|
||||
Shared VIP: `192.168.63.254` on interface `eth0`
|
||||
|
||||
## Quick Install (per node)
|
||||
|
||||
```bash
|
||||
sudo apt install keepalived netcat-openbsd -y
|
||||
sudo cp keepalived-nodeX-*.conf /etc/keepalived/keepalived.conf
|
||||
sudo systemctl enable keepalived --now
|
||||
sudo systemctl status keepalived
|
||||
```
|
||||
|
||||
Watch the syslog:
|
||||
```bash
|
||||
sudo journalctl -u keepalived -f
|
||||
```
|
||||
|
||||
## Network Confirmation
|
||||
|
||||
All three nodes (`.5.x`, `.50.x`) and the VIP (`.63.254`) fall within the same `192.168.0.0/18` network (`192.168.0.0` – `192.168.63.255`). This means they share Layer 2/L3 broadcast domain and VRRP unicast failover works cleanly across all nodes — no VLAN bridging or routing tricks required. ✅
|
||||
|
||||
### What Happens In Practice
|
||||
|
||||
| Scenario | Outcome |
|
||||
|----------|---------|
|
||||
| Node 1 dies, Node 2 healthy | VIP moves to Node 2 ✅ |
|
||||
| Node 1 + Node 2 die, Node 3 alive | VIP moves to Node 3 ✅ (clients reach it directly) |
|
||||
| Node 3 is primary but 1 & 2 alive | Node 3 will NOT win because priority 110 < 150 / 130 ✅ |
|
||||
| All 3 nodes lose peer connectivity | **Split-brain risk** — each may claim VIP. Use fencing/STONITH if this is production-critical. |
|
||||
|
||||
## Track Script Explained
|
||||
|
||||
```bash
|
||||
script "/usr/bin/nc -z -w 2 127.0.0.1 8553"
|
||||
```
|
||||
|
||||
This checks whether the **local** WolfStack UI (port 8553) is accepting TCP connections.
|
||||
|
||||
- **If WolfStack crashes / stops** on the MASTER, keepalived subtracts `weight -40` from that node's priority.
|
||||
- Priority drops from 150 → 110, which is below Node 2 (130). Node 2 promotes to MASTER and takes the VIP.
|
||||
- If Node 1 comes back online *and* WolfStack is healthy, its priority returns to 150. The VIP will move back **unless** you keep `nopreempt`.
|
||||
|
||||
### Preempt Behaviour
|
||||
|
||||
| Setting | Behaviour |
|
||||
|---------|-----------|
|
||||
| `nopreempt` present | Once VIP is on a backup node, it stays there even when higher-priority node recovers. Good for stability. Manual failback required. |
|
||||
| `nopreempt` removed | Highest priority node reclaims VIP automatically. Good for deterministic primary/secondary but causes brief flap. |
|
||||
|
||||
## ⚠️ WolfStack-Specific Caveats (from research)
|
||||
|
||||
1. **No native UI HA**: WolfStack's web UI is single-node. If you point browsers at the VIP and Node 2 takes over, **users must re-authenticate** (sessions are local to each node).
|
||||
2. **No reverse-proxy awareness**: The UI doesn't have a "behind proxy" setting. If you terminate SSL on a proxy and forward to `http://backend:8553`, you may hit mixed-content or CSP issues.
|
||||
3. **WebSockets**: The dashboard uses websockets for the terminal and real-time graphs. Make sure your proxy (or keepalived direct-access path) supports websocket upgrades if not hitting the VIP directly.
|
||||
4. **WolfProxy**: WolfStack ships a sibling `WolfProxy` (NGINX-compatible), but that is a **separate** reverse proxy for *other services*, not for making the WolfStack UI highly available.
|
||||
|
||||
## Suggested Topology
|
||||
|
||||
```
|
||||
Clients / Browsers
|
||||
|
|
||||
v
|
||||
+---------------------+
|
||||
| VIP: 192.168.63.254 |
|
||||
| (keepalived floating) |
|
||||
+----------+----------+
|
||||
|
|
||||
+----------------+----------------+
|
||||
| |
|
||||
Node 1 (MASTER) Node 2 (BACKUP)
|
||||
192.168.5.214:8553 192.168.15.182:8553
|
||||
priority 150 priority 130
|
||||
| |
|
||||
WolfStack WolfStack
|
||||
WolfNet Mesh WolfNet Mesh
|
||||
+------------+ 192.168.50.41 +---+
|
||||
Node 3 (BACKUP)
|
||||
priority 110
|
||||
(same VLAN?)
|
||||
```
|
||||
|
||||
## Testing Failover
|
||||
|
||||
1. From any client, `curl -k https://192.168.63.254:8553` should show the WolfStack UI.
|
||||
2. On the MASTER node: `sudo systemctl stop wolfstack` (simulating WolfStack crash).
|
||||
3. Watch `journalctl -u keepalived -f` on Node 2 — it should promote to MASTER within ~6 seconds (3 x 2s intervals).
|
||||
4. `ip addr show eth0` on Node 2 should now show `192.168.63.254/32`.
|
||||
5. Client traffic should continue via the VIP.
|
||||
6. Restart WolfStack on Node 1. If `nopreempt` is enabled, VIP stays on Node 2. If removed, VIP moves back to Node 1.
|
||||
|
||||
## Rollback
|
||||
|
||||
```bash
|
||||
sudo systemctl stop keepalived
|
||||
sudo ip addr del 192.168.63.254 dev eth0 # if needed manually
|
||||
sudo systemctl disable keepalived
|
||||
```
|
||||
114
artemis-backup-20260614-1920/wolfstack-keepalived/deploy.sh
Normal file
114
artemis-backup-20260614-1920/wolfstack-keepalived/deploy.sh
Normal file
@@ -0,0 +1,114 @@
|
||||
#!/bin/bash
|
||||
# ============================================================
|
||||
# WolfStack Keepalived Deploy Script
|
||||
# Usage:
|
||||
# sudo bash deploy.sh [install] [node1-primary|node2-secondary|node3-tertiary]
|
||||
#
|
||||
# Examples:
|
||||
# sudo bash deploy.sh install # install + auto-detect
|
||||
# sudo bash deploy.sh node1-primary # deploy role, no install
|
||||
# sudo bash deploy.sh install node1-primary # install + deploy role
|
||||
# ============================================================
|
||||
|
||||
set -e
|
||||
|
||||
NODE_TYPE=""
|
||||
CONF_DIR="/home/jarvis/wolfstack-keepalived"
|
||||
KEEPALIVED_CONF="/etc/keepalived/keepalived.conf"
|
||||
|
||||
detect_node() {
|
||||
local ip
|
||||
ip=$(hostname -I | tr ' ' '\n' | grep -E '^192\.168\.' | head -n1)
|
||||
[[ -z "$ip" ]] && ip=$(hostname -I | awk '{print $1}')
|
||||
|
||||
case "$ip" in
|
||||
192.168.50.41)
|
||||
NODE_TYPE="node1-primary"
|
||||
;;
|
||||
192.168.5.214)
|
||||
NODE_TYPE="node2-secondary"
|
||||
;;
|
||||
192.168.15.182)
|
||||
NODE_TYPE="node3-tertiary"
|
||||
;;
|
||||
*)
|
||||
echo "ERROR: Could not auto-detect node type from IP: $ip"
|
||||
echo "Please specify manually: node1-primary | node2-secondary | node3-tertiary"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
echo "Detected IP: $ip -> $NODE_TYPE"
|
||||
}
|
||||
|
||||
install_keepalived() {
|
||||
if command -v apt-get &>/dev/null; then
|
||||
echo "Installing keepalived (Debian/Ubuntu)..."
|
||||
sudo apt-get update -qq
|
||||
sudo apt-get install -y keepalived netcat-openbsd
|
||||
elif command -v dnf &>/dev/null; then
|
||||
echo "Installing keepalived (RHEL/Fedora)..."
|
||||
sudo dnf install -y keepalived nc
|
||||
elif command -v zypper &>/dev/null; then
|
||||
echo "Installing keepalived (openSUSE)..."
|
||||
sudo zypper install -y keepalived netcat-openbsd
|
||||
elif command -v pacman &>/dev/null; then
|
||||
echo "Installing keepalived (Arch)..."
|
||||
sudo pacman -Sy --noconfirm keepalived openbsd-netcat
|
||||
else
|
||||
echo "ERROR: Unsupported package manager. Install keepalived manually."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
deploy_config() {
|
||||
local src="$CONF_DIR/keepalived-${NODE_TYPE}.conf"
|
||||
if [[ ! -f "$src" ]]; then
|
||||
echo "ERROR: Config not found: $src"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Deploying $src -> $KEEPALIVED_CONF"
|
||||
sudo mkdir -p /etc/keepalived
|
||||
sudo cp "$src" "$KEEPALIVED_CONF"
|
||||
sudo chmod 644 "$KEEPALIVED_CONF"
|
||||
}
|
||||
|
||||
enable_service() {
|
||||
if systemctl is-active --quiet keepalived 2>/dev/null; then
|
||||
echo "Keepalived already running. Restarting to pick up new config..."
|
||||
sudo systemctl restart keepalived
|
||||
else
|
||||
echo "Enabling and starting keepalived..."
|
||||
sudo systemctl enable keepalived --now
|
||||
fi
|
||||
sleep 1
|
||||
sudo systemctl status keepalived --no-pager
|
||||
}
|
||||
|
||||
# --- main ---
|
||||
|
||||
ARGS=("$@")
|
||||
|
||||
# Check if first arg is 'install'
|
||||
if [[ "${ARGS[0]:-}" == "install" ]]; then
|
||||
install_keepalived
|
||||
ARGS=("${ARGS[@]:1}") # remove 'install' from args
|
||||
fi
|
||||
|
||||
# If a node type arg remains, use it; otherwise auto-detect
|
||||
if [[ -n "${ARGS[0]:-}" ]]; then
|
||||
NODE_TYPE="${ARGS[0]}"
|
||||
echo "Using provided node type: $NODE_TYPE"
|
||||
else
|
||||
detect_node
|
||||
fi
|
||||
|
||||
deploy_config
|
||||
enable_service
|
||||
|
||||
echo ""
|
||||
echo "========================================"
|
||||
echo "Keepalived deployed for $NODE_TYPE"
|
||||
echo "VIP should appear on this node if it is MASTER."
|
||||
echo "Check: ip addr show | grep 192.168.63.254"
|
||||
echo "========================================"
|
||||
@@ -0,0 +1,48 @@
|
||||
# ============================================================
|
||||
# WolfStack Keepalived — PRIMARY (Node 1 / mk50)
|
||||
# Host: 192.168.50.41
|
||||
# VIP : 192.168.63.254
|
||||
# Interface: nic0
|
||||
# ============================================================
|
||||
|
||||
global_defs {
|
||||
router_id WOLFSTACK_NODE1
|
||||
script_security
|
||||
enable_script_security
|
||||
}
|
||||
|
||||
vrrp_script check_wolfstack {
|
||||
script "/usr/bin/nc -z -w 2 127.0.0.1 8553"
|
||||
interval 2
|
||||
timeout 3
|
||||
fall 3
|
||||
rise 2
|
||||
weight -40
|
||||
}
|
||||
|
||||
vrrp_instance VI_WOLFSTACK {
|
||||
state MASTER
|
||||
interface nic0
|
||||
virtual_router_id 63
|
||||
priority 150
|
||||
advert_int 1
|
||||
nopreempt
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass Letmein1$
|
||||
}
|
||||
|
||||
unicast_peer {
|
||||
192.168.5.214
|
||||
192.168.15.182
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
192.168.63.254
|
||||
}
|
||||
|
||||
track_script {
|
||||
check_wolfstack
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
# ============================================================
|
||||
# WolfStack Keepalived — SECONDARY (Node 2 / mk44)
|
||||
# Host: 192.168.5.214
|
||||
# VIP : 192.168.63.254
|
||||
# Interface: eth0
|
||||
# ============================================================
|
||||
|
||||
global_defs {
|
||||
router_id WOLFSTACK_NODE2
|
||||
script_security
|
||||
enable_script_security
|
||||
}
|
||||
|
||||
vrrp_script check_wolfstack {
|
||||
script "/usr/bin/nc -z -w 2 127.0.0.1 8553"
|
||||
interval 2
|
||||
timeout 3
|
||||
fall 3
|
||||
rise 2
|
||||
weight -40
|
||||
}
|
||||
|
||||
vrrp_instance VI_WOLFSTACK {
|
||||
state BACKUP
|
||||
interface eth0
|
||||
virtual_router_id 63
|
||||
priority 130
|
||||
advert_int 1
|
||||
nopreempt
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass Letmein1$
|
||||
}
|
||||
|
||||
unicast_peer {
|
||||
192.168.50.41
|
||||
192.168.15.182
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
192.168.63.254
|
||||
}
|
||||
|
||||
track_script {
|
||||
check_wolfstack
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
# ============================================================
|
||||
# WolfStack Keepalived — TERTIARY (Node 3 / artemis)
|
||||
# Host: 192.168.15.182
|
||||
# VIP : 192.168.63.254
|
||||
# Interface: nic0
|
||||
# ============================================================
|
||||
|
||||
global_defs {
|
||||
router_id WOLFSTACK_NODE3
|
||||
script_security
|
||||
enable_script_security
|
||||
}
|
||||
|
||||
vrrp_script check_wolfstack {
|
||||
script "/usr/bin/nc -z -w 2 127.0.0.1 8553"
|
||||
interval 2
|
||||
timeout 3
|
||||
fall 3
|
||||
rise 2
|
||||
weight -40
|
||||
}
|
||||
|
||||
vrrp_instance VI_WOLFSTACK {
|
||||
state BACKUP
|
||||
interface nic0
|
||||
virtual_router_id 63
|
||||
priority 110
|
||||
advert_int 1
|
||||
nopreempt
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass Letmein1$
|
||||
}
|
||||
|
||||
unicast_peer {
|
||||
192.168.50.41
|
||||
192.168.5.214
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
192.168.63.254
|
||||
}
|
||||
|
||||
track_script {
|
||||
check_wolfstack
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user