Iron-Legion/documentation
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update Igor IP: 192.168.26.130 — ZimaOS NAS, Trilium, ARR Media Stack, Beszel agent

Browse Source
This commit is contained in:
F.R.I.D.A.Y.
2026-06-04 15:45:19 -04:00
parent ba84a78268
commit 6135fdf6ae
2 changed files with 331 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

319
PRD Drafts/ai-agent-knowledge-management.md Normal file
View File

@@ -0,0 +1,319 @@
# AI Agent Knowledge Management System PRD
**Status:** Draft | **Author:** Artemis (AI Foreman) | **Date:** 2026-06-04
---
## 1. Executive Summary
Artemis (Hermes Agent) generates persistent memory (MEMORY.md, USER.md), skills (SKILL.md files), operational logs, and PRD drafts. These `.md` files currently live in `~/.hermes/` and `~/documentation/` on the Artemis host. Commander Bobby needs a self-hosted knowledge management system with a web UI to:
1. **Review and organize** Artemis's memory/skill files outside the agent context
2. **Bidirectionally sync** edits from the UI back to the filesystem
3. **AI-assisted review** of memory files for optimization (compaction, deduplication, relevance scoring)
4. **Serve as the canonical knowledge base** for Iron Legion operational documentation
This PRD compares four candidates—TriliumNext Notes, Joplin Server, Obsidian, and Google NotebookLM—against these requirements and recommends an architecture.
---
## 2. Requirements Recap
| ID | Requirement | Priority |
|---|---|---|
| R1 | Self-hosted (Docker, LXC, or VM on Proxmox) | **Hard** |
| R2 | Web-based UI for reading/editing notes | **Hard** |
| R3 | Bidirectional sync with filesystem Markdown files | **Hard** |
| R4 | AI-powered review/analysis of notes | **Medium** |
| R5 | Markdown-native or robust Markdown import/export | **Hard** |
| R6 | REST API for automated scripting/cron integration | **Medium** |
| R7 | Full-text search across all notes | **Medium** |
| R8 | Note versioning / revision history | **Low** |
| R9 | Team/collaborative access (Commander Bobby only for now) | **Low** |
---
## 3. Hardware Investigation (Crucial Finding)
Before evaluating candidates, a critical discovery about the Proxmox cluster hardware must be addressed:
### 3.1 The Problem
- **MK33, MK34, MK39** (PVE hosts) run Intel **N100 CPUs** with **4 cores / 4 threads each**
- Proxmox already reports **CPU utilization**: MK33 at 18.50%, MK34 at 32.42%, MK39 at 19.89%
- A medium-weight Node.js app like TriliumNext would consume 10-20% of a single node's total capacity under load
- MK42 has an Intel **J4125** (4 cores, 2.0GHz) which is **even weaker** — marginal but usable with strict cgroup limits
- The bare-metal fleet (MK7, Artemis host) have stronger CPUs but are already purpose-assigned
- **TrueNAS SCALE 25.10.2** (Beelink mini PC): 4-core CPU, 11GB RAM (3.5GB available), load 0.09 — substantial headroom confirmed via live check
### 3.2 The Resource Model: LXC vs VM
**Critical correction:** Proxmox LXC containers use **cgroups v2**, not VM-style resource reservation. CPU is **opportunistic and shared** — an LXC only consumes CPU when actively processing, and the host kernel scheduler dynamically balances contention between containers.
Key LXC resource controls:
| Control | Effect |
|---|---|
| `cores` | Number of visible CPU cores (scheduling masks) |
| `cpulimit` | **Hard ceiling** — the LXC cannot exceed this fraction of host CPU |
| `cpu.weight` | **Relative priority** under contention (default 100, lower = less priority) |
| `memory` | **Hard limit** — kernel OOM-kills processes if exceeded |
This means an idle TriliumNext LXC consumes **near-zero host CPU**, and an active one respects its ceiling regardless of other workloads. The earlier concern about "adding to an already-loaded node pushes it to 50%+" is **mitigated** when using `cpulimit` caps.
### 3.3 Risk Assessment (Revised)
With proper Proxmox LXC cgroup limits (`cores: 2`, `cpulimit: 2`, `memory: 512M`, `cpu.weight: 128`):
| Deployment Target | Risk | Notes |
|---|---|---|
| **TrueNAS SCALE Docker** | ✅ Low | 4-core, 3.5GB free RAM, load 0.09. Preferred target |
| **MK33 PVE LXC** | ⚠️ Medium | 18% baseline, N100. Manageable with cpulimit=1.5 |
| **MK34 PVE LXC** | ⚠️ Medium | 32% baseline, N100. Needs cpulimit=1.0 |
| **MK39 PVE LXC** | ⚠️ Medium | 20% baseline, N100. Manageable with cpulimit=1.5 |
| **MK42 PVE LXC** | ⚠️ High | J4125 2.0GHz, weakest CPU. cpulimit=0.5 minimum |
| **Artemis host** | ⚠️ Medium | Stronger CPU but already runs Hermes agent (latency-sensitive) |
| **MK7** | ❌ | Purpose-assigned: Paperclip + PostgreSQL, no spare capacity |
### 3.4 Recommended Path
**Primary: TrueNAS SCALE 25.10.2 Docker Compose** (confirmed 4-core, 11GB RAM, 0.09 load — ample headroom). **Fallback: PVE LXC on MK34 or MK39** with strict cgroup limits (cpulimit=1.0, memory=512M). Both approaches are viable; TrueNAS is preferred for CPU headroom.
---
## 4. Candidate Evaluation
### 4.1 TriliumNext Notes
**Current state:** Active community fork of the archived `zadam/trilium`. Latest release `v0.103.0` (2026-05-13). 36,300+ GitHub stars. Commits daily.
| Criterion | Status | Details |
|---|---|---|
| **Self-hosted** | ✅ | Official Docker image `triliumnext/trilium:latest`, AMD64+ARM64 |
| **Web UI** | ✅ | Full WYSIWYG editor, tree-based hierarchy, relation maps, mind maps |
| **Bidirectional MD sync** | ⚠️ | Bulk Markdown import/export supported, but no live filesystem watcher. Requires cron + ETAPI scripting |
| **AI integration** | ⚠️ | No built-in AI. JavaScript scripting engine can call external APIs (Ollama). Community themes/scripts exist |
| **REST API** | ✅ | **ETAPI** (External REST API) — OpenAPI-documented, supports CRUD on notes, search, attributes, import/export. Authenticated via token |
| **Markdown support** | ✅ | Native Markdown import/export; WYSIWYG editor auto-formats Markdown |
| **Full-text search** | ✅ | Built-in |
| **Versioning** | ✅ | Per-note revision history |
| **Performance** | ⚠️ | Node.js app. Idle ~150MB RAM, moderate CPU. Scales to 100K+ notes. **May strain N100/J4125 CPUs** |
| **Maintenance risk** | ⚠️ | TriliumNext is a community fork; original author archived `zadam/trilium`. Sync version incompatibility with old instances |
**Deployment:** Minimal Docker Compose:
```yaml
services:
trilium:
image: triliumnext/trilium:v0.103.0
ports:
- "8080:8080"
volumes:
- ~/trilium-data:/home/node/trilium-data
environment:
- TRILIUM_DATA_DIR=/home/node/trilium-data
```
**Verdict:** Best fit from the candidate list. Self-hosted, API-driven, Markdown-capable. AI gap is bridgeable via scripting + Ollama. Performance is manageable with cgroup limits (LXC) or on TrueNAS Docker (preferred, 4-core/11GB/0.09 load).
---
### 4.2 Joplin Server
**Current state:** Actively maintained by `laurent22`. Stable. Docker image `joplin/server:latest`. Sync server for Joplin desktop/mobile clients.
| Criterion | Status | Details |
|---|---|---|
| **Self-hosted** | ✅ | Official Docker image, PostgreSQL backend, well-documented |
| **Web UI** | ❌ | Joplin Server is a **sync backend only**. The web client is minimal (basic note listing). Primary editing requires Joplin desktop/mobile app |
| **Bidirectional MD sync** | ⚠️ | Joplin's sync protocol is its own format (not raw MD). Export to raw Markdown requires the desktop app. Server API can push/pull notes in Joplin's JSON format |
| **AI integration** | ❌ | No server-side AI. Desktop plugins exist (e.g., "Note Overview" with ChatGPT) but require running desktop client |
| **REST API** | ✅ | Joplin Server API (beta), supports note CRUD, but less feature-rich than Trilium's ETAPI |
| **Markdown support** | ✅ | Joplin notes are Markdown internally, but stored in a SQL database, not as raw `.md` files |
| **Full-text search** | ✅ | Via PostgreSQL FTS in Joplin Server 3.x |
| **Versioning** | ✅ | Note history API available |
| **Maintenance risk** | ✅ | Low. Actively maintained, stable releases. Backed by a sole developer but well-established |
**Verdict:** Excellent sync server, **not a web-based knowledge manager**. If Bobby wants to use Joplin desktop/mobile as his primary interface and sync through the server, this works. But requirement R2 (web-based UI) is essentially unmet. No AI pathway without desktop client.
---
### 4.3 Obsidian
**Current state:** Proprietary Electron desktop app. Extremely popular (100K+ GitHub stars for plugin API). No server edition.
| Criterion | Status | Details |
|---|---|---|
| **Self-hosted** | ❌ | **No.** Desktop app only. Obsidian Sync ($5/mo) and Publish ($10/mo) are proprietary cloud services. No official Docker image or server mode |
| **Web UI** | ❌ | No. Obsidian Publish creates read-only static websites, not editable |
| **Bidirectional MD sync** | ⚠️ | Vaults are plain `.md` folders on disk — trivially scriptable. But editing requires the desktop app open. Community `obsidian-local-rest-api` plugin provides REST API **but only when the desktop app is running** |
| **AI integration** | ⚠️ | Community plugins exist (Smart Connections, Copilot, etc.). Local LLM integration possible via plugins. But again: desktop app must be running |
| **REST API** | ⚠️ | Only via community `obsidian-local-rest-api` plugin + running desktop app. Not headless |
| **Markdown support** | ✅ | Gold standard. Native Markdown with extensive extended syntax |
| **Versioning** | ✅ | Via Obsidian Sync ($5/mo) or external git |
| **Maintenance risk** | ⚠️ | Proprietary. Future licensing changes could affect workflows. Community heavy but core is closed-source |
**Verdict:** The best Markdown editor on the market. **Entirely wrong architecture for this use case.** Cannot run headless on a server. If Bobby wants to manually review/edit memory files on his desktop, Obsidian is excellent—but it does not meet the self-hosted web service requirement.
---
### 4.4 Google NotebookLM
**Current state:** Google cloud service. Upload documents → AI-powered Q&A, summaries, audio overviews. No self-hosted edition.
| Criterion | Status | Details |
|---|---|---|
| **Self-hosted** | ❌ | **No.** Pure Google SaaS |
| **Web UI** | ✅ | Excellent web interface, but cloud-only |
| **AI support** | ✅ | **Best-in-class.** Native RAG-powered summarization, Q&A, audio overviews |
| **Markdown support** | ⚠️ | Uploads supported but NotebookLM is document-centric, not a Markdown editor |
| **REST API** | ❌ | No public API for NotebookLM |
| **Bidirectional sync** | ❌ | Upload only. No export back to filesystem |
| **Data sovereignty** | ❌ | All data lives on Google servers |
**Verdict:** Wrong architecture for this use case. No self-hosting. No bidirectional sync. No API.
**However:** NotebookLM's AI capabilities are exactly what Bobby wants for reviewing memory files. See Section 5 for open-source RAG alternatives.
---
## 5. AI-Powered Review: Open-Source RAG Layer
NotebookLM is unavailable for self-hosting, but its core functionality (upload documents → ask questions → get summaries) can be replicated with open-source tools. This would be a **separate service** that ingests Markdown exports from the knowledge base.
### 5.1 Candidate RAG Tools
| Tool | Stars | Deployment | Notes |
|---|---|---|---|
| **AnythingLLM** | 35K+★ | Docker, single binary | All-in-one: document ingestion, RAG chat, multi-model (Ollama, OpenAI, etc.). Agent support. Best fit for "upload and chat" workflow |
| **Dify** | 60K+★ | Docker Compose | Full AI application platform. RAG pipeline builder, chat UI, workflow automation. Overkill but powerful |
| **PrivateGPT** | 60K+★ | Docker, Python | Privacy-focused document Q&A. Simpler than Dify. Good for batch processing docs |
| **Open WebUI** | 65K+★ | Docker | Ollama-native chat UI with RAG and document upload. Already running on the fleet? |
| **Flowise** | 35K+★ | Docker, Node.js | Low-code LLM workflow builder. Drag-and-drop RAG pipeline. Good for custom ingestion chains |
### 5.2 Recommended AI Layer: AnythingLLM
**Why:** Single Docker container, natively supports Ollama (already in the fleet), built-in document ingestion with chunking + embedding, chat-based review, multi-user. Lightweight enough to co-reside with the knowledge base or run on a separate node.
**Architecture:** TriliumNext exports `.md` files via cron → AnythingLLM ingests them → Bobby chats with his memory files via AnythingLLM's UI → Insights inform manual edits in TriliumNext.
---
## 6. Comparative Matrix
| Criterion | TriliumNext | Joplin Server | Obsidian | NotebookLM |
|---|---|---|---|---|
| Self-hosted Docker | ✅ | ✅ | ❌ | ❌ |
| Web UI (edit) | ✅ | ❌ | ❌ | ✅ (cloud) |
| Bidirectional MD sync | ⚠️ scriptable | ⚠️ via desktop | ❌ | ❌ |
| REST API | ✅ ETAPI | ✅ (beta) | ❌ | ❌ |
| AI integration | ⚠️ scriptable | ❌ | ⚠️ plugins | ✅ native |
| Markdown-native | ✅ import/export | ✅ internal | ✅ gold standard | ❌ |
| Full-text search | ✅ | ✅ | ✅ | ✅ |
| Versioning | ✅ | ✅ | ✅ | ✅ |
| Maintenance risk | ⚠️ community fork | ✅ stable | ⚠️ proprietary | ❌ Google SaaS |
| Hardware fit | ⚠️ Node.js on N100 | ⚠️ PostgreSQL needed | N/A | N/A |
---
## 7. Recommended Architecture
### 7.1 Primary Recommendation: TriliumNext on TrueNAS Docker
**Deploy:** TriliumNext as a Custom Docker Compose app on TrueNAS SCALE 25.10.2 (hardware pre-check pending).
**Rationale:**
- Only candidate from Bobby's list that meets all hard requirements (self-hosted web UI, Markdown support, REST API)
- Active community fork with daily commits
- Scripting engine bridges the AI gap
- TrueNAS has stronger hardware than PVE N100 nodes
### 7.2 AI Layer (Phase 2): AnythingLLM alongside TriliumNext
1. Cron job exports Trilium notes as `.md` to a shared volume
2. AnythingLLM watches the volume and ingests new/changed docs
3. Bobby uses AnythingLLM's chat UI for AI-powered memory review
4. Insights from AI review → manual edits in TriliumNext web UI
5. TriliumNext edits → cron syncs back to Artemis `~/.hermes/` filesystem
### 7.3 Bidirectional Sync Flow
```
Artemis ~/.hermes/memory/ ──cron export──→ TriliumNext (web UI)
│
Bobby reviews/edits
│
TriliumNext ←──cron pull── Bobby's edits saved
│
└──cron export──→ ~/.hermes/memory/ (Artemis reads updated files)
```
### 7.4 Fallback: Joplin Server
If TriliumNext proves too heavy for available hardware:
- Joplin Server on a PVE node (or TrueNAS Docker)
- Bobby uses Joplin desktop app for editing (not web UI)
- Sync via Joplin's native protocol
- Markdown export via Joplin CLI → Artemis filesystem
---
## 8. Deployment Plan (Proposed)
### Phase 1: TrueNAS Hardware Verification
1. SSH to TrueNAS — check CPU load, available RAM, Docker compatibility
2. Verify TrueNAS SCALE 25.10.2 supports privileged Custom Docker Compose apps
3. Test-deploy TriliumNext with resource limits (`cpus: 1.0`, `memory: 512M`)
4. Measure idle CPU/RAM consumption
### Phase 2: TriliumNext Deployment
1. Deploy TriliumNext via TrueNAS Docker Compose UI
2. Configure ETAPI token for automation
3. Import existing `~/.hermes/memory/` and `~/.hermes/skills/` as note trees
4. Set up cron for bidirectional `.md` export/import between Artemis host and TriliumNext
### Phase 3: AI Integration
1. Deploy AnythingLLM as a companion Docker Compose app on TrueNAS
2. Configure Ollama backend (point to fleet Ollama instance)
3. Create ingestion pipeline: Trilium export → shared volume → AnythingLLM workspace
4. Test AI-assisted review workflow
### Phase 4: Productionize
1. Document sync schedule and retention policy
2. Add healthcheck monitoring
3. (Stretch) Explore real-time sync via filesystem watcher instead of cron
---
## 9. Open Questions
1. **TrueNAS Docker suitability?** Can TrueNAS SCALE 25.10.2's Docker Compose app platform run a Node.js app with moderate CPU/RAM usage without impacting NAS performance?
2. **Alternative deployment target?** If TrueNAS is unsuitable and PVE nodes are too constrained, is there a bare-metal node with spare capacity? MK7 is assigned to Paperclip + PostgreSQL. Artemis host could run TriliumNext but adds desktop workload to the agent's host.
3. **NotebookLM "light"?** If AI review is the primary goal and web editing is secondary, would a minimal RAG setup (AnythingLLM + direct Markdown file ingestion, no knowledge base UI) suffice for Phase 1?
4. **Sync frequency?** How often should Artemis export memory to TriliumNext, and how often should TriliumNext edits sync back? 5-minute cron? Hourly? Manual trigger?
5. **Scope of "bidirectional"?** Can Artemis **read** TriliumNext as an authoritative source for memory, or is TriliumNext purely a review/edit sandbox where changes are manually promoted?
---
## 10. Decision Required
**Bobby to decide:**
| Decision | Options |
|---|---|
| **Primary tool** | A) TriliumNext on TrueNAS (recommended) — B) Joplin Server + desktop app — C) Minimalist: plain Markdown + AnythingLLM for AI review only |
| **Deployment target** | A) TrueNAS SCALE Docker — B) PVE node (despite CPU concerns) — C) Artemis host — D) MK7 |
| **AI layer** | A) AnythingLLM (recommended) — B) No AI yet, manual review only — C) Open WebUI if already in fleet |
| **Sync direction** | A) Bidirectional (read + write) — B) Read-only archive for review — C) Artemis treats TriliumNext as source of truth |
---
## Appendix A: Source References
- TriliumNext GitHub: `https://github.com/TriliumNext/Trilium` (36.3K ★, active)
- TriliumNext Docker Hub: `triliumnext/trilium` (2.6M pulls)
- Joplin Server: `https://github.com/laurent22/joplin` (47K ★)
- Obsidian Local REST API: `https://github.com/coddingtonbear/obsidian-local-rest-api` (2.4K ★)
- AnythingLLM: `https://github.com/Mintplex-Labs/anything-llm` (35K+ ★)
- TrueNAS SCALE 25.10.2 release notes: Apps migrated to Docker Compose from Kubernetes
- Research conducted: 2026-06-04 via web scraping of official sites, GitHub APIs, and Docker Hub