Compare commits
No commits in common. "38b91136c2f282a8dc358632f168cf17abba78dd" and "6b034c53e24d093e2b048cd57553665ae1d44d97" have entirely different histories.
38b91136c2
...
6b034c53e2
30 changed files with 38 additions and 532 deletions
5
.gitignore
vendored
5
.gitignore
vendored
|
|
@ -1,8 +1,3 @@
|
|||
__pycache__/
|
||||
*.pyc
|
||||
.alert_state.json
|
||||
|
||||
# Backup-/Snapshot-Dateien (keine Versionierung)
|
||||
*.bak
|
||||
*.bak.*
|
||||
*-bak
|
||||
|
|
|
|||
|
|
@ -190,39 +190,3 @@ Neue Leitlinie:
|
|||
- **M5 Ultra** nur dann, wenn ein konkretes Modell das rechtfertigt.
|
||||
|
||||
`homelab.conf` und `MASTER_INDEX.md` werden parallel angepasst.
|
||||
|
||||
## Entscheidungsnotiz: Qwen3.6 + BeeLlama fuer opencode-Tests
|
||||
|
||||
Stand: 2026-05-18
|
||||
|
||||
Qwen3.6 ueber BeeLlama ist fuer einfache, kontrollierte `opencode`-Tests ausreichend:
|
||||
|
||||
- Git-Repos klonen
|
||||
- README, `package.json` und Setup-Hinweise lesen
|
||||
- Projektstruktur erkunden
|
||||
- Shell-/Git-Kommandos ausfuehren
|
||||
- Tests finden und starten
|
||||
- kleine, klar begrenzte Codeaenderungen vorbereiten
|
||||
|
||||
Wichtig: BeeLlama macht das Modell schneller und lokal praktikabler, aber nicht automatisch intelligenter. Fuer komplexe Coding-Agent-Aufgaben wie grosse Refactorings, viele Dateien gleichzeitig, schwierige Debug-Schleifen oder produktive Aenderungen bleibt ein spezialisiertes Coding-Modell zuverlaessiger.
|
||||
|
||||
Empfohlener erster Testauftrag:
|
||||
|
||||
```text
|
||||
Klon dieses Repo nach /tmp/opencode-test, lies README und package.json, aber aendere nichts. Sag danach, wie man es startet und welche Tests definiert sind.
|
||||
```
|
||||
|
||||
## Entscheidungsnotiz: zweite RTX 3090
|
||||
|
||||
Eine zweite RTX 3090 verbessert die Qualitaet desselben Modells nicht direkt. Dasselbe Modell mit derselben Quantisierung liefert grundsaetzlich dieselbe Antwortqualitaet.
|
||||
|
||||
Eine zweite GPU hilft indirekt durch:
|
||||
|
||||
- groessere Modelle, die vorher nicht in den VRAM passten
|
||||
- bessere Quantisierung, z.B. Q5/Q6 statt Q4
|
||||
- mehr Kontext/KV-Cache
|
||||
- weniger CPU-Offload
|
||||
- hoehere Tokenrate bei gutem Layer-/Tensor-Splitting
|
||||
- mehr parallele Requests
|
||||
|
||||
Fuer `opencode` waere der echte Qualitaetsgewinn daher nicht die zweite GPU selbst, sondern dass dadurch eventuell ein besseres oder groesseres Coding-Modell lokal betrieben werden kann.
|
||||
|
|
|
|||
|
|
@ -140,58 +140,3 @@ curl -s "http://100.77.105.3:8096/Items?api_key=7285b4a8793541648bf156599ae05b43
|
|||
| Ralf | nein |
|
||||
| rolf | nein |
|
||||
| wilfrid | nein |
|
||||
|
||||
## Troubleshooting: Details fehlen / Remote-Zugriff wirkt kaputt
|
||||
|
||||
Stand: 2026-05-16
|
||||
|
||||
Symptom:
|
||||
- Filmübersicht ist erreichbar, aber Detailseiten/Filmdetails werden im Client nicht angezeigt.
|
||||
- Zugriff über Tailscale-IP `100.77.105.3:8096` hängt oder läuft in Timeouts.
|
||||
|
||||
Befund vom 2026-05-16:
|
||||
- Jellyfin selbst war lokal gesund:
|
||||
- `http://192.168.178.149:8096/health` -> `Healthy`
|
||||
- API `/Items/Counts` lieferte `897` Filme.
|
||||
- Beispiel-Film `2 Guns` hatte Beschreibung, Genres, TMDB/IMDB IDs und Bilder.
|
||||
- Medien-Mounts waren aktiv:
|
||||
- `/mnt/nas`
|
||||
- `/srv/media`
|
||||
- Docker-Container `jellyfin` war `healthy`.
|
||||
- Ursache war Tailscale auf dem Jellyfin-Host:
|
||||
- `tailscale ip -4` meldete zuerst `no current Tailscale IPs; state: NeedsLogin`
|
||||
- `tailscale status` meldete `Logged out`.
|
||||
- Nach Re-Login war `100.77.105.3` wieder aktiv und `http://100.77.105.3:8096/health` antwortete wieder `Healthy`.
|
||||
- Danach waren die Filmdetails im Client wieder sichtbar.
|
||||
|
||||
Schnelltest:
|
||||
```bash
|
||||
# Von pve3 lokal im LAN:
|
||||
curl -sS http://192.168.178.149:8096/health
|
||||
|
||||
# Von CT 116 / Hausmeister oder einem Tailscale-Host:
|
||||
curl -sS http://100.77.105.3:8096/health
|
||||
curl -sS "http://100.77.105.3:8096/Items/Counts?api_key=7285b4a8793541648bf156599ae05b43"
|
||||
```
|
||||
|
||||
Fix:
|
||||
1. Auf dem Jellyfin-Host Tailscale-Status prüfen:
|
||||
```bash
|
||||
tailscale ip -4
|
||||
tailscale status --self
|
||||
```
|
||||
2. Wenn `NeedsLogin` oder `Logged out`: Tailscale neu anmelden.
|
||||
3. Danach erneut prüfen:
|
||||
```bash
|
||||
curl -sS http://100.77.105.3:8096/health
|
||||
```
|
||||
4. Im Browser Jellyfin hart neu laden oder neu einloggen, falls der Webclient alte kaputte API-Antworten gecacht hat.
|
||||
|
||||
Wichtig:
|
||||
- Fehlende Details bedeuten nicht automatisch kaputte Jellyfin-Datenbank oder fehlende Metadaten.
|
||||
- Erst lokale API gegen `192.168.178.149:8096` prüfen, dann Tailscale/Cloudflare-Zugriff.
|
||||
- `systemctl is-active jellyfin` kann irreführend sein, weil Jellyfin hier im Docker-Container läuft. Aussagekräftiger ist:
|
||||
```bash
|
||||
docker ps | grep jellyfin
|
||||
curl -sS http://127.0.0.1:8096/health
|
||||
```
|
||||
|
|
|
|||
BIN
homelab-ai-bot/__pycache__/context.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/__pycache__/context.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/__pycache__/llm.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/__pycache__/llm.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/__pycache__/monitor.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/__pycache__/monitor.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/__init__.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/__init__.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/__init__.cpython-313.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/__init__.cpython-313.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/config.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/config.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/config.cpython-313.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/config.cpython-313.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/forgejo_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/forgejo_client.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/loki_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/loki_client.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/loki_client.cpython-313.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/loki_client.cpython-313.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/mail_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/mail_client.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/mail_client.cpython-313.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/mail_client.cpython-313.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/pbs_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/pbs_client.cpython-311.pyc
Normal file
Binary file not shown.
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/proxmox_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/proxmox_client.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/proxmox_client.cpython-313.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/proxmox_client.cpython-313.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/seafile_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/seafile_client.cpython-311.pyc
Normal file
Binary file not shown.
BIN
homelab-ai-bot/core/__pycache__/wordpress_client.cpython-311.pyc
Normal file
BIN
homelab-ai-bot/core/__pycache__/wordpress_client.cpython-311.pyc
Normal file
Binary file not shown.
|
|
@ -5,28 +5,6 @@ from datetime import datetime, timezone, timedelta
|
|||
|
||||
LOKI_URL = "http://100.109.206.43:3100"
|
||||
|
||||
# Transiente cloudflared/Cloudflare-Edge-Flaps erzeugen oft error="..."
|
||||
# in WRN/ERR-Zeilen, sind aber keine App-Fehler von WordPress/RSS.
|
||||
ERROR_EXCLUDE_RE = (
|
||||
"caller=metrics|query_hash=|executing query|scheduler_processor|"
|
||||
"Aborted connection|systemd-networkd-wait-online|context canceled|"
|
||||
"AH01630: client denied|flag evaluation succeeded|pluginsAutoUpdate|"
|
||||
"control stream encountered a failure while serving|Serve tunnel error|"
|
||||
"failed to serve tunnel connection|Connection terminated|"
|
||||
"failed to dial to edge with quic|Failed to dial a quic connection|"
|
||||
"no recent network activity|failed to accept QUIC stream|"
|
||||
"control stream|accept stream listener|serve tunnel|tunnel connection|"
|
||||
"edge with quic|quic connection|QUIC stream|"
|
||||
"accept stream listener encountered a failure while serving|"
|
||||
"datagram manager encountered a failure while serving|"
|
||||
"datagram manager|datagram handler|"
|
||||
"failed to run the datagram handler|failed to accept incoming stream requests|"
|
||||
"Application error 0x0|connIndex=|"
|
||||
"beellama-tunnel.service: Main process exited|"
|
||||
"beellama-expose.service: Main process exited|"
|
||||
"health\(warnable=mapresponse-timeout\)"
|
||||
)
|
||||
|
||||
|
||||
def _query(endpoint: str, params: dict, base_url: str = None) -> dict:
|
||||
url = f"{base_url or LOKI_URL}{endpoint}"
|
||||
|
|
@ -71,9 +49,9 @@ def query_logs(query: str, hours: float = 1, limit: int = 100) -> list[dict]:
|
|||
def get_errors(container: str = None, hours: float = 1, limit: int = 200) -> list[dict]:
|
||||
"""Get error-level logs, optionally filtered by container hostname."""
|
||||
if container:
|
||||
q = f'{{host="{container}"}} |~ "(?i)(error|fatal|panic|traceback|exception)" !~ "{ERROR_EXCLUDE_RE}"'
|
||||
q = f'{{host="{container}"}} |~ "(?i)(error|fatal|panic|traceback|exception)" !~ "caller=metrics|query_hash=|executing query|scheduler_processor|Aborted connection|systemd-networkd-wait-online|context canceled|AH01630: client denied|flag evaluation succeeded|pluginsAutoUpdate|terror"'
|
||||
else:
|
||||
q = f'{{job=~".+"}} |~ "(?i)(error|fatal|panic|traceback|exception)" !~ "{ERROR_EXCLUDE_RE}"'
|
||||
q = '{job=~".+"} |~ "(?i)(error|fatal|panic|traceback|exception)" !~ "caller=metrics|query_hash=|executing query|scheduler_processor|Aborted connection|systemd-networkd-wait-online|context canceled|AH01630: client denied|flag evaluation succeeded|pluginsAutoUpdate|terror"'
|
||||
return query_logs(q, hours=hours, limit=limit)
|
||||
|
||||
|
||||
|
|
@ -124,8 +102,7 @@ def get_health(container: str, hours: float = 24) -> dict:
|
|||
|
||||
return {
|
||||
"host": container,
|
||||
"error_count": error_count,
|
||||
f"errors_last_{hours}h": error_count,
|
||||
"errors_last_{hours}h": error_count,
|
||||
"sending_logs": has_recent,
|
||||
"status": "healthy" if error_count < 5 and has_recent else
|
||||
"warning" if error_count < 20 else "critical",
|
||||
|
|
@ -137,7 +114,6 @@ WATCHED_SERVICES = [
|
|||
("wordpress-v2", "wordpress"),
|
||||
("fuenfvoracht", "fuenfvoracht"),
|
||||
("homelab-ai-bot", "hausmeister"),
|
||||
("hermes-mu", "hermes-gateway"),
|
||||
]
|
||||
|
||||
|
||||
|
|
@ -145,7 +121,7 @@ def count_errors(hours: float = 24) -> dict:
|
|||
"""Zählt Fehler-Log-Einträge über einen Zeitraum via Loki metric query."""
|
||||
now = datetime.now(timezone.utc)
|
||||
start = now - timedelta(hours=hours)
|
||||
q = f'{{job=~".+"}} |~ "(?i)(error|fatal|panic|traceback|exception)" !~ "{ERROR_EXCLUDE_RE}"'
|
||||
q = '{job=~".+"} |~ "(?i)(error|fatal|panic|traceback|exception)" !~ "caller=metrics|query_hash=|executing query|scheduler_processor|Aborted connection|systemd-networkd-wait-online|context canceled|AH01630: client denied|flag evaluation succeeded|pluginsAutoUpdate|terror"'
|
||||
# Loki instant metric query für Gesamtanzahl
|
||||
data = _query("/loki/api/v1/query_range", {
|
||||
"query": q,
|
||||
|
|
@ -184,7 +160,7 @@ def check_service_restarts(minutes: int = 35) -> list[dict]:
|
|||
ERROR_RATE_THRESHOLDS = {
|
||||
"rss-manager": 15,
|
||||
"wordpress-v2": 10,
|
||||
"forgejo": 200, # Oeffentlich – Scanner-404er sind normal
|
||||
"forgejo": 200, # Oeffentlich — Scanner-404er sind normal
|
||||
}
|
||||
ERROR_RATE_DEFAULT = 25
|
||||
|
||||
|
|
@ -195,7 +171,7 @@ def check_error_rate(minutes: int = 30) -> list[dict]:
|
|||
alerts = []
|
||||
now = datetime.now(timezone.utc)
|
||||
for host in all_hosts:
|
||||
q = f'count_over_time({{host="{host}"}} |~ "(?i)error" !~ "{ERROR_EXCLUDE_RE}" [{minutes}m])'
|
||||
q = f'count_over_time({{host="{host}"}} |~ "(?i)error" !~ "caller=metrics|query_hash=|executing query|scheduler_processor|Aborted connection|systemd-networkd-wait-online|context canceled|AH01630: client denied|flag evaluation succeeded|pluginsAutoUpdate|terror" [{minutes}m])'
|
||||
data = _query("/loki/api/v1/query", {"query": q, "time": _ns(now)})
|
||||
count = sum(
|
||||
int(float(r.get("value", [None, "0"])[1]))
|
||||
|
|
|
|||
|
|
@ -27,8 +27,8 @@ OLLAMA_MODELS = set()
|
|||
|
||||
|
||||
def warmup_ollama():
|
||||
"""No-Op: Text-Modell laeuft ueber OpenRouter (x-ai/grok-4.3), kein Ollama-Warmup noetig."""
|
||||
log.info('Ollama warmup uebersprungen — Text laeuft ueber OpenRouter (x-ai/grok-4.3)')
|
||||
"""No-Op: Text-Modell laeuft jetzt ueber OpenRouter (Grok 4.1 Fast), kein Ollama-Warmup noetig."""
|
||||
log.info('Ollama warmup uebersprungen — Text laeuft ueber OpenRouter (Grok 4.1 Fast)')
|
||||
PASSTHROUGH_TOOLS = {"get_temperaturen", "get_energie", "get_heizung"}
|
||||
|
||||
_LOCAL_OVERRIDES = [
|
||||
|
|
@ -341,7 +341,7 @@ def ask(question: str, context: str) -> str:
|
|||
return f"LLM-Fehler: {e}"
|
||||
|
||||
|
||||
def ask_with_tools(question: str, tool_handlers: dict, session_id: str = None, document_mode: bool = False, model_override: str = None) -> str:
|
||||
def ask_with_tools(question: str, tool_handlers: dict, session_id: str = None, document_mode: bool = False) -> str:
|
||||
"""Freitext-Frage mit automatischem Routing und Tool-Calling.
|
||||
|
||||
Routing:
|
||||
|
|
@ -573,12 +573,11 @@ def ask_with_tools(question: str, tool_handlers: dict, session_id: str = None, d
|
|||
return f"Online-Suche Fehler: {e}"
|
||||
|
||||
# --- Lokal: Tool-Calling mit allen Tools ---
|
||||
local_model = model_override or MODEL_LOCAL
|
||||
passthrough_result = None
|
||||
|
||||
try:
|
||||
for _round in range(MAX_TOOL_ROUNDS):
|
||||
data = _call_api(messages, api_key, use_tools=True, model=local_model)
|
||||
data = _call_api(messages, api_key, use_tools=True, model=MODEL_LOCAL)
|
||||
choice = data["choices"][0]
|
||||
msg = choice["message"]
|
||||
|
||||
|
|
@ -624,7 +623,7 @@ def ask_with_tools(question: str, tool_handlers: dict, session_id: str = None, d
|
|||
|
||||
if passthrough_result:
|
||||
return passthrough_result
|
||||
data = _call_api(messages, api_key, use_tools=False, model=local_model)
|
||||
data = _call_api(messages, api_key, use_tools=False, model=MODEL_LOCAL)
|
||||
return data["choices"][0]["message"]["content"]
|
||||
|
||||
except Exception as e:
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@ import sys
|
|||
import os
|
||||
import json
|
||||
import hashlib
|
||||
import re
|
||||
import requests
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
|
|
@ -23,8 +22,6 @@ ALERT_COOLDOWN_SECONDS = {
|
|||
"memory_expiry": 43200,
|
||||
"default": 3600,
|
||||
"error_rate": 1800,
|
||||
"hermes": 7200,
|
||||
"backup": 21600,
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -48,18 +45,10 @@ HTTP_HEALTH_CHECKS = [
|
|||
{"name": "WordPress (CT 101)", "url": "http://10.10.10.101/robots.txt"},
|
||||
{"name": "Matomo (CT 109)", "url": "http://10.10.10.109"},
|
||||
{"name": "Grafana (CT 110)", "url": "http://10.10.10.110:3000/api/health"},
|
||||
{"name": "Flugscanner-Agent (pve-pp-1)", "url": "http://100.126.26.46:5010/status",
|
||||
"retries": 5, "timeout": 25, "retry_delay": 6, "host": "pve-pp-1"},
|
||||
]
|
||||
|
||||
HERMES_HEALTH_CHECKS = [
|
||||
{"name": "Hermes Promtail", "url": "http://100.109.174.120:9080/ready", "timeout": 5},
|
||||
{"name": "Hermes node_exporter", "url": "http://100.109.174.120:9100/metrics", "timeout": 5},
|
||||
{"name": "Hermes Websuche CT121", "url": "http://100.74.196.29:8080/search?q=hausmeister-health&format=json", "timeout": 8},
|
||||
{"name": "Loki", "url": "http://100.109.206.43:3100/ready", "timeout": 5},
|
||||
{"name": "PBS Muldenstein", "url": "https://100.99.139.22:8007", "timeout": 5, "allow_status": {200, 401}},
|
||||
]
|
||||
|
||||
HERMES_LOG_HOST = "hermes-mu"
|
||||
|
||||
EXPECTED_STOPPED = {
|
||||
(115, "pve-ka-1"), # flugscanner-asia-old (gestoppt, Cluster pve1) — Live CT auf pve-pp-1
|
||||
(115, "pve-ka-3"), # dieselbe CT, zweite API-Sicht (Cluster)
|
||||
|
|
@ -68,9 +57,6 @@ EXPECTED_STOPPED = {
|
|||
(504, "pve-ka-2"), # Shop-Template — stopped
|
||||
(8000, "pve-ka-2"), # Kunde0-Shop — stopped
|
||||
(8010, "pve-ka-2"), # Kunde1-Shop — stopped
|
||||
(121, "pve-ka-1"), # Schawarma-Shop — gestoppt
|
||||
(121, "pve-ka-2"), # Schawarma-Shop — gestoppt
|
||||
(121, "pve-ka-3"), # Schawarma-Shop — gestoppt
|
||||
}
|
||||
|
||||
# VMIDs, die auf jedem Proxmox-Host in CONFIG ok sind, solange status == stopped
|
||||
|
|
@ -97,60 +83,12 @@ def _is_host_suppressed(host: str, suppressed_hosts: set) -> bool:
|
|||
return False
|
||||
|
||||
|
||||
def check_hermes() -> list[str]:
|
||||
"""Prueft Hermes (CT151) als kritischen Dienst ueber HTTP und Loki."""
|
||||
alerts = []
|
||||
headers = {"User-Agent": "Mozilla/5.0 (Hausmeister-Bot/1.0 hermes-check)"}
|
||||
for check in HERMES_HEALTH_CHECKS:
|
||||
allow_status = check.get("allow_status") or set(range(200, 400))
|
||||
try:
|
||||
r = requests.get(
|
||||
check["url"],
|
||||
timeout=check.get("timeout", 5),
|
||||
allow_redirects=False,
|
||||
headers=headers,
|
||||
verify=False,
|
||||
)
|
||||
if r.status_code not in allow_status:
|
||||
alerts.append(f"🔴 Hermes: {check['name']} HTTP {r.status_code}")
|
||||
except requests.RequestException as e:
|
||||
alerts.append(f"🔴 Hermes: {check['name']} nicht erreichbar: {str(e)[:80]}")
|
||||
|
||||
health = loki_client.get_health(HERMES_LOG_HOST, hours=1)
|
||||
if health.get("status") == "silent":
|
||||
alerts.append("⚠️ Hermes: keine aktuellen Loki-Logs von hermes-mu")
|
||||
elif health.get("status") == "critical":
|
||||
cnt = health.get("error_count", health.get("errors_last_1h", "?"))
|
||||
alerts.append(f"🔴 Hermes: {cnt} Fehlerlogs in Loki (1h)")
|
||||
|
||||
errors = loki_client.get_errors(container=HERMES_LOG_HOST, hours=0.5, limit=20)
|
||||
for e in errors:
|
||||
if "error" in e:
|
||||
continue
|
||||
line = (e.get("line") or "").lower()
|
||||
if "diskstats_linux.go" in line and "disabling udev device properties" in line:
|
||||
continue
|
||||
if "collector failed" in line and "node_exporter" in line:
|
||||
continue
|
||||
if "beellama-tunnel.service" in line or "beellama-expose.service" in line:
|
||||
if "main process exited" in line:
|
||||
continue
|
||||
if "health(warnable=mapresponse-timeout)" in line:
|
||||
continue
|
||||
if any(term in line for term in ["traceback", "exception", "failed", "timeout", "tool_call", "connection refused"]):
|
||||
alerts.append(f"🔴 Hermes Log: {(e.get('line') or '')[:140]}")
|
||||
break
|
||||
|
||||
return alerts
|
||||
|
||||
|
||||
def check_all() -> list[str]:
|
||||
"""Regelbasierter Check (Stufe 1). Gibt Liste von Alarmen zurück."""
|
||||
cfg = config.parse_config()
|
||||
suppressed_hosts = config.get_suppressed_hosts(cfg)
|
||||
suppressed_names = config.get_suppressed_container_names(cfg)
|
||||
alerts = []
|
||||
alerts.extend(check_hermes())
|
||||
|
||||
containers = proxmox_client.get_all_containers(
|
||||
_get_passwords(cfg), _get_tokens(cfg)
|
||||
|
|
@ -384,19 +322,6 @@ def _save_alert_state(state: dict):
|
|||
|
||||
|
||||
def _alert_key(alert_text: str) -> str:
|
||||
# Volatile Metrik-Alarme (error_rate) ueber einen stabilen Schluessel
|
||||
# deduplizieren: nur Host, nicht der schwankende Zaehlwert. Sonst
|
||||
# erzeugt jede neue Zahl einen neuen Hash und der Cooldown greift nie.
|
||||
if "Fehler in 30 Min" in alert_text:
|
||||
m = re.search(r"([\w.\-]+):\s*\d+\s+Fehler in 30 Min", alert_text)
|
||||
if m:
|
||||
return hashlib.md5(("error_rate|" + m.group(1)).encode()).hexdigest()
|
||||
if "Hermes:" in alert_text and "Fehlerlogs in Loki" in alert_text:
|
||||
return hashlib.md5(b"hermes|loki_errors").hexdigest()
|
||||
if alert_text.startswith("🔴 Hermes Log:"):
|
||||
m = re.search(r"([\w.\-]+\.service)", alert_text)
|
||||
unit = m.group(1) if m else "generic"
|
||||
return hashlib.md5(("hermes|log|" + unit).encode()).hexdigest()
|
||||
return hashlib.md5(alert_text.encode()).hexdigest()
|
||||
|
||||
|
||||
|
|
@ -415,10 +340,6 @@ def _alert_category(alert_text: str) -> str:
|
|||
return "http"
|
||||
if "Service-Neustart" in alert_text:
|
||||
return "restart"
|
||||
if "Hermes" in alert_text:
|
||||
return "hermes"
|
||||
if "Backup" in alert_text or "PBS" in alert_text:
|
||||
return "backup"
|
||||
if "Memory läuft ab" in alert_text:
|
||||
return "memory_expiry"
|
||||
return "default"
|
||||
|
|
|
|||
|
|
@ -661,7 +661,7 @@ setInterval(load,30000);
|
|||
return
|
||||
raw_title_part = m.group(1)
|
||||
|
||||
clean_title = raw_title_part.replace('__', ' - ').replace('_-_', ' - ').replace('_', ' ').strip()
|
||||
clean_title = raw_title_part.replace('_-_', ' - ').replace('_', ' ').strip()
|
||||
|
||||
cache = _load_filminfo_cache()
|
||||
matched = _find_cache_match(cache, clean_title)
|
||||
|
|
@ -700,26 +700,6 @@ setInterval(load,30000);
|
|||
import logging
|
||||
logging.getLogger("savetv").warning(f"Rename fehlgeschlagen {raw_filename}: {e}")
|
||||
|
||||
def _rename_all_pending():
|
||||
import re as _re
|
||||
renamed, skipped = [], []
|
||||
for fp in sorted(SAVETV_DIR.glob("*.mp4")):
|
||||
m = _re.match(r"^(.+)_(\d{6,9})\.mp4$", fp.name)
|
||||
if not m:
|
||||
continue
|
||||
old = fp.name
|
||||
tid = m.group(2)
|
||||
_rename_to_jellyfin(old, tid)
|
||||
if (SAVETV_DIR / old).exists():
|
||||
skipped.append(old)
|
||||
else:
|
||||
renamed.append(old)
|
||||
return {"renamed": renamed, "skipped": skipped}
|
||||
|
||||
@app.route("/api/rename_pending", methods=["POST"])
|
||||
def api_rename_pending():
|
||||
return jsonify(_rename_all_pending())
|
||||
|
||||
@app.route("/health")
|
||||
def health():
|
||||
from tools import savetv
|
||||
|
|
|
|||
|
|
@ -1,60 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Benennt Raw-MP4s auf dem NAS um (Titel_ID.mp4 -> Titel (Jahr).mp4)."""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
ZIEL = os.environ.get("SAVETV_NAS_DIR", "/mnt/nas/Filme zum nachbearbeiten")
|
||||
FILMINFO_API = os.environ.get("SAVETV_FILMINFO_API", "http://138.201.84.95:9443/api/filminfo")
|
||||
RAW_RE = re.compile(r"^(.+)_(\d{6,9})\.mp4$")
|
||||
|
||||
|
||||
def clean_title(raw_part):
|
||||
return raw_part.replace("_-_", " - ").replace("_", " ").strip()
|
||||
|
||||
|
||||
def filminfo(title):
|
||||
url = FILMINFO_API + "?" + urllib.parse.urlencode({"title": title})
|
||||
with urllib.request.urlopen(url, timeout=60) as r:
|
||||
return json.loads(r.read())
|
||||
|
||||
|
||||
def rename_one(name):
|
||||
m = RAW_RE.match(name)
|
||||
if not m:
|
||||
return None
|
||||
clean = clean_title(m.group(1))
|
||||
try:
|
||||
info = filminfo(clean)
|
||||
year = (info or {}).get("year", "")
|
||||
except Exception as e:
|
||||
print(f" filminfo fehlgeschlagen fuer {clean}: {e}")
|
||||
year = ""
|
||||
safe = re.sub(r'[\\/:*?"<>|]', "", clean).strip()
|
||||
dest = f"{safe} ({year}).mp4" if year else f"{safe}.mp4"
|
||||
src = os.path.join(ZIEL, name)
|
||||
dst = os.path.join(ZIEL, dest)
|
||||
if os.path.exists(dst):
|
||||
print(f" Ziel existiert, loesche Raw: {name}")
|
||||
os.remove(src)
|
||||
return dest
|
||||
os.rename(src, dst)
|
||||
print(f" OK: {name} -> {dest}")
|
||||
return dest
|
||||
|
||||
|
||||
def main():
|
||||
if not os.path.isdir(ZIEL):
|
||||
print(f"Ordner fehlt: {ZIEL}")
|
||||
return 1
|
||||
pending = [n for n in os.listdir(ZIEL) if RAW_RE.match(n)]
|
||||
print(f"{len(pending)} Raw-Dateien auf NAS")
|
||||
for n in sorted(pending):
|
||||
rename_one(n)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -447,14 +447,18 @@ async def _run_freitext_llm_pipeline(
|
|||
document_mode=document_mode,
|
||||
)
|
||||
)
|
||||
try:
|
||||
answer = await asyncio.wait_for(asyncio.shield(llm_task), timeout=15.0)
|
||||
except asyncio.TimeoutError:
|
||||
try:
|
||||
await update.message.reply_text("⏳ Noch dran — dauert etwas länger…")
|
||||
except Exception as te:
|
||||
log.warning("Fortschritts-Nachricht fehlgeschlagen: %s", te)
|
||||
answer = await llm_task
|
||||
waited = 0
|
||||
while not llm_task.done():
|
||||
await asyncio.sleep(10)
|
||||
waited += 10
|
||||
if not llm_task.done():
|
||||
try:
|
||||
await update.message.reply_text(
|
||||
"⏳ Noch dran (" + str(waited) + "s) — Save.TV/Modell kann etwas brauchen…"
|
||||
)
|
||||
except Exception as te:
|
||||
log.warning("Fortschritts-Nachricht fehlgeschlagen: %s", te)
|
||||
answer = await llm_task
|
||||
|
||||
if session_id:
|
||||
memory_client.log_message(session_id, "user", text)
|
||||
|
|
@ -502,14 +506,18 @@ async def _run_voice_llm_pipeline(
|
|||
document_mode=document_mode,
|
||||
)
|
||||
)
|
||||
try:
|
||||
answer = await asyncio.wait_for(asyncio.shield(llm_task), timeout=15.0)
|
||||
except asyncio.TimeoutError:
|
||||
try:
|
||||
await update.message.reply_text("⏳ Noch dran — dauert etwas länger…")
|
||||
except Exception as te:
|
||||
log.warning("Fortschritts-Nachricht fehlgeschlagen: %s", te)
|
||||
answer = await llm_task
|
||||
waited = 0
|
||||
while not llm_task.done():
|
||||
await asyncio.sleep(10)
|
||||
waited += 10
|
||||
if not llm_task.done():
|
||||
try:
|
||||
await update.message.reply_text(
|
||||
"⏳ Noch dran (" + str(waited) + "s) — Save.TV/Modell kann etwas brauchen…"
|
||||
)
|
||||
except Exception as te:
|
||||
log.warning("Fortschritts-Nachricht fehlgeschlagen: %s", te)
|
||||
answer = await llm_task
|
||||
|
||||
if session_id:
|
||||
memory_client.log_message(session_id, "user", text)
|
||||
|
|
@ -1008,28 +1016,6 @@ async def _send_daily_forecast(job_context):
|
|||
session_id=None,
|
||||
document_mode=False,
|
||||
)
|
||||
if (analysis or "").startswith(("LLM-Fehler:", "Vision-LLM-Fehler:")):
|
||||
log.error("Systemvorhersage LLM-Fehler: %s", analysis[:200])
|
||||
analysis = await asyncio.to_thread(
|
||||
llm.ask_with_tools,
|
||||
prompt,
|
||||
handlers,
|
||||
session_id=None,
|
||||
document_mode=False,
|
||||
model_override=llm.MODEL_VISION,
|
||||
)
|
||||
if (analysis or "").startswith(("LLM-Fehler:", "Vision-LLM-Fehler:")):
|
||||
await bot.send_message(
|
||||
chat_id=CHAT_ID,
|
||||
text=(
|
||||
"🔭 *Taegliche Systemvorhersage*\n\n"
|
||||
"⚠️ Analyse fehlgeschlagen (OpenRouter). "
|
||||
"Bitte spaeter erneut versuchen."
|
||||
),
|
||||
parse_mode="Markdown",
|
||||
)
|
||||
log.error("Systemvorhersage endgueltig fehlgeschlagen: %s", analysis[:200])
|
||||
return
|
||||
msg = "🔭 *Taegliche Systemvorhersage*\n\n" + analysis
|
||||
await bot.send_message(chat_id=CHAT_ID, text=msg, parse_mode="Markdown")
|
||||
log.info("Taegl. Systemvorhersage gesendet")
|
||||
|
|
|
|||
|
|
@ -43,48 +43,6 @@ DOWNLOAD_FORMAT_SD = 5
|
|||
AUTO_RECORD_SCORE = 80
|
||||
SUGGEST_SCORE = 60
|
||||
|
||||
_jellyfin_set_cache = None
|
||||
_jellyfin_set_ts = 0.0
|
||||
JELLYFIN_SET_TTL = 3600
|
||||
|
||||
|
||||
def _jellyfin_normalized_set():
|
||||
"""Set normalisierter Jellyfin-Titel (1h Cache). None = Abgleich fehlgeschlagen."""
|
||||
global _jellyfin_set_cache, _jellyfin_set_ts
|
||||
import time
|
||||
now = time.time()
|
||||
if _jellyfin_set_cache is not None and now - _jellyfin_set_ts < JELLYFIN_SET_TTL:
|
||||
return _jellyfin_set_cache
|
||||
try:
|
||||
from savetv_extra_routes import _jellyfin_fetch_library
|
||||
data = _jellyfin_fetch_library()
|
||||
titles = data.get("normalized_titles") or []
|
||||
_jellyfin_set_cache = set(titles)
|
||||
_jellyfin_set_ts = now
|
||||
log.info(
|
||||
"Jellyfin-Abgleich: %d Titel geladen (cached=%s)",
|
||||
len(_jellyfin_set_cache),
|
||||
data.get("cached"),
|
||||
)
|
||||
return _jellyfin_set_cache
|
||||
except Exception as e:
|
||||
log.warning("Jellyfin-Abgleich fehlgeschlagen: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
def _normalize_epg_title(title):
|
||||
from savetv_extra_routes import _normalize_film_title, _strip_trailing_year_in_parens
|
||||
return _normalize_film_title(_strip_trailing_year_in_parens(title or ""))
|
||||
|
||||
|
||||
def _is_in_jellyfin(title):
|
||||
"""True wenn Film (normalisierter Titel) bereits in Jellyfin-Bibliothek."""
|
||||
jf = _jellyfin_normalized_set()
|
||||
if jf is None:
|
||||
return False
|
||||
key = _normalize_epg_title(title)
|
||||
return bool(key and key in jf)
|
||||
|
||||
|
||||
def _title_pattern(title):
|
||||
"""Titel exakt genug matchen, aber Leerzeichen/Interpunktion tolerant halten."""
|
||||
|
|
@ -929,13 +887,6 @@ def get_new_films():
|
|||
if f.get("BEXISTRECORD"):
|
||||
continue
|
||||
|
||||
if _is_in_jellyfin(f.get("STITLE", "")):
|
||||
log.info(
|
||||
"Auto-Aufnahme übersprungen (bereits in Jellyfin): %s",
|
||||
f.get("STITLE"),
|
||||
)
|
||||
continue
|
||||
|
||||
if score >= AUTO_RECORD_SCORE:
|
||||
tid = int(f.get("ITELECASTID", 0))
|
||||
result = _record_telecast(tid)
|
||||
|
|
@ -965,15 +916,6 @@ def handle_savetv_record(telecast_id=0, **kw):
|
|||
title = tc.get("STITLE", title)
|
||||
break
|
||||
|
||||
if _is_in_jellyfin(title):
|
||||
return (
|
||||
"Save.TV: nicht aufgenommen — bereits in Jellyfin: "
|
||||
+ title
|
||||
+ "\n(TelecastId "
|
||||
+ str(tid)
|
||||
+ ")"
|
||||
)
|
||||
|
||||
result = _record_telecast(tid)
|
||||
return "Save.TV: " + result + "\nSendung: " + title
|
||||
|
||||
|
|
|
|||
|
|
@ -48,110 +48,3 @@
|
|||
|---|---|
|
||||
| @MutterbotAI_bot | Watchdog-Alerts |
|
||||
| @Orbitalo_Hausmeister_bot | Homelab AI-Bot |
|
||||
|
||||
## Hermes CT151 Monitoring und Backup
|
||||
|
||||
Stand: 2026-05-16
|
||||
|
||||
Hermes (`hermes-mu`, CT151) ist als kritischer Dienst eingestuft und wird separat abgesichert.
|
||||
|
||||
### Proxmox / PBS Backup
|
||||
|
||||
Backup-Job auf `pve3` / `pve-mu-3`:
|
||||
|
||||
| Feld | Wert |
|
||||
|------|------|
|
||||
| Job-ID | `hermes-mu-backup` |
|
||||
| VMID | `151` |
|
||||
| Container | `hermes-mu` |
|
||||
| Node | `pve3` |
|
||||
| Ziel | `pbs-nvme` / Muldenstein PBS (`100.99.139.22:nvme-pool`) |
|
||||
| Modus | `snapshot` |
|
||||
| Kompression | `zstd` |
|
||||
| Zeitplan | täglich `03:20` |
|
||||
| Nachholen | `repeat-missed=1` |
|
||||
| Prune | `keep-daily=7`, `keep-weekly=4`, `keep-monthly=3` |
|
||||
|
||||
Manuelles Safety-Backup wurde am 2026-05-16 erfolgreich erstellt:
|
||||
|
||||
```text
|
||||
ct/151/2026-05-16T10:51:53Z
|
||||
Dauer: 56s
|
||||
Gesichert: ca. 6.15 GiB
|
||||
Komprimiert übertragen: ca. 3.98 GiB
|
||||
Ergebnis: Backup job finished successfully
|
||||
```
|
||||
|
||||
PBS-Auslastung danach:
|
||||
|
||||
```text
|
||||
pbs-nvme active, ca. 5.63% belegt
|
||||
```
|
||||
|
||||
Schnelltest:
|
||||
|
||||
```bash
|
||||
# Auf pve3 / pve-mu-3
|
||||
pvesh get /cluster/backup/hermes-mu-backup --output-format json
|
||||
pvesm status --storage pbs-nvme
|
||||
```
|
||||
|
||||
### Prometheus / node_exporter
|
||||
|
||||
Auf CT151 ist `prometheus-node-exporter` installiert und aktiv.
|
||||
|
||||
```bash
|
||||
systemctl is-active prometheus-node-exporter
|
||||
curl -sS http://127.0.0.1:9100/metrics | grep -m1 '^node_uname_info'
|
||||
```
|
||||
|
||||
Von CT116 / Hausmeister erreichbar über:
|
||||
|
||||
```bash
|
||||
curl -sS http://100.109.174.120:9100/metrics
|
||||
```
|
||||
|
||||
### Loki / Promtail
|
||||
|
||||
Auf CT151 läuft `promtail` und schickt Logs mit Host-Label `hermes-mu` an Loki.
|
||||
|
||||
Wichtige Checks:
|
||||
|
||||
```bash
|
||||
systemctl is-active promtail
|
||||
curl -sS http://100.109.174.120:9080/ready
|
||||
```
|
||||
|
||||
### Hermes-Service-Checks
|
||||
|
||||
Aktive Dienste auf CT151:
|
||||
|
||||
```bash
|
||||
systemctl is-active hermes-gateway beellama-tunnel promtail prometheus-node-exporter
|
||||
curl -sS http://127.0.0.1:11435/v1/models
|
||||
```
|
||||
|
||||
Der lokale BeeLlama/OpenAI-kompatible Tunnel lauscht in CT151 auf `127.0.0.1:11435`; er ist deshalb von CT116 nicht direkt per HTTP erreichbar. Der Health-Check erfolgt lokal auf CT151 bzw. indirekt über Hermes/Hausmeister.
|
||||
|
||||
### Hausmeister-Integration
|
||||
|
||||
Hausmeister (`CT116`) enthält in `/opt/homelab-ai-bot/monitor.py` einen `check_hermes()`.
|
||||
|
||||
Der Check prüft:
|
||||
|
||||
- Hermes Promtail: `http://100.109.174.120:9080/ready`
|
||||
- Hermes node_exporter: `http://100.109.174.120:9100/metrics`
|
||||
- Websuche/SearXNG CT121: `http://100.74.196.29:8080/search?...`
|
||||
- Loki: `http://100.109.206.43:3100/ready`
|
||||
- PBS Muldenstein: `https://100.99.139.22:8007`
|
||||
- Loki-Health für Host `hermes-mu`
|
||||
- auffällige Hermes-Logs der letzten 30 Minuten
|
||||
|
||||
Harmloser `node_exporter`-Diskstats-Logspam (`diskstats_linux.go`, `disabling udev device properties`) wird gefiltert.
|
||||
|
||||
Finaler Test am 2026-05-16:
|
||||
|
||||
```text
|
||||
monitor.check_hermes() -> OK
|
||||
hausmeister-bot.service -> active
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1093,38 +1093,3 @@ _(reasoning: Der Memory wurde erfolgreich gespeichert. Ich bestätige das dem Us
|
|||
|
||||
Willst du noch etwas anderes speichern? 🔥
|
||||
|
||||
## Entscheidungsnotiz: Qwen3.6 + BeeLlama fuer opencode-Tests
|
||||
|
||||
Stand: 2026-05-18
|
||||
|
||||
Qwen3.6 ueber BeeLlama ist fuer einfache, kontrollierte `opencode`-Tests ausreichend:
|
||||
|
||||
- Git-Repos klonen
|
||||
- README, `package.json` und Setup-Hinweise lesen
|
||||
- Projektstruktur erkunden
|
||||
- Shell-/Git-Kommandos ausfuehren
|
||||
- Tests finden und starten
|
||||
- kleine, klar begrenzte Codeaenderungen vorbereiten
|
||||
|
||||
Wichtig: BeeLlama macht das Modell schneller und lokal praktikabler, aber nicht automatisch intelligenter. Fuer komplexe Coding-Agent-Aufgaben wie grosse Refactorings, viele Dateien gleichzeitig, schwierige Debug-Schleifen oder produktive Aenderungen bleibt ein spezialisiertes Coding-Modell zuverlaessiger.
|
||||
|
||||
Empfohlener erster Testauftrag:
|
||||
|
||||
```text
|
||||
Klon dieses Repo nach /tmp/opencode-test, lies README und package.json, aber aendere nichts. Sag danach, wie man es startet und welche Tests definiert sind.
|
||||
```
|
||||
|
||||
## Entscheidungsnotiz: zweite RTX 3090
|
||||
|
||||
Eine zweite RTX 3090 verbessert die Qualitaet desselben Modells nicht direkt. Dasselbe Modell mit derselben Quantisierung liefert grundsaetzlich dieselbe Antwortqualitaet.
|
||||
|
||||
Eine zweite GPU hilft indirekt durch:
|
||||
|
||||
- groessere Modelle, die vorher nicht in den VRAM passten
|
||||
- bessere Quantisierung, z.B. Q5/Q6 statt Q4
|
||||
- mehr Kontext/KV-Cache
|
||||
- weniger CPU-Offload
|
||||
- hoehere Tokenrate bei gutem Layer-/Tensor-Splitting
|
||||
- mehr parallele Requests
|
||||
|
||||
Fuer `opencode` waere der echte Qualitaetsgewinn daher nicht die zweite GPU selbst, sondern dass dadurch eventuell ein besseres oder groesseres Coding-Modell lokal betrieben werden kann.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue