Jump to content

Dasbaum

Members
  • Posts

    1
  • Joined

  • Last visited

Everything posted by Dasbaum

  1. I'm having the exact same issue on 25.10.2.1 — so the update alone doesn't fix it. I dug into it and found the actual root cause: The ix-vendor.service runs /usr/bin/start_vendor_service on boot, which calls vendor_service.py. This script starts a transient systemd unit called websocat.service (the websocket bridge between your local middleware and deck.hexos.com). The problem is that the script doesn't check whether websocat is already running before calling systemd-run --unit=websocat. If websocat is already active, the call fails with "Unit websocat.service was already loaded", but the script always exits with code 0 (it has a finally: sys.exit(0) block). Something then re-triggers ix-vendor, it fails again, and you get a rapid restart loop. Each loop iteration spawns a Python process that opens Docker sockets, and since the middlewared service only has a soft file descriptor limit of 1024, the FDs get exhausted quickly — hence the "Too many open files" error. Here's the fix that worked for me in truenas shell — a systemd override that adds a simple check before running the script: sudo mkdir -p /etc/systemd/system/ix-vendor.service.d/ sudo tee /etc/systemd/system/ix-vendor.service.d/no-loop.conf << 'EOF' [Service] ExecStart= ExecStart=/bin/bash -c 'if systemctl is-active --quiet websocat.service; then echo "websocat already running, skipping"; exit 0; fi; /usr/bin/start_vendor_service' EOF sudo systemctl daemon-reload sudo systemctl restart ix-vendor.service After this, ix-vendor shows active (exited) with status 0, websocat keeps running, and deck.hexos.com works fine. No more loop, no more FD exhaustion. I also raised the middleware FD limit as an extra safety net: sudo mkdir -p /etc/systemd/system/middlewared.service.d/ sudo tee /etc/systemd/system/middlewared.service.d/fd-limit.conf << 'EOF' [Service] LimitNOFILE=65536 EOF sudo systemctl daemon-reload sudo systemctl restart middlewared Both overrides live in /etc/systemd/system/ which - as far as I understand it "so far" - survives reboots. After a HexOS update you may want to verify they're still in place. Hopefully the HexOS team can add the is-active check to vendor_service.py itself so this gets fixed upstream.
×
×
  • Create New...