Jump to content

Leaderboard

Popular Content

Showing content with the highest reputation on 03/13/26 in all areas

  1. I'm having the exact same issue on 25.10.2.1 — so the update alone doesn't fix it. I dug into it and found the actual root cause: The ix-vendor.service runs /usr/bin/start_vendor_service on boot, which calls vendor_service.py. This script starts a transient systemd unit called websocat.service (the websocket bridge between your local middleware and deck.hexos.com). The problem is that the script doesn't check whether websocat is already running before calling systemd-run --unit=websocat. If websocat is already active, the call fails with "Unit websocat.service was already loaded", but the script always exits with code 0 (it has a finally: sys.exit(0) block). Something then re-triggers ix-vendor, it fails again, and you get a rapid restart loop. Each loop iteration spawns a Python process that opens Docker sockets, and since the middlewared service only has a soft file descriptor limit of 1024, the FDs get exhausted quickly — hence the "Too many open files" error. Here's the fix that worked for me in truenas shell — a systemd override that adds a simple check before running the script: sudo mkdir -p /etc/systemd/system/ix-vendor.service.d/ sudo tee /etc/systemd/system/ix-vendor.service.d/no-loop.conf << 'EOF' [Service] ExecStart= ExecStart=/bin/bash -c 'if systemctl is-active --quiet websocat.service; then echo "websocat already running, skipping"; exit 0; fi; /usr/bin/start_vendor_service' EOF sudo systemctl daemon-reload sudo systemctl restart ix-vendor.service After this, ix-vendor shows active (exited) with status 0, websocat keeps running, and deck.hexos.com works fine. No more loop, no more FD exhaustion. I also raised the middleware FD limit as an extra safety net: sudo mkdir -p /etc/systemd/system/middlewared.service.d/ sudo tee /etc/systemd/system/middlewared.service.d/fd-limit.conf << 'EOF' [Service] LimitNOFILE=65536 EOF sudo systemctl daemon-reload sudo systemctl restart middlewared Both overrides live in /etc/systemd/system/ which - as far as I understand it "so far" - survives reboots. After a HexOS update you may want to verify they're still in place. Hopefully the HexOS team can add the is-active check to vendor_service.py itself so this gets fixed upstream.
    1 point
  2. Hello, One way that could fix this is to do the following. Update to TrueNAS version 25.10.2.1. which applies permanent resource limit fixes. Thank you
    1 point
  3. It seems like the TrueNAS Scale build repo on GitHub is "no longer actively maintained": https://github.com/truenas/scale-build Will this affect HexOS in the long term?
    0 points
×
×
  • Create New...