How screwed am i?

[TitanNas]

Two days ago i got an email stating that there was an error on one of my spinning drives.  Then i noticed the whole server went down.  It does not seem to be booting and i cannot reach it on the network. 

 

The osd is attached below but i can't really make sense of what it is telling me.  Thank you in advance for any help you can provide. 

[Mobius]

i would start troubleshooting my unplugging all drives other than your boot drive and see if you can boot into HexOS/TrueNAS

[TitanNas]

Can get it to boot now as long as one drive that i think was the one that had 1 zfs error but it was online. One drive unfortunately isn't be recognized as being connected. The last two of the four are showing connected. Unfortunately the pool is still offline since i am missing two drives. 

 

I still need to trouble shoot with new cords and power yet. 

 

I'm going to order four new drives as well hoping i can still save it. 

 

What's anyone's recommendations to best recover this pool. 

 

 

 

I also really need to finish my off site back up. I hope i don't loose anything terrible feeling.

[TitanNas]

Also here are the nots i am getting from hexos.

Alert from TitanHexNas:
Failed to configure docker for Applications: Missing "HDDs/ix-apps/docker, HDDs/ix-apps/app_configs, HDDs/ix-apps/app_mounts, HDDs/ix-apps/truenas_catalog" dataset(s) required for starting docker.

Edited Thursday at 05:04 PM by TitanNas

[TitanNas]

The two drives that were part of the 4 drive array are showing as unused drivers.

[Mobius]

Very good chance the one drive that is preventing you from booting has gone bad.
I would also try to double check if all the cables are connected to the 2nd drive that is not showing up in hopes that it has not gone bad as well.

[TitanNas]

Update:

I exported the pool from true nas and i can get it to boot with all 4 drives connect. However it crashed immediately when trying to import.

invalid or corrupt cache file contents: invalid or missing cache file
cannot import 'HDDs': no such pool available

XXXXXXXXXXXXXXXX[~]# zpool import
  pool: HDDs
    id: 14370529907006270995
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        HDDs                                      ONLINE
          raidz1-0                                ONLINE
            6a2f9722-779d-4ad9-a174-66d6856dff9b  ONLINE
            b610e5ec-7716-4689-9f8c-e4d25b40959b  ONLINE
            5c46ab8b-82b6-436d-bcc6-e85461f55589  ONLINE
            0a46c882-dcb8-457e-8733-fd2a13e37e14  ONLINE
xxxxxxxxxxxxxxxxxxxx [~]# zpool import -f -c /dev/null HDDs
invalid or corrupt cache file contents: invalid or missing cache file
cannot import 'HDDs': no such pool available
 

xxxxxxxxxxxxxxxxxx# zpool import -f -c /dev/null 14370529907006270995

invalid or corrupt cache file contents: invalid or missing cache file
cannot import '14370529907006270995': no such pool available
 

[TitanNas]

I got it to import as read only 

 

XXXXXXXXXXXXXXXX# zpool status HDDs
  pool: HDDs
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 252K in 08:33:06 with 17 errors on Mon Jan 12 17:28:14 2026
expand: expanded raidz1-0 copied 4.65T in 12:39:26, on Fri Dec 13 05:59:05 2024
config:

        NAME                                      STATE     READ WRITE CKSUM
        HDDs                                      ONLINE       0     0     0
          raidz1-0                                ONLINE       0     0     0
            6a2f9722-779d-4ad9-a174-66d6856dff9b  ONLINE       0     0     0
            b610e5ec-7716-4689-9f8c-e4d25b40959b  ONLINE       0     0     0
            5c46ab8b-82b6-436d-bcc6-e85461f55589  ONLINE       0     0     0
            0a46c882-dcb8-457e-8733-fd2a13e37e14  ONLINE       0     0     0

errors: 17 data errors, use '-v' for a list