PsychoWards

If you can still afford to wipe your drives, you can setup RaidZ2 in Truenas.

At this point I wouldn't bother, if it's running correctly and without issues, as long as you keep an eye on the drives you can use it as is without the need to use SW Raid and just rely on the HW Raid. If you decide to convert to SW Raid you need to wipe your drives. Truenas, the underlying OS currently supports 2 types of cache: this is L2ARC, which is an extension of ARC and ZIL/SLOG. The ARC is an in memory read buffer which stores files, which truenas thinks you might access next, in memory to be able to read them faster. The L2ARC is an extension where more files can be stored on an SSD to increase reading speed of those files. This is mostly useful if you have +10GBit/s networking and if you absolutely can profit from faster read speeds (video editing etc.) If however you mostly use your NAS as a backup or media player or have only 2.5Gbit/s or less, you likely don't need the L2ARC. It is a read only buffer. ZIL/SLOG is only useful for synchronous writes and not used with asynchronous writes. If you don't know what synchronous/asynchronous writes are you don't need SLOG 🙂 Hexos might support those 2 types (or just L2ARC) but it's not a write buffer and a normal write buffer is currently not foreseen to get in Truenas. I hope this was understandable 🙂

Oh wow, this is quite the setup! If you don't want to connect your boot drives to your MoBo then I don't think there is a way to passthrough the controller and still have proxmox running, but that's not needed for your setup, you are fine with your HW-Raid. Your SSD cache, is that going to be used in Hexos/Truenas? If so, the cache is probable not doing what you think it does, it is a read cache only not a write cache. If it's used for something else, disregard this 🙂

Can you give us a bit more details about your HW Raid and boot drive? Are all your drives part of the HW Raid, your proxmox is running on the HW RAID and you are passing the unused space as a virtual drive to Hexos? Or do you have a dedicated boot drive which is connected to the raid controller but not part of the HW RAID?

Hey, Unfortunately Hexos/Truenas only supports ZFS you cannot not use it. I understand the constraints and is a said, single drive pool support in Hexos will come, until then you unfortunately will have to wait. =/

Hello, Yes single disk pools will be supported in a future update. However, according to the official ZFS documentation you should not use HW Raid in combination with ZFS: https://openzfs.github.io/openzfs-docs/Performance and Tuning/Hardware.html#hardware-raid-controllers When using something like Proxmox, you can pass the SATA controller directly to the VM to allow full control over the drives and the best possible SW Raid.

I don't know if anyone else in the Hexos community is using M-Disc as a way of backing up data (I had to Google M-Disc, genuinely didn't know what it was). People commonly use one of following 3 things: 1. Backup to another NAS (might be another Hexos, Truenas, Unraid, Synology, QNAP etc...) Hexos will offer buddy backup to another Hexos server in the future however no details are known so far. 2. Backup into a cloud. (you can encrypt the data before uploading if privacy is a concern) 3. Backup onto an external drive (USB HDD or SSD), this however is currently breaking Hexos. You can also use a combination of any of those. The big advantage of doing this, is you can program it to be done automatically, you can choose the frequency, you can have versioning of files which is great if you overwrote a file and want to restore a previous version etc. It certainly does add additional costs compared to M-Disc but you don't need to do anything manually and you got backups of way more data and way more often compared to doing it manually. And your second server doesn't need to be as powerful and therefore expensive as your main server.

Hey, 1. You can just use Immich, it is a one click install app in Hexos and it will take care of your photo management. It basically works like google photo, but it's running locally on your server, you can create multiple users and share folders, there is an app which is automatically backing up all your pictures from your smartphone, it takes care of everything for you. 2. Truenas, the underlying system supports a lot of apps, from game servers to surveillance apps, so Hexos will be able to run those things as well. In the recent Roadmap it was announced that community templates for apps will come, meaning the number of one click install apps will likely sky rocket with this feature and most commonly used apps will likely rather quickly get a template. Out of the box, Hexos is only usable in the internal network, as is every NAS software, but you can setup a VPN, or Tailscale or a reverse proxy with a domain to make the required services reachable over the Internet. You can even create a VM in Truenas to run Home Assistant OS if this is system which you are interested in. So yes, Hexos can do everything you want, but it's still in beta and some features are currently only accessible via truenas. If you decide to go with Hexos here are a couple of tips: - the boot drive can only be used as a boot drive and you cannot store any data on it, meaning there is no point of buying a 1 TB SSD. - Once you decided for your ZFS pool type (mirrored with 2 disks) RaidZ1 (with 3 or more disks) you cannot change it afterwards and are stuck with this ZFS type, thus plan accordingly. You can also use RaidZ2 (2 drive redundancy) but you need to create the pool in Truenas.

I have to thank Synology for their current line-up, since without them I wouldn't be here. I have an old DS412+ which I was looking to replace a couple of months ago with a new Synology model. However their line up was so bad, that I decided to go down the custom NAS route, which eventually made me end up here. And I do know from other people which are now looking for alternatives because Synology just doesn't offer what they want. So, yes I agree, it's definitely not an exciting refresh and some people might be disappointed, just as I was and switch to another solution, maybe this makes someone else switch to Hexos. I do think that it's an exciting time for NAS in general and this line up only helps to push some people over the edge and start looking for alternatives.

You cannot install Hexos on any Synology NAS, it's locked down and does not support installing 3th party OS, you can only use the Synology own DSM with it. You might find some guides, but it's pretty janky and really not worth the hassle, if you want to use Hexos you need to look for another device.

Also Noctua stats that the CPU cooler should only be used with care on CPUs with more then 65W, your CPU has 95W: https://noctua.at/en/nh_l9i_tdp_guidelines

Hey, A NAS is a Network Attached Storage, so the goal of it is to have it connected to the network and not just a single device else it would use an USB connection or sth. Similar. 🙂 You can connect your PC directly to the server, but that is not the intended use case and requires some setup. The Internet connection also does not play a role in this case, the only important thing is the internal network and as long as you have a cable connection, the internal network should be fast enough. The recommended way in your use case is to connect your NAS and PC to the network instead of each other directly. If you have multiple ethernet ports in your wall outlet you can connect both devices separately, if you have only 1 port, you can add a switch, and then connect both PC and NAS to this switch. This way, every device on the local network (PCs, Smartphones, TVs, TV Media Player etc.) can access the NAS for backups and reading/streaming.

I just had a brief look, but this looks to be very interesting, both the article and the tool. This will definitely help with understanding ZFS better! Thanks for posting it.

First of all it seems to be a very thought out setup which will be a good inspiration for the community in general. But you are running Proxmox (so do I) and a Proxmox Backup server (which I also wanted to have a look at), you have some scripts running, Uptime Kuma will be one of the apps which I will now also install and in general seems like there's a lot to learn. Also you have a Rack. I want a Rack as well.

You absolutely need to create your own topic. From your posts in the forum it sounds like you have a some things which I will shamelessly copy from your setup =D

We need to distinguish between the boot drive dying and some other HW dying. If something else dies, you just switch HW and you are good to go. If the boot drive dies or you reinstall hexos from scratch it's a bit different but those 2 scenario are functionally the same thing. Since Hexos currently doesn't support pool import during installation and is insisting on creating new pools, this currently not that easy to do with hexos. With Truenas (and hopefully Hexos as well once pool import is supported) it's as easy as it gets. You just need to backup your truenas config (no need to backup the boot drive), then you just reinstall truenas, you import your config and the system is up and running as if nothing ever happened. This is working because all the information about the pools and apps are stored on the pool drive itself and the boot drive only contains your config.

If you have an use case for additional VMs just go for it. However be aware that you have an additional layer which needs to be properly setup and configured and that some things might not be as straightforward as with a bare metal installation. 🙂

Pretty much exactly what @Sonic says. Hexos in a VM only makes sense if you want to run multiple OS on the same machine and if the VMs are the important part for you. If you primarily want a NAS and running other OS in a VM is not important and the built-in apps are enough for you, go bare metal there is no point in going down the VM part. If, however, VMs are important to you and Hexos is only a piece of your setup and not the most import part, proxmox or similar is the way to go.

If we look at the specs for the Seagate Ironwolf Pro, they are rated for up to 550TB/year, which is a ludicrous amount of data and which you will never write in the lifetime of the disk. I wouldn't worry about writing surveillance data to the drives, that's what the are ment for (writing and reading data 24/7) and it's likely not going to impact the life at all, or at least not in a significant way. Usually the other alternative for home surveillance would be micro sd cards and those die very fast compared to the lifetime off your surveillance setup and you need to swap them out every other year to be safe.

In Truenas under Data Protection there is an area for replication tasks, which is the preferred way to backup from zfs to zfs.

Lucky you, I had 3 HDDs and 3 SSDs dying on me, 1 HDD was DOA and have 3-4 HDDs with SMART errors indicating they will soon kick the bucket. 1 of the dead HDD and the DOA were WD Red (CMR) and Red Pro, all the other were consumer grade and I got plenty of lifetime out of them. Thankfully I never lost any important data or data which I cared about.

Hexos will write your USB Flash Drive to death very fast, that's why they are not recommended because of the constant writes which Truenas is doing to the drive (Logs, Graphs and the system dataset for the both drive). Also the performance might not be the best. It is definitely possible but you should definitely backup your system config regularly and don't wait multiple years before replacing the drive. 🙂

Hey, I just did 24h Mem Test, to be sure that the Memory is stable, but I didn't do any HDD burn in. My data is coming from a working NAS and I still have multiple copies of the imported files and I'm running a RaidZ2, so I'm taking my chances and hope that I'm not gonna lose 3 drives at the same time 🙂 Technically, I would need to loose at least 9 drives in various different NAS all at the same time before I start loosing my most important data. So yes, a dead drive is really more a nuisance at this point then a real issue with my current setup.

Used NAS are probably more expensive then used PCs. Also with an used NAS you might not have any upgrade possibilities. Additionally a Synology NAS will not work, because you are limited to there own OS. Also you nees to make sure to get a NAS with enough memory (RAM) and x86 CPU. So your best bet, in my opinion is to go for a PC, because there you have less restrictions and less things to worry about, only that it meets the minimum recommendation (especially the 8GB memory bit) and that it's not using a Realtek NIC (network chip, because most of them are not supported by truenas). Last but not least, don't forget that you need a dedicated boot drive, the boot drive cannot be used as a data drive, it only contains the OS and you cannot store anything on it. Ideally you would use a small SSD. A HDD would also work but not an USB stick.

Good point, I believe the same build with the 14600K and without dedicated GPU would be a good alternative for less demanding users and/or users on a tighter budget. Yes, I completely understand you there also some things just don't run on an iGPU, looking at you Immich...