PsychoWards

No, VPNs are generally considered one of the, if not the safest option to connect to your self hosted stuff. The Deck website will stay available in the future, the local deck will be added as an alternative and not as a replacement to the existing website.

I've made some progress, Immich is installed and up and running and currently using my PCs GPU for the ML stuff... My current plan is to get my old 2080Ti installed and working in the server itself. PaperlessNGX is now also running, which I use to store and archive all sorts of documents. I've made the Hexos Gods very angry at me, by create a single drive pool with an external USB drive, which I use as a replication task target of some of my datasets. Next on the list is getting rsync setup to my UNRAID and Synology NAS and at that point I think I should have enough backups of my most important files. I'll post some pictures once I get to install the GPU.

The 128GB ECC Memory would be a compelling argument for the server HW (but unfortunately the only compelling you, but a good one)

@Ioannis pretty much spot on, you would just select import pool instead of create pool in Truenas.

Schweizer Franken, not that uncommon of a currency 🙂

Hey @Ioannis I'm happy that I could help. 🙂 Concerning your storage usage, if you decide to look into Plex and to store your existing DVDs/Blurays as well you can quickly use up another 1-2TB, just worth a consideration. About the need for the windows machine, is there nothing similar which you can run in Docker which does the same thing? And last but not least, take your time, it's supposed to be a fun ride and not a stressful one.

Hey @Ioannis Hexos/Truenas is using ZFS and not normal RAID, meaning there is different wording and numbers involved in describing the different redundancy modes. ZFS Raids are called as fallowing: Stripe: similar to Raid0 (the Data is split across 2 drives, if 1 drive dies you lose everything) Mirrored: similar to Raid1 (the Data is mirrored on both drives you can loose 1 drive) RaidZ1: similar to Raid5 (you need at least 3 drives and can loose 1 drive without loosing data) RaidZ2: you can loose 2 drives without loosing data RaidZ3: you can loose 3 drives. As you see Raid1 and RaidZ1 are very different things and in this forum you will most likely only find the ZFS naming schema and I will use it in this post as well. Already today, during the installation, or afterwards in Truenas, you can create mirrored boot drives with 2 drives, this is supported out of the box and not difficult to setup at all. If you want to have the piece of mind just chuck in an additional 256GB NVME SSD in there and you are good to go. Just avoid HW Raid when using Truenas, let Truenas/ZFS take care of the Raid. Concerning your Networking, it appears that Realtek NICs are not supported, that might be why your motherboard ethernet port isn't working. If this is the case you can check if you can install a PCIe network card, an USB network card is not the optimal solution. If you only use 2 drives, Hexos will create a mirrored Pool which you cannot extend afterwards. Once you have a mirrored Pool you are stuck with 2 drives. You might want to consider adding a 3 drive now, to create a RaidZ1 Pool, have 12TB of usable storage and still have the ability to extend in the future (which you might not need, but it's worth considering since once you go with the mirrored pool you are stuck with it) Dual Boot with Windows, what is your use case? I suppose if it's your family NAS your family want to be uploading/browsing there images, or watch movies using plex all the time, they don't take it to kindly if they cannot watch their movies because you need to do sth. in Windows. I would suggest running Windows in a VM in Hexos/Truenas if this would be a valid approach. Concerning SSH into Linux/Truenas, you first need to enable ssh for your user/truenas admin, enable ssh itself and then you can use putty or the onboard windows terminal using: ssh truenas_admin@<Hexos-IP> And then enter your password and you are connected via SSH with the server.

It's not a backup, it's a mirror, meaning everything is stored on both drives at the same time and any change is written to both drives, there is never a delta between both drives. Meaning there is not a first/second drive, it's on both and any one drive can die (or unplug for that matter) and accept Truenas shouting vary angrily at you that 1 drive is missing, everything is still working as if nothing ever happened. You don't even need to do anything at this point, Truenas is handling everything automagically. Once you got a new replacement drive, you put it in, setup the mirror in Truenas/Hexos again and you got full redundancy again.

There is no dedicated admin interface, 2283 is the normal port which truenas seems to be mapping to 30041. To access the admin panel you click on the top right where your user icon is and from the pop up menu you can select Administration. This does not work in the App only in a browser. You are not supposed to do that, you should upload your photos via the app/gui.

I suppose you mean when your boot drive dies how do you recover your data from your data drives? As long as you have not lost more drives then your pool type can recover from, you can chuck the drives into any Truenas, Hexos (once pool import is supported) or any other OS supporting ZFS, import the pools (the ZFS information are all stored on the data drives) and you can access all your data from those drives.

That is very interesting, what's your intended use case if I may ask? I've spent some time thinking about L2ARC and SLOG/ZIL but I have never found any viable usecase for me, but I'm also new to those concepts.

Data will not be retained when creating a new pool. With 2 drives Hexos will only give you the option to have a mirrored setup, with 3+ drives Hexos will create a RaidZ1. There is currently no way to expand a mirrored to a RaidZ1 pool and I highly doubt that this will ever be possible. If you have a 2 drive mirrored pool and want to extend it, you need to backup all your data, create a new RaidZ1 pool (with all your drives) and if you want to add more drives in the future you can then do it.

Oh, that's awesome to hear. I've a N100 running Opnsense with only 8GB of memory, doubt I will ever need more, but it's really helpful information, I can imagine a lot of people are running N100 in there NAS. Would you mind sharing which module you got? 🙂

Can confirm, that's what I did and I have mirrored boot drives working flawlessly.

SSD caching in Truenas works probably very differently from what you expect them to. You have the possible to add the SSDs as L2ARC (ARC is responsible for keeping the frequently used files in memory and to predict and preload files which you are going to use. This is usually done in Memory, but you can expand it using a drive), this can improve read speeds, if the ARC isn't hitting the correct files but the L2ARC is, but only if you are using 10GBits network, or else the network will very likely be the bottleneck anyway. https://www.truenas.com/docs/references/l2arc/ Or you can add it as a SLOG which is used for synchronous writing. This is quite a complexe topic however. https://www.truenas.com/docs/references/slog/ If you are not using 10Gbit Networking and not using a lot of sycn. Writes I wouldn't bother and just either get a second boot drive to put as a mirror or make a "faster" mirrored pool with 2 SSDs. If you are using 10Gbit Networking, I still would not bother and go for a mirrored boot drive or mirrored fast SSD pool. Either way, L2ARC and SLOG/ZIL are the caching possibilities which you have.

Concerning question 1: Nowadays, the memory controller is baked into the CPU, so the CPU defines the supported Memory speed and quantity (mainly). In this case it is the 2666 which was mentioned and since everything higher then this is considered overclocking, and a lot of H370 boards don't support overclocking, that's why you are likely limited to this speed. Some Mid- to High End H370 boards support overclocking but I doubt that an OEM board does. Therefore, don't go with faster RAM and take more RAM instead.

Oh boy I was thinking about the 5600... Good catch with the PSU size! Thanks for correcting those things 🙂 You are right about the RAM it's not needed, but it's wasted potential, if you already go with relativ newish HW, you might want to consider using it to it's full potential. Also +1 for Intel CPU with QuickSync, that's the only reason why I went with Intel instead of AMD in my built.

Hey, The Ryzen 5 7600 does not have an integrated GPU, so make sure you have one ready when installing Hexos if you decided to use that CPU. If you only use 1 RAM stick you will only get half of the bandwidth of the memory, because it is running in single channel. If you want to have the full speed you need at least 2 modules so that they can run in dual channel. Since Hexos is doing a lot of caching in memory, you probably want to go dual channel to maximise performance. AM5 is using DDR5, which does include on-die ECC (Error Correction Code) and can correct a 1 bit error in the memory itself. It is, however, not the same kind of EEC which is used in server applications, which can detect bit flips outside of the memory as well. Powersupply: make sure to use a well known brand with gold or platinum rating, the Corsair RM650 for example can be had for around 90€, you don't need the 150€ which you have planned. Motherboard: you can always get an HBA card to add more SATA ports, however in that case you cannot add a dedicated GPU. But for most things, the integrated GPU should be more then enough. Another possibility would be a m.2 to SATA port adapter, which you plug in into an M.2 port, so there are possibilities on how to get more ports to your motherboard. M.2 SSDs: take something which doesn't have a lot of storage, since you cannot use it for anything else in Hexos, it will only run the OS.

@Ornival thank you so much for the very detailed explanations and remarks. This helps me a ton. Funny enough, my Server is already running in Proxmox anyway, so that part is already done =D I forgot to mention this earlier, but my Immich is running in Docker in an Ubuntu VM, it's not running in Hexos itself. I still have an 1060 with 3GB VRAM (which is really week both GPU an VRAM) and an 2080Ti, which however crashes while gaming. But maybe only the rasterization part is causing issues, the CUDA part may be fine, I'll have to test this. If the 2080Ti isn't usable, I will go on the hunt for one of your recommended GPUs if I find a good deal. I don't think that a lot of VRAM will be required for my use cases, because must programms with machine learning capabilities for HomeLabs are meant to run on any kind of HW and to not specifically need a high end GPU with a lot of VRAM. Another example would be paperless ngx, which also uses machine learning but doesn't even support HW acceleration. So I technically don't require a GPU, but the thinkering is the fun part.

I believe he meant as a boot drive not as the storage drives 🙂 If so, a HDD as a boot drive shouldn't be an issue, at least not with Truenas itself, but I don't believe that Hexos adds much overhead which would require an SSD to work properly. The NVME/M.2 SSD is often recommended because it frees up 1 (or in case of a mirror 2) SATA port, which there are not many left on modern consumer grade HW and because they aren't that expensive anymore and consum less energy. The boot might take longer, but most of the OS runs in Memory anyway. So if you are fine with longer boot time, decent HDDs should be fine.

Don't buy Synology, you can only use their own DSM OS but not Hexos. Make sure it has an x86 CPU (Intel or AMD) and not an ARM CPU (Hexos doesn't run on ARM) and that it has a video output.

Normally, the boot drive dying isn't an issue (except for the fact that you have to buy a new drive) as long as you have a backup off your settings. You would just reinstall the OS, import the backup and you are good to go. However since Hexos is still in Beta, this is currently not possible, there is no option to skip the pool creation during setup. This will come in the future but it is not there yet. Until then, if your boot drive stops working, your are going to install truenas directly and import the pools (the pool information is stored on the drives, which means you can directly read it from them) this way you can still access your data, however you have lost the Hexos part and can only switch back to Hexos once it supports pool import. Since the data drives using ZFS you have to read them as such, you cannot just mount them individually and try to get data from them. But as I said a boot drive dying will not result in data loss, you would only need to switch to truenas for the time being.

Hey, thank you so much for this detailed answer. My current setup consists of an i5 14600K and no dedicated GPU in the server and Immich is connecting to my PC to have the ML stuff run on my 3080Ti. The "problem" however is that my PC is not running 24/7 and that I don't want Immich to hit my GPU while i'm gaming. Therefore I thought maybe a dedicated GPU would be helpful for this part. I don't want to upgrade my GPU, especially since it has a water cooling block installed, and I don't want to put a customer water cooling loop in my server. My only current usecase is indeed Immich, (which does not support ML on the iGPU) but in the feature some AI recognition for security cameras might be added as well. I'm not looking to get a high end GPU, but maybe something lower class or used Nvidia P model GPUs (P40) or sth. Similar in the 300€/$ price range, since they have way faster single and double FP performance then any high end 4000/5000 series GPU. For transcoding I'm using my iGPU anyway.

It depends on your use case and what you need. Do you mainly need the NAS and the VMs are just "nice to have" or are the VMs important and Hexos is just one part/VM of many for you? If I see that you want to spin up Ubuntu and Windows, you might as well go with Proxmox and Hexos as a VM, however this comes with it's own challenges. A lot of guides on how to install truenas are outdated/recommend suboptimal settings and you can only pass-through your GPU to 1 VM, meaning if you pass through your GPU to hexos for plex streaming, you can not use it in your ubuntu VM for machine learning in Frigate NVR (this is just an example, not sure Frigate NVR uses GPU acccleration). But on the plus sidey Proxmox is specifically developed to manage VMs and you typically find a lotnof guides and help for itm Whereas if you use barebone Hexos, you don't have to fiddle with the VM pass-through stuff for Hexos, but the VM managing part might be less optimal and less performant. According to the Intel spec page, your CPU supports virtualisation. However, it will likely be on the week side if you want to run those 3 OSes together. Also Hexos requires minimum 8GB of RAM, meaning you need to significantly upgrade your memory. Your CPU supports max up to 32GB of RAM and that is probably what you should aim for, since 16GB might be to little.

Exactly, Hexos is just a simple NAS with a ZFS Raid in my setup, where everything will be stored at and which is going to do backups to different drives/servers. I will likely not use any of the fancy app or VMs features, I handle those things in VMs or opnsense directly.