Feature Request related to Storage

[brandman]

While testing if HexOS could replace most of my current used infrastructure like NAS and so on, I  found something, which is bothering me...

As by now you can only create a pool with at least two disks, but what if I have only one? I am using HexOS virtualized on a Host with multiple disks, all of them managed by RAID - so I only have "one" disk, even if there are many physical disk behind this one virtual disk.
Not sure if this will be "fixed" in the upcoming release in Q2 - see thte "HexOS Q1 Status Update" - https://hexos.com/blog/hexos-q1-status-update

As by now my only way to create pools is not to use the HW-RAID, which I would prefer.

[PsychoWards]

Hello,

Yes single disk pools will be supported in a future update.

However, according to the official ZFS documentation you should not use HW Raid in combination with ZFS:

https://openzfs.github.io/openzfs-docs/Performance and Tuning/Hardware.html#hardware-raid-controllers

When using something like Proxmox, you can pass the SATA controller directly to the VM to allow full control over the drives and the best possible SW Raid.

[brandman]

Hi PsychoWords

Thank you for your reply. I am aware that the combination of ZFS and HW-RAID is not the best solution.
Passing the RAID controller to the VM did not work for me due to my current setup. I also need to configure every disk first in the Controller, otherwise you can't find it within proxmox as a attached disk.

The best way would be to be able to add a single disk to HexOS without using ZFS 🙂

[PsychoWards]

Hey,

Unfortunately Hexos/Truenas only supports ZFS you cannot not use it.

I understand the constraints and is a said, single drive pool support in Hexos will come, until then you unfortunately will have to wait. =/

[ubergeek]

What hyper hypervisor are you using?

[brandman]

@ubergeek I am running all my infrastructure on Proxmox.
Adding the RAID controller to the VM directly (as PsychoWards proposed) will cause my server to crash ... Only a reboot will help - I am assuming it's because I am using the RAID to control the disks on which Proxmox is installed.

[PsychoWards]

Can you give us a bit more details about your HW Raid and boot drive?

Are all your drives part of the HW Raid, your proxmox is running on the HW RAID and you are passing the unused space as a virtual drive to Hexos?

Or do you have a dedicated boot drive which is connected to the raid controller but not part of the HW RAID?

[brandman]

All drives are connected to the RAID controller:

• 2 of the as RAID 1 for the OS (Proxmox)
• 4 of them as RAID 10 for Storage (for all virtual machines and containers)
• 2 of them as RAID 1 for testing purposes as "SSD cache"
• 4 of them as RAID 5 + 1 as hotspare for testing purposes as "fast storage" - all SSDs
• 4 of them as RAID 5 + 1 as Hotspare for testing purposes as "slow storage" - spinning rust 😉
• maybe more to come in the future ... 🙂 

Everything which is for testing purposes is directly mounted to the VM (Proxmox does see this virtual disks but is not using it)

If I don't configure them in the RAID controller (ether as RAID or as standalone) I am not able to do anything with them - Proxmox does not see them, they don't appear  anywhere, like there would be nothing- so I need to configure all my physical drives first to create virtual drives to do anything with them 😄 This is the main reason I am "complaining" about HexOS asking for more drives to create a pool - all redundancy is covered by the RAID controller - no need for HexOS or TrueNAS to create any kind of pool with multiple disks ... There are some free SATA ports on my MB, but to be honest, I don't really want to connect the storage there

Due to the fact everything important is running (not yet virtualized) there no rush for me, so I will patiently wait for the upcoming release of Hex.

Thanks guys for all the tips and comments on that! This is really very much appreciated, talking to guys like you 

[PsychoWards]

Oh wow, this is quite the setup!

If you don't want to connect your boot drives to your MoBo then I don't think there is a way to passthrough the controller and still have proxmox running, but that's not needed for your setup, you are fine with your HW-Raid.

Your SSD cache, is that going to be used in Hexos/Truenas? If so, the cache is probable not doing what you think it does, it is a read cache only not a write cache. If it's used for something else, disregard this 🙂 

[brandman]

Thanks for the reply @PsychoWards 

Maybe I will remove the two disks for OS from the RAID controller and plug them directly to the MB, maybe then I could passthrough the RAID controller without issues to the VM (may be worth a try sometime in the future)
The only thing which would be "not solved" - what do I do with the 4 SSDs running as Proxmox storage for the VMs and the containers? Connect them directly too? I need to count the free SATA ports first 😄 

As I remember correctly from another topic in this forum, it should be possible to use SSD cache in HexOS in a upcoming release

[PsychoWards]

At this point I wouldn't bother, if it's running correctly and without issues, as long as you keep an eye on the drives you can use it as is without the need to use SW Raid and just rely on the HW Raid. If you decide to convert to SW Raid you need to wipe your drives.

 

Truenas, the underlying OS currently supports 2 types of cache: this is L2ARC, which is an extension of ARC and ZIL/SLOG.

The ARC is an in memory read buffer which stores files, which truenas thinks you might access next, in memory to be able to read them faster. The L2ARC is an extension where more files can be stored on an SSD to increase reading speed of those files. This is mostly useful if you have +10GBit/s networking and if you absolutely can profit from faster read speeds (video editing etc.) If however you mostly use your NAS as a backup or media player or have only 2.5Gbit/s or less, you likely don't need the L2ARC. It is a read only buffer.

ZIL/SLOG is only useful for synchronous writes and not used with asynchronous writes. If you don't know what synchronous/asynchronous writes are you don't need SLOG 🙂

Hexos might support those 2 types (or just L2ARC) but it's not a write buffer and a normal write buffer is currently not foreseen to get in Truenas. 

I hope this was understandable  🙂