Expandable pool question

[keith]

I'm very familiar with PCs but new to servers. I've bought components for a new home server build, including two 14tb HDD. My build supports up to 5 drives. The early access FAQ says:

• Expandable pools require a minimum of 3 devices and can be grown one device at a time.

• Non-expandable pools can be created with 2 devices.

Does this mean that I will be unable to add hard drives in the future if I only use the two drives I currently have? If I want to add drives in the future, would I then have to backup the data from the server before I can recreate a new bigger pool? I'm wondering if I should buy a third drive now to minimize the hassle it would take to add drives in the future.

[Theo]

I have the same question.

i've got 4 drives. 2 will be mirrored (Raid-1) so wouldn't need to expand.
The other 2 i'd want to run stripped (Raid-0) but with the possibility to add another drive or two in the future. but not sure if the above statements are specifically about those creating redundant pools like Raid-5/6?

[Midnight2ss]

I’m pretty sure with 2 drives it will run in raid 1 aka mirror so adding additional drives would just make it a third disk of the same thing as you can’t change a raid after it’s been created at least not with anything on them. I’m sure they will probably do that as this is for home users who do not know much about raid as it is the safest option for two drives. However on that note I wonder if you could manually make it striped array and have that expandability if your willing to accept the risks

Edited November 29, 2024 by Midnight2ss

[Theo]

1 minute ago, Midnight2ss said:

I’m pretty sure with 2 drives it will run in raid 1 aka mirror so adding additional drives would just make it a third disk of the same thing as you can’t change a raid after it’s been created at least not with anything on them. I’m sure they will probably do that as this is for home users who do not know much about raid as it is the safest option for two drives. However on that note I wonder if you could manually make it striped array and have that expandability if your willing to accept the risks

Yeah exactly. We obviously don’t know how it will function until we get it. But my hope is that we can still have striped, and those requirements are just mirror vs expandable raids. 

[brwainer]

This isn't really a HexOS thing, but an OpenZFS thing (which is the basis of TrueNAS as well)

ZFS pools are made up of one or move volumes, called zvols, and each zvol is independently responsible for data redundancy. If there is one zvol then you can pretend the rest of this doesn't exist, but if there are two or more zvols the data will be striped across them (like RAID0 except chunks of data are assigned to zvols, instead of bytes)

Making ZFS equivalents of common RAID types involves some mixing and matching.

• RAID0 - each disk is its own zvol, the pool contains one zvol per disk and allocates data across all of them
• RAID1 - a single zvol of the "mirror" type contains all disks
• RAID10 - each pair of disks is a separate mirror zvol, all the zvols are placed into the pool.
• RAID5/6 - all disks are placed in a single "raidz1", "raidz2", "raidz3" zvol, where the number indicates how many parity disks should be used
• RAID50/60 - create as many raidz zvols as you want (each with at least 3 disks) and add them all to the pool to be striped

Remember I am purely talking about OpenZFS here, the concepts below might not be implemented in HexOS' management UI, but could be done manually using the local TrueNAS Scale UI/CLI (with the appropriate knowledge expectations and risks)

• With 2 disks you can either make a RAID0 (two single-disk zvols) or a RAID1 (one mirror zvol).
• When adding a single disk to a RAID0, you are simply expanding the storage and not introducing redundancy.
• When adding two disks to an existing 2-disk RAID0, you can convert this to a RAID10 by converting the two single-disk zvols to mirror zvols.
• When adding two disks to an existing 2-disk RAID1, you can add the new disks as a second mirror zvol, resulting in RAID10
• OpenZFS has no mechanism to convert a single-disk or mirror zvol to a raidz zvol (can't convert RAID0/1 to 5). Other NAS systems that are capable of this are using mdadm or a bespoke RAID software.
• With relatively recent new code (released as Stable only a few months ago) ZFS is now able to expand raidz zvols one disk at a time, and additionally is able to move from raidz1 to raidz2 or raidz2 to raidz3 (add the new disk's space as parity instead of data)

[Theo]

9 minutes ago, brwainer said:

This isn't really a HexOS thing, but an OpenZFS thing (which is the basis of TrueNAS as well)

ZFS pools are made up of one or move volumes, called zvols, and each zvol is independently responsible for data redundancy. If there is one zvol then you can pretend the rest of this doesn't exist, but if there are two or more zvols the data will be striped across them (like RAID0 except chunks of data are assigned to zvols, instead of bytes)

Making ZFS equivalents of common RAID types involves some mixing and matching.

• RAID0 - each disk is its own zvol, the pool contains one zvol per disk and allocates data across all of them
• RAID1 - a single zvol of the "mirror" type contains all disks
• RAID10 - each pair of disks is a separate mirror zvol, all the zvols are placed into the pool.
• RAID5/6 - all disks are placed in a single "raidz1", "raidz2", "raidz3" zvol, where the number indicates how many parity disks should be used
• RAID50/60 - create as many raidz zvols as you want (each with at least 3 disks) and add them all to the pool to be striped

Remember I am purely talking about OpenZFS here, the concepts below might not be implemented in HexOS' management UI, but could be done manually using the local TrueNAS Scale UI/CLI (with the appropriate knowledge expectations and risks)

• With 2 disks you can either make a RAID0 (two single-disk zvols) or a RAID1 (one mirror zvol).
• When adding a single disk to a RAID0, you are simply expanding the storage and not introducing redundancy.
• When adding two disks to an existing 2-disk RAID0, you can convert this to a RAID10 by converting the two single-disk zvols to mirror zvols.
• When adding two disks to an existing 2-disk RAID1, you can add the new disks as a second mirror zvol, resulting in RAID10
• OpenZFS has no mechanism to convert a single-disk or mirror zvol to a raidz zvol (can't convert RAID0/1 to 5). Other NAS systems that are capable of this are using mdadm or a bespoke RAID software.
• With relatively recent new code (released as Stable only a few months ago) ZFS is now able to expand raidz zvols one disk at a time, and additionally is able to move from raidz1 to raidz2 or raidz2 to raidz3 (add the new disk's space as parity instead of data)

Yes that’s all understood, hence the questions really Will the Hex UI allow us to do what ever mention but maybe warn users, or block them completely?

[brwainer]

8 minutes ago, Theo said:

Yes that’s all understood, hence the questions really Will the Hex UI allow us to do what ever mention but maybe warn users, or block them completely?

I didn't think Keith understood it, nor was your prior post offering any explanation under the hood. Over here it is stated that users will always be able to do things directly via TrueNAS, so if someone is knowledgable or has sufficient handholding they could do the RAID0/1 -> RAID10 migration by hand. As long as the pool mounts the rest of the automation shouldn't care. Based on the other answers that a 2-drive pool won't be expandable, I don't expect them to put effort in making this a builtin option (nor does TrueNAS [EDIT: TrueNAS' interface can handle RAID1 -> RAID10 but not a simple zvol to mirror zvol as far as I know])

 

Edited November 29, 2024 by brwainer

[Theo]

My interpretation of the FAQ’s was almost saying it could potentially be locked down for people trying to create a pool with let’s say a single drive. 
 

now yes you could easily do that in the TNS Ui, but what we don’t know is how that gets translated when setting up hex as a fresh install, and what issues it might cause (if any)

[brwainer]

5 minutes ago, Theo said:

My interpretation of the FAQ’s was almost saying it could potentially be locked down for people trying to create a pool with let’s say a single drive. 
 

now yes you could easily do that in the TNS Ui, but what we don’t know is how that gets translated when setting up hex as a fresh install, and what issues it might cause (if any)

Ah I guess it would default to only allowing a RAID1 or RAID5 during initial setup, that's what I would do in the situation of trying to make a simple NAS software that protects users from most ways of shooting themself in the foot 🙂 

[Theo]

3 minutes ago, brwainer said:

Ah I guess it would default to only allowing a RAID1 or RAID5 during initial setup, that's what I would do in the situation of trying to make a simple NAS software that protects users from most ways of shooting themself in the foot 🙂 

Exactly, but let’s say someone’s setting up a media server and the content isn’t critical (downloadable) then they’d want maximum capacity, which could be annoying if forced into mirror or raid 5.

 

so one would hope if it is a limitation, is a beta thing, not permanent.  

Edited November 30, 2024 by Theo

[keith]

1 hour ago, brwainer said:

This isn't really a HexOS thing, but an OpenZFS thing (which is the basis of TrueNAS as well)

ZFS pools are made up of one or move volumes, called zvols, and each zvol is independently responsible for data redundancy. If there is one zvol then you can pretend the rest of this doesn't exist, but if there are two or more zvols the data will be striped across them (like RAID0 except chunks of data are assigned to zvols, instead of bytes)

Making ZFS equivalents of common RAID types involves some mixing and matching.

• RAID0 - each disk is its own zvol, the pool contains one zvol per disk and allocates data across all of them
• RAID1 - a single zvol of the "mirror" type contains all disks
• RAID10 - each pair of disks is a separate mirror zvol, all the zvols are placed into the pool.
• RAID5/6 - all disks are placed in a single "raidz1", "raidz2", "raidz3" zvol, where the number indicates how many parity disks should be used
• RAID50/60 - create as many raidz zvols as you want (each with at least 3 disks) and add them all to the pool to be striped

Remember I am purely talking about OpenZFS here, the concepts below might not be implemented in HexOS' management UI, but could be done manually using the local TrueNAS Scale UI/CLI (with the appropriate knowledge expectations and risks)

• With 2 disks you can either make a RAID0 (two single-disk zvols) or a RAID1 (one mirror zvol).
• When adding a single disk to a RAID0, you are simply expanding the storage and not introducing redundancy.
• When adding two disks to an existing 2-disk RAID0, you can convert this to a RAID10 by converting the two single-disk zvols to mirror zvols.
• When adding two disks to an existing 2-disk RAID1, you can add the new disks as a second mirror zvol, resulting in RAID10
• OpenZFS has no mechanism to convert a single-disk or mirror zvol to a raidz zvol (can't convert RAID0/1 to 5). Other NAS systems that are capable of this are using mdadm or a bespoke RAID software.
• With relatively recent new code (released as Stable only a few months ago) ZFS is now able to expand raidz zvols one disk at a time, and additionally is able to move from raidz1 to raidz2 or raidz2 to raidz3 (add the new disk's space as parity instead of data)

I have researched RAID configurations before and have now researched ZFS configurations. I have a much better understanding now thanks to your breakdown and the comments on this thread, thanks so much! I will likely start with two mirrored drives and in the future add two more drives, either as another mirrored pair or in raidz2.

[brwainer]

1 hour ago, keith said:

I will likely start with two mirrored drives and in the future add two more drives, either as another mirrored pair or in raidz2.

Just remember that if you want to move from a mirror of 2 drives to a raidz2 of four drives you would need to wipe and start over with the array, meaning having somewhere else to store the data temporarily.

[fozzebear]

21 hours ago, brwainer said:

Just remember that if you want to move from a mirror of 2 drives to a raidz2 of four drives you would need to wipe and start over with the array, meaning having somewhere else to store the data temporarily.

Total OpenZFS/TrueNAS newbie here. If I'm interpreting the FAQ correctly, if I start with three drives in a raidz1, I can add a fourth down the line if I want, correct? 

[Calscats]

On 11/30/2024 at 10:03 AM, Theo said:

Exactly, but let’s say someone’s setting up a media server and the content isn’t critical (downloadable) then they’d want maximum capacity, which could be annoying if forced into mirror or raid 5.

 

so one would hope if it is a limitation, is a beta thing, not permanent.  

I am somewhat in this in this situation atm. I have ~20tb (8+8+4tb) media server that I'm planning to migrate over but I don't necessarily have somewhere to store all of the data in one go while I migrate.
I was hoping to empty 1 drive at a time and convert to ZFS rather than dump everything and do it in one go.
Do drives have to be mirrored or striped or can I do JBOD style storage?
I am also planning on buying a 16tb drive earlyish next year to add so I'm not too sure how my slightly funky setup will go
As it stands can anybody speak as to whether this is feasible?

Will have to keep investigating or waiting for more features added down the line I guess

[arvinpw]

As a initial setup to HexOS, you do require to have at least 2 drives with no expandable option. Or at least 3 drives to have expandable option. And all drives have to be roughly a same size. (No mix and match like unraid)

[Maximus]

To recap my understanding of options when wanting to Expand a 2-Drives setup:

Option 1: Expand to RAID10
Add 2 new drives as a second mirror pair (zvol) to your pool, creating RAID10.

Option 2: Rebuild as RAIDZ
Recreate the pool as RAIDZ1, RAIDZ2, or RAIDZ3. Note: RAIDZ1 can be done with only 3 drives, so you may only need to buy one additional drive.
 

	RAID10	RAIDZ
Backup Required?	No	Yes
Ease of Implementation	Easy	Complex (requires pool recreation)
Usable Storage	50% of total capacity	67%+ (with RAIDZ1 on 3 drives)
Redundancy	High (tolerates 1 drive failure per mirror)	Varies (1–3 drives depending on RAIDZ level)
Performance	Faster (striping + mirroring)	Slower (parity calculations)
Cost	Requires 2 additional drives	Can start with only 1 extra drive (RAIDZ1)