M.2 SSDs for caching ?

[Thejs1234]

Hi,

I'm almost done building my NAS with the following hardware : 

• ASUS B650-PLUS
• AMD Ryzen 5 7600
• Corsair DDR5 32Gb (2x16Gb) 6000MHz
• Noctua NH-U12S (in chromax.black, soooooo cool)
• Fractal Design Meshify 2
• Seasonic Vertex PX-750
• 4x Seagate IronWolf ST12000VN0007 12TB HDD

My motherboard has room for 3 M.2 SSDs, I already use one of the slots for my boot drive.

Knowing that HexOS (rather TrueNAS) uses RAM caching, I wanted to know in which cases does SSD caching help ? Does it improve writing speeds ? And reading speeds ?

 

Thanks !

[PsychoWards]

SSD caching in Truenas works probably very differently from what you expect them to.

You have the possible to add the SSDs as L2ARC (ARC is responsible for keeping the frequently used files in memory and to predict and preload files which you are going to use. This is usually done in Memory, but you can expand it using a drive), this can improve read speeds, if the ARC isn't hitting the correct files but the L2ARC is, but only if you are using 10GBits network, or else the network will very likely be the bottleneck anyway. https://www.truenas.com/docs/references/l2arc/

Or you can add it as a SLOG which is used for synchronous writing. This is quite a complexe topic however. https://www.truenas.com/docs/references/slog/

If you are not using 10Gbit Networking and not using a lot of sycn. Writes I wouldn't bother and just either get a second boot drive to put as a mirror or make a "faster" mirrored pool with 2 SSDs.

If you are using 10Gbit Networking, I still would not bother and go for a mirrored boot drive or mirrored fast SSD pool.

Either way, L2ARC and SLOG/ZIL are the caching possibilities which you have.

 

Edited January 2 by PsychoWards

[Thejs1234]

21 hours ago, PsychoWards said:

SSD caching in Truenas works probably very differently from what you expect them to.

You have the possible to add the SSDs as L2ARC (ARC is responsible for keeping the frequently used files in memory and to predict and preload files which you are going to use. This is usually done in Memory, but you can expand it using a drive), this can improve read speeds, if the ARC isn't hitting the correct files but the L2ARC is, but only if you are using 10GBits network, or else the network will very likely be the bottleneck anyway. https://www.truenas.com/docs/references/l2arc/

Or you can add it as a SLOG which is used for synchronous writing. This is quite a complexe topic however. https://www.truenas.com/docs/references/slog/

If you are not using 10Gbit Networking and not using a lot of sycn. Writes I wouldn't bother and just either get a second boot drive to put as a mirror or make a "faster" mirrored pool with 2 SSDs.

If you are using 10Gbit Networking, I still would not bother and go for a mirrored boot drive or mirrored fast SSD pool.

Either way, L2ARC and SLOG/ZIL are the caching possibilities which you have.

 

Thanks ! As for now, my server is only plugged in via 2.5GbE so SSD caching doesn't seem to be useful for me. I could add two other SSDs which would indeed make a fast SSD pool (probably useful for my Time Machine backups ?). I'll see...

[dinecoj]

1 hour ago, Thejs1234 said:

I could add two other SSDs which would indeed make a fast SSD pool

Something to consider is that you may be network bottlenecked with your HDDs already 🙂 

My SSD pool is faster than my 10Gb networking

[Thejs1234]

3 hours ago, dinecoj said:

Something to consider is that you may be network bottlenecked with your HDDs already 🙂 

My SSD pool is faster than my 10Gb networking

That’s for sure!

Edited January 3 by Thejs1234