[CentOS] Building a NFS server with a mix of HDD and SSD (for caching)

Tue Mar 24 16:16:25 UTC 2020
kadafax at gmail.com <kadafax at gmail.com>

Hi list,

I'm building a NFS server on top of CentOS 8.
It has 8 x 8 TB HDDs and 2 x 500GB SSDs.
The spinning drives are in a RAID-6 array. They are 4K sector size.
The SSDs are in RAID-1 array and with a 512bytes sector size.

I want to use the SSDs as a cache using dm-cache. So here what I've done 
so far:
/dev/sdb ==> SSD raid1 array
/dev/sdd ==> spinning raid6 array

I've added "allow_mixed_block_sizes = 1" to lvm.conf to be able to add 
sdb and sdd in the same VG (because of the sector size missmatch). But 
as the LVs will only use one and only one PV I guess it's OK.

# lvcreate -L 500M -n lv_cache_meta VGnfs /dev/sdb
# lvcreate -l +100%FREE -n lv_cache VGnfs /dev/sdb
# lvconvert --type cache-pool /dev/VGnfs/lv_cache --poolmetadata 
# lvcreate -l +100%FREE -n LVnfs VGnfs /dev/sdd
# lvconvert --type cache /dev/VGnfs/LVnfs --cachepool /dev/VGnfs/lv_cache
# lvconvert --cachemode writeback --type cache /dev/VGnfs/LVnfs 
--cachepool /dev/VGnfs/lv_cache
# mkfs.xfs /dev/VGnfs/LVnfs

And now I'm asking myself if I'm not doing something somewhat dangerous 
for a server that will be critical when in production :]
The server has 128GB of RAM so maybe the caching in memory will be 
sufficient to achieve good performance ?

What do you think ? Do you use something similar ?

Thanks !