[Gluster-users] hardware issues and new server advice

[Strahil Nikolov]

Generally,the recommended approach is to have  4TB disks and no more than 10-12 per HW RAID.Of course , it's not always possible but a resync of a failed 14 TB drive will take eons.
I'm not sure if the Ryzens can support ECC memory, but if they do - go for it.
In both scenarios, always align the upper layers (LVM , FS) with the stripe width and stripe size.
What kind of workload do you have ?
Best Regards,Strahil Nikolov 
 
 
  On Sat, Mar 18, 2023 at 14:36, Martin Bähr<mbaehr+gluster at realss.com> wrote:   
hi,

our current servers are suffering from a weird hardware issue that
forces us to start over.

in short we have two servers with 15 disks at 6TB each, divided into
three raid5 arrays for three bricks per server at 22TB per brick.
each brick on one server is replicated to a brick on the second server.
the hardware issue is that somewhere in the backplane random I/O errors
happen when the system is under load. these cause the raid to fail
disks, although the disks themselves are perfectly fine. reintegration
of the disks causes more load and is therefore difficult.

we have been running these servers for at least four years, and the problem
only started appearing about three months ago
our hostingprovider acknowledged the issue but does not support moving
the disks to different servers. (they replaced the hardware but that
didn't help)

so we need to start over.

my first intuition was that we should have smaller servers with less
disks to avoid repeating the above scenario.

we also previously had issues with the load created by raid resync so we
are considering to skip raid alltogether and rely on gluster replication
instead. (by compensating with three replicas per brick instead of two)

our options are:

6 of these:
AMD Ryzen 5 Pro 3600 - 6c/12t - 3.6GHz/4.2GHz
32GB - 128GB RAM
4 or 6 × 6TB HDD SATA 
6Gbit/s

or three of these:
AMD Ryzen 7 Pro 3700 - 8c/16t - 3.6GHz/4.4GHz
32GB - 128GB RAM
6× 14TB HDD SAS
6Gbit/s

i would configure 5 bricks on each server (leaving one disk as a hot
spare)

the engineers prefer the second option due to the architecture and SAS
disks. it is also cheaper.

i am concerned that 14TB disks will take to long to heal if one ever has
to be replaced and would favor the smaller disks.

the other question is, is skipping raid a good idea?

greetings, martin.
________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users at gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20230321/a2aaf96c/attachment.html>