[Gluster-users] State of Gluster project

[Stefan]

Hi,

we use both Ceph and GlusterFS quite a bit.
It is very hard to compare them because they are structurally very different. For example, GlusterFS performance will depend *a lot* on the underlying file system performance. Ceph eliminated that factor by using Bluestore.
Ceph is very well performing for VM storage, since it's block based and as such optimized for that. I haven't tested CephFS a lot (I used it but only for very small storage) so I cannot speak for its performance, but I am guessing it's not ideal. For large amount of files thus GlusterFS is still a good choice.

One *MAJOR* advantage of Ceph over GlusterFS is tooling. Ceph's self-analytics, status reporting and problem fixing toolset is just so far beyond GlusterFS that it's really hard for me to recommend GlusterFS for any but the most experienced sysadmins. It does come with the type of implementation Ceph has chosen that they have to have such good tooling (because honestly, poking around in binary data structures really wouldn't be practical for most users), but whenever I had a problem with Ceph the solution was just a couple of command line commands (even if it meant to remove a storage device, wipe it and add it back), where with GlusterFS it means poking around in the .glusterfs directory, looking up inode numbers, extended attributes etc. which is a real pain if you have a multi-million-file filesystem to work on. And that's not even with sharding or distributed volumes.

Also, Ceph has been a lot more stable that GlusterFS for us. The amount of hand-holding GlusterFS needs is crazy. With Ceph, there is this one bug (I think in certain Linux kernel versions) where it sometimes reads only zeroes from disk and complains about that and then you have to restart that OSD to not run into problems, but that's one "swatch" process on each machine that will do that automatically for us. I have run some Ceph clusters for several years now and only once or twice I had to deal with problems. The several GlusterFS clusters we operate constantly run into troubles. We now shut down all GlusterFS clients before we reboot any GlusterFS node because it was near impossible to reboot a single node without running into unrecoverable troubles (heal entries that will not heal etc.). With Ceph we can achieve 100% uptime, we regularly reboot our hosts one by one and some minutes later the Ceph cluster is clean again.

If others have more insights I'd be very happy to hear them.

Stefan


----- Original Message -----
> Date: Tue, 16 Jun 2020 20:30:34 -0700
> From: Artem Russakovskii <archon810 at gmail.com>
> To: Strahil Nikolov <hunter86_bg at yahoo.com>
> Cc: gluster-users <gluster-users at gluster.org>
> Subject: Re: [Gluster-users] State of Gluster project
> Message-ID:
>	<CAD+dzQdf_TiPBSDj57hY=t8AQ=mACrxinPX7iU4hmuxNMo+omg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Has anyone tried to pit Ceph against gluster? I'm curious what the ups and
> downs are.
> 
> On Tue, Jun 16, 2020, 4:32 PM Strahil Nikolov <hunter86_bg at yahoo.com> wrote:
> 
>> Hey Mahdi,
>>
>> For me it looks like Red Hat are focusing more  on CEPH  than on Gluster.
>> I hope the project remains active, cause it's very difficult to find a
>> Software-defined Storage as easy and as scalable as Gluster.
>>
>> Best Regards,
>> Strahil Nikolov
>>
>> ?? 17 ??? 2020 ?. 0:06:33 GMT+03:00, Mahdi Adnan <mahdi at sysmin.io> ??????:
>> >Hello,
>> >
>> > I'm wondering what's the current and future plan for Gluster project
>> >overall, I see that the project is not as busy as it was before "at
>> >least
>> >this is what I'm seeing" Like there are fewer blogs about what the
>> >roadmap
>> >or future plans of the project, the deprecation of Glusterd2, even Red
>> >Hat
>> >Openshift storage switched to Ceph.
>> >As the community of this project, do you feel the same? Is the
>> >deprecation
>> >of Glusterd2 concerning? Do you feel that the project is slowing down
>> >somehow? Do you think Red Hat is abandoning the project or giving fewer
>> >resources to Gluster?
>> ________
>>
>>
>>
>> Community Meeting Calendar:
>>
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968
>>
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users
>>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://lists.gluster.org/pipermail/gluster-users/attachments/20200616/a1d0f142/attachment-0001.html>