[Gluster-users] State of Gluster project

Thu Jun 18 03:51:31 UTC 2020

Hm... Actually comparing CEPH to Gluster is like to compare  apples to pears.

I  have played with both and I can say that CEPH is optimised for disk images - which gives benefit in Openstack and virtualizations,  but CEPH is harder to learn and lacks some features of Gluster.

For examlle Gluster has:
1. On Fedora-bases  systems, you can use Virtual Data Optimizer (VDO)  for data deduplication or data compression
On Debian Based - ZOL is an option
2. Geo-replication - very useful for setting up DR
3. In replica  and dispersed volumes , you can loose any node (sometimes more) without worrying about the type of the system - they have the same role and same purpose
4. Gluster clients are more diverse - it is nice that Windows(cifs),  Apple(whatever they call it), Linux(fuse/nfs) and BSDs(nfs) can use the volume.
5. Easier setup with less nodes - Gluster is easy to setup and  with sharding - is great even for virtualization. The  client uses  the algorithm to find the file without the necessity  to query another server.

Of course Gluster has it's own drawbacks - but the software is optimal for HPC and Archive storage.If you know the path to your file, Gluster will retrieve it quite fast.

I have to mention that in the past years (since Dec  2017 to be more precise)  , I had only 2 major issues - always during upgrades. When I compare another environment with enterprise-class  storages (in cluster mode) , which also failed once after an update (3  years period  of comparison) - the issue rate is not so big,  but the peice  for Gluster  is not millions :)

Best Regards,
Strahil Nikolov

На 17 юни 2020 г. 19:15:00 GMT+03:00, Erik Jacobson <erik.jacobson at hpe.com> написа:
>> It is very hard to compare them because they are structurally very
>different. For example, GlusterFS performance will depend *a lot* on
>the underlying file system performance. Ceph eliminated that factor by
>using Bluestore.
>> Ceph is very well performing for VM storage, since it's block based
>and as such optimized for that. I haven't tested CephFS a lot (I used
>it but only for very small storage) so I cannot speak for its
>performance, but I am guessing it's not ideal. For large amount of
>files thus GlusterFS is still a good choice.
>
>
>Was your experience above based on using a sharded volume or a normal
>one? When we worked with virtual machine images, we followed the volume
>sharding advice. I don't have a comparison for Ceph handy. I was just
>curious. It worked so well for us (but maybe our storage is "too good")
>that we found it hard to imagine it could be improved much. This was a
>simple case though of a single VM, 3 gluster servers, a sharded volume,
>and a raw virtual machine image. Probably a simpler case than yours.
>
>Thank you for writing this and take care,
>
>Erik
>
>> 
>> One *MAJOR* advantage of Ceph over GlusterFS is tooling. Ceph's
>self-analytics, status reporting and problem fixing toolset is just so
>far beyond GlusterFS that it's really hard for me to recommend
>GlusterFS for any but the most experienced sysadmins. It does come with
>the type of implementation Ceph has chosen that they have to have such
>good tooling (because honestly, poking around in binary data structures
>really wouldn't be practical for most users), but whenever I had a
>problem with Ceph the solution was just a couple of command line
>commands (even if it meant to remove a storage device, wipe it and add
>it back), where with GlusterFS it means poking around in the .glusterfs
>directory, looking up inode numbers, extended attributes etc. which is
>a real pain if you have a multi-million-file filesystem to work on. And
>that's not even with sharding or distributed volumes.
>> 
>> Also, Ceph has been a lot more stable that GlusterFS for us. The
>amount of hand-holding GlusterFS needs is crazy. With Ceph, there is
>this one bug (I think in certain Linux kernel versions) where it
>sometimes reads only zeroes from disk and complains about that and then
>you have to restart that OSD to not run into problems, but that's one
>"swatch" process on each machine that will do that automatically for
>us. I have run some Ceph clusters for several years now and only once
>or twice I had to deal with problems. The several GlusterFS clusters we
>operate constantly run into troubles. We now shut down all GlusterFS
>clients before we reboot any GlusterFS node because it was near
>impossible to reboot a single node without running into unrecoverable
>troubles (heal entries that will not heal etc.). With Ceph we can
>achieve 100% uptime, we regularly reboot our hosts one by one and some
>minutes later the Ceph cluster is clean again.
>> 
>> If others have more insights I'd be very happy to hear them.
>> 
>> Stefan
>> 
>> 
>> ----- Original Message -----
>> > Date: Tue, 16 Jun 2020 20:30:34 -0700
>> > From: Artem Russakovskii <archon810 at gmail.com>
>> > To: Strahil Nikolov <hunter86_bg at yahoo.com>
>> > Cc: gluster-users <gluster-users at gluster.org>
>> > Subject: Re: [Gluster-users] State of Gluster project
>> > Message-ID:
>>
>>	<CAD+dzQdf_TiPBSDj57hY=t8AQ=mACrxinPX7iU4hmuxNMo+omg at mail.gmail.com>
>> > Content-Type: text/plain; charset="utf-8"
>> > 
>> > Has anyone tried to pit Ceph against gluster? I'm curious what the
>ups and
>> > downs are.
>> > 
>> > On Tue, Jun 16, 2020, 4:32 PM Strahil Nikolov
><hunter86_bg at yahoo.com> wrote:
>> > 
>> >> Hey Mahdi,
>> >>
>> >> For me it looks like Red Hat are focusing more  on CEPH  than on
>Gluster.
>> >> I hope the project remains active, cause it's very difficult to
>find a
>> >> Software-defined Storage as easy and as scalable as Gluster.
>> >>
>> >> Best Regards,
>> >> Strahil Nikolov
>> >>
>> >> ?? 17 ??? 2020 ?. 0:06:33 GMT+03:00, Mahdi Adnan <mahdi at sysmin.io>
>??????:
>> >> >Hello,
>> >> >
>> >> > I'm wondering what's the current and future plan for Gluster
>project
>> >> >overall, I see that the project is not as busy as it was before
>"at
>> >> >least
>> >> >this is what I'm seeing" Like there are fewer blogs about what
>the
>> >> >roadmap
>> >> >or future plans of the project, the deprecation of Glusterd2,
>even Red
>> >> >Hat
>> >> >Openshift storage switched to Ceph.
>> >> >As the community of this project, do you feel the same? Is the
>> >> >deprecation
>> >> >of Glusterd2 concerning? Do you feel that the project is slowing
>down
>> >> >somehow? Do you think Red Hat is abandoning the project or giving
>fewer
>> >> >resources to Gluster?
>> >> ________
>> >>
>> >>
>> >>
>> >> Community Meeting Calendar:
>> >>
>> >> Schedule -
>> >> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> >> Bridge: https://bluejeans.com/441850968 
>> >>
>> >> Gluster-users mailing list
>> >> Gluster-users at gluster.org
>> >> https://lists.gluster.org/mailman/listinfo/gluster-users 
>> >>
>> > -------------- next part --------------
>> > An HTML attachment was scrubbed...
>> > URL:
>> >
><http://lists.gluster.org/pipermail/gluster-users/attachments/20200616/a1d0f142/attachment-0001.html
>>
>> ________
>> 
>> 
>> 
>> Community Meeting Calendar:
>> 
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968 
>> 
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> https://lists.gluster.org/mailman/listinfo/gluster-users 
>________
>
>
>
>Community Meeting Calendar:
>
>Schedule -
>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>Bridge: https://bluejeans.com/441850968
>
>Gluster-users mailing list
>Gluster-users at gluster.org
>https://lists.gluster.org/mailman/listinfo/gluster-users