[Gluster-users] Need help making a decision choosing MS DFS or Gluster+SAMBA+CTDB

Mon Aug 10 17:02:24 UTC 2015

----- Original Message -----
> From: "David" <david.peer at gmail.com>
> To: "Daniel Müller" <mueller at tropenklinik.de>
> Cc: "gluster-users" <gluster-users at gluster.org>
> Sent: Monday, August 10, 2015 4:04:30 AM
> Subject: Re: [Gluster-users] Need help making a decision choosing MS DFS or	Gluster+SAMBA+CTDB
> 
> Thanks everyone.
> 
> So from reading all your comments, I understand that if I need an active /
> active synchronized setup for higher workloads, Gluster is for me.
> Other then that, DFS-R is a good option for data replication at the expanse
> of latency of the replicated data to the secondary node, and only one server
> is active per CIFS share.

If smallfile performance is a concern I HIGHLY recommend you steer clear of GLUSTER + SMB + CTDB.  Large file sequential and random IO is not great but OK, but smallfile and metadata operations(especially from 
Windows clients) are poor.  To put it in perspective I can create 3500 64k files / second on my glusterFS mount, on SMB I can only do 308(same HW / config).  This is something I am working on improving but there is quite a bit to be done on the gluster side for smallfile workloads to make sense performance wise:

- on creates, the extra xattrs that SMB requires (ACLs, etc) cause extra round trips, proposed solution:

http://www.gluster.org/community/documentation/index.php/Features/composite-operations#CREATE-AND-WRITE

- lack of good, coherent client-side caching (cache invalidation enables longer caching of metadata)
- incomplete metadata reads (READDIRPLUS) cause per-file round trips for directory scans, proposed solution:

http://www.gluster.org/community/documentation/index.php/Features/composite-operations#READDIRPLUS_used_to_prefetch_xattrs

- case-insensitive file lookup semantics, proposed solution:

http://www.gluster.org/community/documentation/index.php/Features/composite-operations#case-insensitive_volume_support

- high latency of file creates and even reads at the brick level, due to excessive system calls, proposed solution:

http://www.gluster.org/community/documentation/index.php/Features/stat-xattr-cache

-b

> Does DFS-R works well on high rate of changes?
> Found from other users use cases that DFS-R caused server hangs and such,
> hope it was fixed in Win2K12 server.
> 
> David
> 
> 
> On Mon, Aug 10, 2015 at 10:34 AM, Daniel Müller < mueller at tropenklinik.de >
> wrote:
> 
> 
> An example of a working share on samba4:
> 
> You can choose to work with vfs objects= glusterfs
> Glusterfs:volume=yourvolume
> Glusterfs:volfile.server=Your.server
> Form e it turned out to be too buggy.
> 
> 
> I just used instead the path=/path/toyour/mountedgluster
> 
> You will need this:
> posix locking =NO
> kernel share modes = No
> 
> [edv]
> comment=edv s4master verzeichnis auf gluster node1
> vfs objects= recycle
> ##vfs objects= recycle, glusterfs
> recycle:repository= /%P/Papierkorb
> ##glusterfs:volume= sambacluster
> ##glusterfs:volfile_server = XXX.XXXX.XXXX
> recycle:exclude = *.tmp,*.temp,*.log,*.ldb,*.TMP,?~$*,~$*,Thumbs.db
> recycle:keeptree = Yes
> recycle:exclude_dir = .Papierkorb,Papierkorb,tmp,temp,profile,.profile
> recycle:touch_mtime = yes
> recycle:versions = Yes
> recycle:minsize = 1
> msdfs root=yes
> path=/mnt/glusterfs/ads/wingroup/edv
> read only=no
> posix locking =NO
> kernel share modes = No
> access based share enum=yes
> hide unreadable=yes
> hide unwriteable files=yes
> veto files = Thumbs.db
> delete veto files = yes
> 
> Greetings
> Daniel
> 
> 
> EDV Daniel Müller
> 
> Leitung EDV
> Tropenklinik Paul-Lechler-Krankenhaus
> Paul-Lechler-Str. 24
> 72076 Tübingen
> Tel.: 07071/206-463, Fax: 07071/206-499
> eMail: mueller at tropenklinik.de
> Internet: www.tropenklinik.de
> 
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: gluster-users-bounces at gluster.org
> [mailto: gluster-users-bounces at gluster.org ] Im Auftrag von Dan Mons
> Gesendet: Montag, 10. August 2015 09:08
> An: Mathieu Chateau
> Cc: gluster-users; David
> Betreff: Re: [Gluster-users] Need help making a decision choosing MS DFS or
> Gluster+SAMBA+CTDB
> 
> If you're looking at a Gluster+Samba setup of any description for people
> extensively using Microsoft Office tools (either Windows or Mac clients), I
> *strongly* suggested exhaustive testing of Microsoft Word and Excel.
> 
> I've yet to find a way to make these work 100% on Gluster. Strange
> client-side locking behaviour with these tools often make documents
> completely unusable when hosted off Gluster. We host our large
> production files (VFX industry) off Gluster, however have a separate Windows
> Server VM purely for administration to host their legacy Microsoft Office
> documents (we've since migrated largely to Google Apps + Google Drive for
> that stuff, but the legacy requirement remains for a handful of users).
> 
> -Dan
> 
> ----------------
> Dan Mons - R&D Sysadmin
> Cutting Edge
> http://cuttingedge.com.au
> 
> 
> On 10 August 2015 at 15:42, Mathieu Chateau < mathieu.chateau at lotp.fr >
> wrote:
> > Hello,
> > 
> > what do you mean by "true" clustering ?
> > We can do a Windows Failover cluster (1 virtual ip, 1 virtual name),
> > but this mean using a shared storage like SAN.
> > 
> > Then it depends on your network topology. If you have multiple
> > geographical sites / datacenter, then DFS-R behave a lot better than
> > Gluster in replicated mode. Users won't notice any latency, At the
> > price that replication is async.
> > 
> > 
> > Cordialement,
> > Mathieu CHATEAU
> > http://www.lotp.fr
> > 
> > 2015-08-10 7:26 GMT+02:00 Ira Cooper < ira at redhat.com >:
> >> 
> >> Mathieu Chateau < mathieu.chateau at lotp.fr > writes:
> >> 
> >> > I do have DFS-R in production, that replaced sometimes netapp ones.
> >> > But no similar workload as my current GFS.
> >> > 
> >> > In active/active, the most common issue is file changed on both
> >> > side (no global lock) Will users access same content from linux &
> >> > windows ?
> >> 
> >> If you want to go active/active. I'd recommend Samba + CTDB + Gluster.
> >> 
> >> You want true clustering, and a system that can handle the locking etc.
> >> 
> >> I'd layer normal DFS to do "namespace" control, and to help with
> >> handling failover, or just use round robin DNS.
> >> 
> >> Thanks,
> >> 
> >> -Ira
> > 
> > 
> > 
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
> 
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users