[Gluster-users] Performance drop when upgrading from 3.8 to 6.5
Strahil
hunter86_bg at yahoo.com
Tue Oct 29 13:11:45 UTC 2019
Hi Riccardo,
You can set your mounts with 'noatime,nodiratime' options for better performance.
Best Regards,
Strahil NikolovOn Oct 29, 2019 12:50, Riccardo Murri <riccardo.murri at gmail.com> wrote:
>
> Hello Anoop,
>
> many thanks for your fast reply! My comments inline below:
>
>
> > > [1]: I have tried both the config where SAMBA 4.8 is using the
> > > vfs_glusterfs.so backend, and the one where `smbd` is just writing to
> > > a locally-mounted directory. Doesn't seem to make a difference.
> >
> > Samba v4.8 is an EOL ed version. Please consider updating Samba to at
> > least v4.9(rather v4.10) or higher.
>
> This is going to be tricky: I could find no backport package of recent
> SAMBA to Ubuntu 16.04; I am using this one which has SAMBA 4.8
> https://launchpad.net/~mumblepins
>
> More recent packages from either the Ubuntu or Debian repositories do
> not build on Ubuntu 16.04 because of changes in the packaging
> infrastructure.
>
> Anyway, I was running SAMBA 4.8 before the upgrade and still getting
> 40MB/s, so I don't think SAMBA is the core of the issue...
>
> > Can you paste the output of `testparm -s` along with the output of
> > `gluster volume info <VOLNAME>` ?
>
> Here's `testparm -s` on the server using `vfs_glusterfs` (the "active"
> share is the one with the perf problems)::
>
> ```
> $ testparm -s
> Load smb config files from /etc/samba/smb.conf
> rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
> WARNING: The "syslog only" option is deprecated
> Processing section "[homes]"
> Processing section "[active]"
> Loaded services file OK.
> WARNING: some services use vfs_fruit, others don't. Mounting them in
> conjunction on OS X clients results in undefined behaviour.
>
> Server role: ROLE_STANDALONE
>
> # Global parameters
> [global]
> dns proxy = No
> load printers = No
> map to guest = Bad User
> name resolve order = lmhosts
> netbios name = REDACTED1
> obey pam restrictions = Yes
> pam password change = Yes
> passwd chat = *Enter\snew\s*\spassword:* %n\n
> *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
> passwd program = /usr/bin/passwd %u
> printcap cache time = 0
> printcap name = /dev/null
> security = USER
> server role = standalone server
> server string = SAMBA Server %v
> syslog only = Yes
> unix password sync = Yes
> workgroup = REDACTED
> idmap config * : backend = tdb
>
>
> [homes]
> browseable = No
> comment = Work Directories
> create mask = 0700
> directory mask = 0700
> read only = No
> valid users = %S
> vfs objects = fruit streams_xattr
>
>
> [active]
> create mask = 0775
> directory mask = 0775
> kernel share modes = No
> path = /active
> read only = No
> vfs objects = glusterfs
> glusterfs:volume = glusterfs
> glusterfs:volfile_server = glusterfs5 glusterfs4 glusterfs3
> glusterfs2 glusterfs1
> glusterfs:logfile = /var/log/samba/glusterfs-vol-active.log
> glusterfs:loglevel = 1
> ```
>
>
> Here's `testparm -s` on the server writing directly to the GlusterFS
> mount point::
>
> ```
> $ testparm -s
> Load smb config files from /etc/samba/smb.conf
> rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
> WARNING: The "syslog only" option is deprecated
> Processing section "[homes]"
> Processing section "[active]"
> Loaded services file OK.
> WARNING: some services use vfs_fruit, others don't. Mounting them in
> conjunction on OS X clients results in undefined behaviour.
>
> Server role: ROLE_STANDALONE
>
> # Global parameters
> [global]
> allow insecure wide links = Yes
> dns proxy = No
> load printers = No
> map to guest = Bad User
> name resolve order = lmhosts
> netbios name = REDACTED2
> obey pam restrictions = Yes
> pam password change = Yes
> passwd chat = *Enter\snew\s*\spassword:* %n\n
> *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
> passwd program = /usr/bin/passwd %u
> printcap cache time = 0
> printcap name = /dev/null
> security = USER
> server role = standalone server
> server string = SAMBA Server %v
> syslog only = Yes
> unix password sync = Yes
> workgroup = REDACTED
> idmap config * : backend = tdb
>
>
> [homes]
> browseable = No
> comment = Work Directories
> create mask = 0700
> directory mask = 0700
> read only = No
> valid users = %S
> vfs objects = fruit streams_xattr
>
>
> [active]
> create mask = 0775
> directory mask = 0775
> path = /data/active
> read only = No
> wide links = Yes
> ```
>
> Here's the volume info:
> ```
> $ sudo gluster volume info glusterfs
>
> Volume Name: glusterfs
> Type: Distribute
> Volume ID: a3358ff6-5cec-4a65-9ecf-a63bbe56dfd9
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 5
> Transport-type: tcp
> Bricks:
> Brick1: glusterfs5:/srv/glusterfs
> Brick2: glusterfs4:/srv/glusterfs
> Brick3: glusterfs3:/srv/glusterfs
> Brick4: glusterfs1:/srv/glusterfs
> Brick5: glusterfs2:/srv/glusterfs
> Options Reconfigured:
> diagnostics.client-log-level: WARNING
> diagnostics.brick-log-level: INFO
> features.uss: disable
> features.barrier: disable
> performance.client-io-threads: on
> transport.address-family: inet
> nfs.disable: on
> snap-activate-on-create: enable
> ```
>
>
> > > [2]: Actually, since the servers are VMs on an OpenStack cloud, I
> > > created new virtual machines, installed GlusterFS 6 fresh, mounted
> > > the old bricks in the same brick locations,
> >
> > How did you mount old bricks in the new location?
>
> They're `ext4` disks:
> ```
> $ fgrep glusterfs /etc/fstab
> LABEL=/srv/glusterfs /srv/glusterfs ext4 rw 0 0
> ```
>
> Actually, the GlusterFS volume is 95% full -- could this have an impact?
>
> Thanks,
> Riccardo
> ________
>
> Community Meeting Calendar:
>
> APAC Schedule -
> Every 2nd and 4th Tuesday at 11:30 AM IST
> Bridge: https://bluejeans.com/118564314
>
> NA/EMEA Schedule -
> Every 1st and 3rd Tuesday at 01:00 PM EDT
> Bridge: https://bluejeans.com/118564314
>
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
More information about the Gluster-users
mailing list