[Gluster-users] Performance drop when upgrading from 3.8 to 6.5
riccardo.murri at gmail.com
Tue Oct 29 09:59:56 UTC 2019
I recently upgraded our servers from GlusterFS 3.8 (old GlusterFS
repo for Ubuntu 16.04) to 6.0 (gotten from the GlusterFS PPA for
Ubuntu 16.04 "xenial").
The sustained write performance nearly dropped to half it was before.
We copy a large (a few 10'000s) number of image files (each 2 to 10 MB
size) from the microscope where they were acquired to a SAMBA server
which mounts the GlusterFS volume; before the upgrade, we could
write at about 40MB/s, after the upgrade, this dropped to 20MB/s.
This is the version of server and client software installed:
$ dpkg -l '*gluster*'
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture
ii glusterfs-client 6.5-ubuntu1~xenial1 amd64
clustered file-system (client package)
ii glusterfs-common 6.5-ubuntu1~xenial1 amd64
GlusterFS common libraries and translator modules
ii glusterfs-server 6.5-ubuntu1~xenial1 amd64
clustered file-system (server package)
Op version has been upped to 60000:
$ sudo gluster volume get all cluster.op-version
$ sudo gluster volume get all cluster.max-op-version
Running `sudo gluster volume status all clients` reports that all
clients are on op-version 60000, too.
Any suggestions on what to look for or changes to try out?
: I have tried both the config where SAMBA 4.8 is using the
vfs_glusterfs.so backend, and the one where `smbd` is just writing to
a locally-mounted directory. Doesn't seem to make a difference.
: Actually, since the servers are VMs on an OpenStack cloud, I
created new virtual machines, installed GlusterFS 6 fresh, mounted the
old bricks in the same brick locations, and restarted the cluster. I
had to fiddle a bit with the files in `/var/lib/glusterfs` because the
hostnames and IPs changed but did not do anything else than `sed -e
s/old_hostname/new_hostname/` or similarly renaming files. In
particular, I did not touch the extended attributes in the brick
More information about the Gluster-users