[Gluster-users] client is terrible with large amount of small files
Ben Turner
bturner at redhat.com
Thu Apr 30 11:44:00 UTC 2015
----- Original Message -----
> From: "Atin Mukherjee" <amukherj at redhat.com>
> To: "gjprabu" <gjprabu at zohocorp.com>
> Cc: "Ben Turner" <bturner at redhat.com>, gluster-users at gluster.org
> Sent: Thursday, April 30, 2015 7:37:19 AM
> Subject: Re: [Gluster-users] client is terrible with large amount of small files
>
>
> On 04/30/2015 03:09 PM, gjprabu wrote:
> > Hi Amukher,
> >
> > How to resolve this issue, till we need to wait for 3.7 release
> > or any work around is there.
> You will have to as this feature is in for 3.7.
My apologies, I didn't realize that MT epoll didn't land in 3.6. If you want to test it out there is an alpha build available:
http://download.gluster.org/pub/gluster/glusterfs/nightly/glusterfs-3.7/epel-6-x86_64
I wouldn't run this in production until 3.7 is released though. Again sorry for the confusion.
-b
> >
> > RegardsPrabu
> >
> >
> >
> >
> >
> > ---- On Thu, 30 Apr 2015 14:49:46 +0530 Atin
> > Mukherjee<amukherj at redhat.com> wrote ----
> >
> >
> >
> > On 04/30/2015 02:32 PM, gjprabu wrote:
> > > Hi bturner,
> > >
> > >
> > > I am getting below error while adding server.event
> > >
> > > gluster v set integvol server.event-threads 3
> > > volume set: failed: option : server.event-threads does not exist
> > > Did you mean server.gid-timeout or ...manage-gids?
> > This option is not available in 3.6, its going to come in 3.7
> >
> > >
> > >
> > > Glusterfs version has been upgraded to 3.6.3
> > > Also os kernel upgraded to 6.6 kernel
> > > Yes two brick are running in KVM and one is physical machine and we
> > are not using thinp.
> > >
> > > Regards
> > > G.J
> > >
> > >
> > >
> > >
> > >
> > > ---- On Thu, 30 Apr 2015 00:37:44 +0530 Ben
> > Turner&lt;bturner at redhat.com&gt; wrote ----
> > >
> > > ----- Original Message -----
> > > &gt; From: "gjprabu" &lt;gjprabu at zohocorp.com&gt;
> > > &gt; To: "A Ghoshal" &lt;a.ghoshal at tcs.com&gt;
> > > &gt; Cc: gluster-users at gluster.org,
> > gluster-users-bounces at gluster.org
> > > &gt; Sent: Wednesday, April 29, 2015 9:07:07 AM
> > > &gt; Subject: Re: [Gluster-users] client is terrible with large
> > amount of small files
> > > &gt;
> > > &gt; Hi Ghoshal,
> > > &gt;
> > > &gt; Please find the details below.
> > > &gt;
> > > &gt; A) Glusterfs version
> > > &gt; glusterfs 3.6.2
> > >
> > > Upgrade to 3.6.3 and set client.event-threads and server.event-threads
> > to at least 4. Here is a guide on tuning MT epoll:
> > >
> > >
> > https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html
> > >
> > > &gt;
> > > &gt; B) volume configuration (gluster v &lt;volname&gt;
> > info)
> > > &gt; gluster volume info
> > > &gt;
> > > &gt;
> > > &gt; Volume Name: integvol
> > > &gt; Type: Replicate
> > > &gt; Volume ID: b8f3a19e-59bc-41dc-a55a-6423ec834492
> > > &gt; Status: Started
> > > &gt; Number of Bricks: 1 x 3 = 3
> > > &gt; Transport-type: tcp
> > > &gt; Bricks:
> > > &gt; Brick1: integ-gluster2:/srv/sdb1/brick
> > > &gt; Brick2: integ-gluster1:/srv/sdb1/brick
> > > &gt; Brick3: integ-gluster3:/srv/sdb1/brick
> > > &gt;
> > > &gt;
> > > &gt; C) host linux version
> > > &gt; CentOS release 6.5 (Final)
> > >
> > > Are your bricks on LVM? Are you using thinp? If so update to the
> > latest kernel as thinp perf was really bad in 6.5 and early 6.6 kernels.
> > >
> > > &gt;
> > > &gt; D) details about the kind of network you use to connect your
> > servers making
> > > &gt; up your storage pool.
> > > &gt; We are connecting LAN to LAN there is no special network
> > configuration done
> > > &gt;
> > > &gt; Frome client we use to mount like below
> > > &gt; mount -t glusterfs gluster1:/integvol /mnt/gluster/
> > > &gt;
> > > &gt;
> > > &gt; Regards
> > > &gt; Prabu
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; ---- On Wed, 29 Apr 2015 17:58:16 +0530 A
> > Ghoshal&lt;a.ghoshal at tcs.com&gt; wrote
> > > &gt; ----
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; Performance would largely depend upon setup. While I cannot
> > think of any
> > > &gt; setup that would cause write to be this slow, if would help
> > if you share the
> > > &gt; following details:
> > > &gt;
> > > &gt; A) Glusterfs version
> > > &gt; B) volume configuration (gluster v &lt;volname&gt;
> > info)
> > > &gt; C) host linux version
> > > &gt; D) details about the kind of network you use to connect your
> > servers making
> > > &gt; up your storage pool.
> > > &gt;
> > > &gt; Thanks,
> > > &gt; Anirban
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; From: gjprabu &lt; gjprabu at zohocorp.com &gt;
> > > &gt; To: &lt; gluster-users at gluster.org &gt;
> > > &gt; Date: 04/29/2015 05:52 PM
> > > &gt; Subject: Re: [Gluster-users] client is terrible with large
> > amount of small
> > > &gt; files
> > > &gt; Sent by: gluster-users-bounces at gluster.org
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; Hi Team,
> > > &gt;
> > > &gt; If anybody know the solution please share us.
> > > &gt;
> > > &gt; Regards
> > > &gt; Prabu
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; ---- On Tue, 28 Apr 2015 19:32:40 +0530 gjprabu &lt;
> > gjprabu at zohocorp.com &gt;
> > > &gt; wrote ----
> > > &gt; Hi Team,
> > > &gt;
> > > &gt; We are using glusterfs newly and testing data transfer part
> > in client using
> > > &gt; fuse.glusterfs file system but it is terrible with large
> > amount of small
> > > &gt; files (Large amount of small file 150MB of size it's writing
> > around 18min).
> > > &gt; I can able copy small files and syncing between the server
> > brick are working
> > > &gt; fine but it is terrible with large amount of small files.
> > > &gt;
> > > &gt; if anybody please share the solution for the above issue.
> > > &gt;
> > > &gt; Regards
> > > &gt; Prabu
> > > &gt;
> > > &gt; _______________________________________________
> > > &gt; Gluster-users mailing list
> > > &gt; Gluster-users at gluster.org
> > > &gt; http://www.gluster.org/mailman/listinfo/gluster-users
> > > &gt;
> > > &gt; _______________________________________________
> > > &gt; Gluster-users mailing list
> > > &gt; Gluster-users at gluster.org
> > > &gt; http://www.gluster.org/mailman/listinfo/gluster-users
> > > &gt;
> > > &gt;
> > > &gt; =====-----=====-----=====
> > > &gt; Notice: The information contained in this e-mail
> > > &gt; message and/or attachments to it may contain
> > > &gt; confidential or privileged information. If you are
> > > &gt; not the intended recipient, any dissemination, use,
> > > &gt; review, distribution, printing or copying of the
> > > &gt; information contained in this e-mail message
> > > &gt; and/or attachments to it are strictly prohibited. If
> > > &gt; you have received this communication in error,
> > > &gt; please notify us by reply e-mail or telephone and
> > > &gt; immediately and permanently delete the message
> > > &gt; and any attachments. Thank you
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; _______________________________________________
> > > &gt; Gluster-users mailing list
> > > &gt; Gluster-users at gluster.org
> > > &gt; http://www.gluster.org/mailman/listinfo/gluster-users
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Gluster-users mailing list
> > > Gluster-users at gluster.org
> > > http://www.gluster.org/mailman/listinfo/gluster-users
> > >
> >
> >
>
> --
> ~Atin
>
More information about the Gluster-users
mailing list