[Gluster-users] Gluster 3.5.2 gluster, how does cache work?

[Pranith Kumar Karampuri]

CC Raghavendra G, who maintains performance translators to explain how 
io-cache works.

Pranith
On 08/22/2014 09:14 PM, Juan José Pavlik Salles wrote:
> Hi Ben, I also thought of that, but look at this:
>
> [root at gluster-client-1 gluster_vol]# echo 3 > 
> /proc/sys/vm/drop_caches; dd if=/mnt/gluster_vol/128 of=/dev/null bs=1M;
> 128+0 records in
> 128+0 records out
> 134217728 bytes (134 MB) copied, 6,70874 s, 20,0 MB/s
> [root at gluster-client-1 gluster_vol]# echo 3 > 
> /proc/sys/vm/drop_caches; dd if=/mnt/gluster_vol/128 of=/dev/null bs=1M;
> 128+0 records in
> 128+0 records out
> 134217728 bytes (134 MB) copied, 0,160485 s, 836 MB/s
> [root at gluster-client-1 gluster_vol]# echo 3 > 
> /proc/sys/vm/drop_caches; dd if=/mnt/gluster_vol/128 of=/dev/null bs=1M;
> 128+0 records in
> 128+0 records out
> 134217728 bytes (134 MB) copied, 0,148219 s, 906 MB/s
> [root at gluster-client-1 gluster_vol]#
>
> even dropping kernel caches I get a really fast answer. I think that 
> this is io-cache translator working on the client side for sure.
>
>
> 2014-08-22 12:27 GMT-03:00 Ben Turner <bturner at redhat.com 
> <mailto:bturner at redhat.com>>:
>
>     ----- Original Message -----
>     > From: "Juan José Pavlik Salles" <jjpavlik at gmail.com
>     <mailto:jjpavlik at gmail.com>>
>     > To: gluster-users at gluster.org <mailto:gluster-users at gluster.org>
>     > Sent: Thursday, August 21, 2014 4:07:28 PM
>     > Subject: [Gluster-users] Gluster 3.5.2 gluster, how does cache work?
>     >
>     > Hi guys, I've been reading a bit about caching in gluster
>     volumes, but I
>     > still don't get a few things. I set up a gluster replica 2
>     volume like this:
>     >
>     > [root at gluster-test-1 ~]# gluster vol info vol_rep
>     > Volume Name: vol_rep
>     > Type: Replicate
>     > Volume ID: b77db06d-2686-46c7-951f-e43bde21d8ec
>     > Status: Started
>     > Number of Bricks: 1 x 2 = 2
>     > Transport-type: tcp
>     > Bricks:
>     > Brick1: gluster-test-1:/ladrillos/l1/l
>     > Brick2: gluster-test-2:/ladrillos/l1/l
>     > Options Reconfigured:
>     > performance.cache-min-file-size: 90MB
>     > performance.cache-max-file-size: 256MB
>     > performance.cache-refresh-timeout: 60
>     > performance.cache-size: 256MB
>     > [root at gluster-test-1 ~]#
>     >
>     > Then I mounted the volume with gluster client on another
>     machine. I created
>     > an 80Mbytes file called 80, and here you have the reading test:
>     >
>     > [root at gluster-client-1 gluster_vol]# dd if=/mnt/gluster_vol/80
>     of=/dev/null
>     > bs=1M
>     > 80+0 records in
>     > 80+0 records out
>     > 83886080 bytes (84 MB) copied, 1,34145 s, 62,5 MB/s
>     > [root at gluster-client-1 gluster_vol]# dd if=/mnt/gluster_vol/80
>     of=/dev/null
>     > bs=1M
>     > 80+0 records in
>     > 80+0 records out
>     > 83886080 bytes (84 MB) copied, 0,0246918 s, 3,4 GB/s
>     > [root at gluster-client-1 gluster_vol]# dd if=/mnt/gluster_vol/80
>     of=/dev/null
>     > bs=1M
>     > 80+0 records in
>     > 80+0 records out
>     > 83886080 bytes (84 MB) copied, 0,0195678 s, 4,3 GB/s
>     > [root at gluster-client-1 gluster_vol]#
>
>     You are seeing the effect of client side kernel caching.  If you
>     want to see the actual throughput for reads run:
>
>     sync; echo 3 > /proc/sys/vm/drop_caches; dd blah
>
>     Kernel caching happens on both the client and server side, when I
>     want to see uncached performance I drop caches on both clients and
>     servers:
>
>     run_drop_cache()
>     {
>         for host in $MASTERNODE $NODE $CLIENT
>         do
>             ssh -i /root/.ssh/my_id root@${host} echo "Dropping cache
>     on $host"
>             ssh -i /root/.ssh/my_id root@${host} sync
>             ssh -i /root/.ssh/my_id root@${host} "echo 3 >
>     /proc/sys/vm/drop_caches"
>         done
>     }
>
>     HTH!
>
>     -b
>
>     > Cache is working flawlessly, (even though that 80 Mbytes is
>     smaller than the
>     > min-file-size value, but I don't care about it right now) what I
>     don't get
>     > is where cache is being stored. Is it stored on the client side
>     or on the
>     > server side? According to documentation, the io-cache translator
>     could be
>     > loaded on both sides (client and server), how can I know where
>     it is being
>     > loeaded? It looks like as it was being stored locally because of
>     the speed,
>     > but I'd like to be sure.
>     >
>     > Thanks!
>     >
>     > --
>     > Pavlik Salles Juan José
>     > Blog - http://viviendolared.blogspot.com
>     >
>     > _______________________________________________
>     > Gluster-users mailing list
>     > Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
>     > http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>
>
>
> -- 
> Pavlik Salles Juan José
> Blog - http://viviendolared.blogspot.com
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20140823/97d045c8/attachment.html>