[Gluster-users] Caching differences in Gluster vs Local Storage

[Jon Swanson]

Yeah, obviously it's not actually writing to physical disks. I'm 
assuming that because it's a small file size (32GB), most of that is 
just hitting the filesystem cache.

What i'm curious about is why Gluster is not seeing similar benefits 
from filesystem cache.

It is getting /some/ benefit:
[root at linuxdb1 tiobench-gluster.2]# tiotest -b 16384 -r 4096 -f 32 -t 16 
-d .
Tiotest results for 16 concurrent io threads:
,----------------------------------------------------------------------.
| Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write         512 MBs |   16.7 s |  30.731 MB/s |   2.0 %  |  33.5 % |
| Random Write 1024 MBs |   38.9 s |  26.314 MB/s |   1.8 %  |  32.5 % |
| Read          512 MBs |    4.8 s | 107.145 MB/s |   4.0 %  | 221.4 % |
| Random Read  1024 MBs |    4.2 s | 241.220 MB/s |  11.6 %  | 543.4 % |

There's no way it's getting 241 MB/s over gigabit with Random Read. I'm 
sure there's a reason for this, just curious as to what it is.

On 04/02/2010 04:29 PM, Marcus Bointon wrote:
> On 2 Apr 2010, at 09:10, Jon Swanson wrote:
>
>    
>> ,----------------------------------------------------------------------.
>> | Item                  | Time     | Rate         | Usr CPU  | Sys CPU |
>> +-----------------------+----------+--------------+----------+---------+
>> | Write         512 MBs |   35.7 s |  14.361 MB/s |   0.5 %  | 833.8 % |
>> | Random Write 1024 MBs |  100.6 s |  10.182 MB/s |   0.4 %  | 379.5 % |
>> | Read          512 MBs |    0.1 s | 4043.978 MB/s |  74.2 %  | 5832.1 % |
>> | Random Read  1024 MBs |    0.2 s | 4171.521 MB/s | 131.2 %  | 6425.0 % |
>> `----------------------------------------------------------------------'
>>      
> Either these numbers or units are wrong or you have some outrageously fast disks! 4Gbytes/sec?? You have multiple FusionIOs or something?
>
> Marcus
>