[Gluster-devel] Problems with ioc again

[Dan Parsons]

I'd love to use 1.4rc4 but are there any issues in it that would  
effect me? I have 4 glusterfs servers, each with 2gbit ethernet  
(bonded), provididing sustained 8gbit/s to 33 client nodes. Below is  
my entire config file. If you see anything in there using a system  
that is either buggy or non-optimal in 1.4rc4, or would be difficult  
to upgrade, please let me know. If not, I can possibly upgrade.

Below is my current config file. The one I was using when gluster was  
using all memory is identical except for 'cache-size' was changed to  
4096MB and 'page-size' was changed to 512KB.

-----------
### Add client feature and attach to remote subvolume of server1
volume distfs01
  type protocol/client
  option transport-type tcp/client     # for TCP/IP transport
  option remote-host 10.8.101.51      # IP address of the remote brick
  option remote-subvolume brick        # name of the remote volume
end-volume

### Add client feature and attach to remote subvolume of server2
volume distfs02
  type protocol/client
  option transport-type tcp/client     # for TCP/IP transport
  option remote-host 10.8.101.52      # IP address of the remote brick
  option remote-subvolume brick        # name of the remote volume
end-volume

volume distfs03
  type protocol/client
  option transport-type tcp/client
  option remote-host 10.8.101.53
  option remote-subvolume brick
end-volume

volume distfs04
  type protocol/client
  option transport-type tcp/client
  option remote-host 10.8.101.54
  option remote-subvolume brick
end-volume

volume stripe0
   type cluster/stripe
   option block-size *.gff:1KB,*.nt:1KB,*.best: 
1KB,*.txt3:1KB,*.nbest.info:1KB*:1MB
   option scheduler alu
   option alu.order read-usage:write-usage:disk-usage
   option alu.read-usage.entry-threshold 20%
   option alu.read-usage.exit-threshold 4%
   option alu.write-usage.entry-threshold 20%
   option alu.write-usage.exit-threshold 4%
   option alu.disk-usage.entry-threshold 2GB
   option alu.disk-usage.exit-threshold 100MB
   subvolumes distfs01 distfs02 distfs03 distfs04
end-volume


volume ioc
   type performance/io-cache
   subvolumes stripe0         # In this example it is 'client' you may  
have to change it according to your spec file.
   option page-size 1MB      # 128KB is default
   option cache-size 2048MB    # 32MB is default
   option force-revalidate-timeout 5 # 1second is default
   option priority *.psiblast:3,*.seq:2,*:1
end-volume

volume fixed
   type features/fixed-id
   option fixed-uid 0
   option fixed-gid 900
   subvolumes ioc
end-volume

Dan Parsons


On Dec 17, 2008, at 2:09 PM, Anand Avati wrote:

> Dan,
> Is it feasible for you to try 1.4.0pre4? We have fixed a couple of
> ioc leaks in that branch. Backporting them to the 1.3.x tree is a bit
> hard for us at the moment since all the developers are maxed out on
> tasks. Please let us know if there is no way around than upgrading to
> 1.4rc for you.
>
> Avati
>
> 2008/12/18 Dan Parsons <dparsons at nyip.net>:
>> I am once more having a problem with the ioc translator.  
>> Specifically, I
>> upgraded the RAM in my gluster client from 4GB to 12GB. Under the 4GB
>> configuration, I had cache-size set to 2048MB. That was working  
>> alright.
>> Once I upgraded the server to 12GB, I changed cache-size to 4GB.  
>> After doing
>> that, glusterfs proceeded to use every last drop of RAM, and all  
>> swap, on
>> the system. Not right away of course, only after I read about,  
>> well, 12GB of
>> data.
>>
>> This behavior was repeated across all 33 of my gluster clients.  
>> Even back
>> when it was set to 2GB, there were times when it would ignore that  
>> setting,
>> and if you look through the mailing list, you'll find at least 2  
>> people that
>> had the same problem, and didn't get much support.
>>
>> The ioc translator is an extremely essential part to the operation  
>> of my
>> department. I really need this fixed ASAP- if there is anyone I can  
>> pay to
>> do this, please contact me off-list, as it needs to be done ASAP.
>>
>> [root at admin01 conf]# glusterfs --version
>> glusterfs 1.3.11 built on Aug 21 2008 11:26:38
>> Repository revision: glusterfs--mainline--2.5--patch-795
>>
>> volume ioc
>> type performance/io-cache
>> subvolumes stripe0         # In this example it is 'client' you may  
>> have to
>> change it according to your spec file.
>> option page-size 1MB      # 128KB is default
>> option cache-size 2048MB    # 32MB is default
>> option force-revalidate-timeout 5 # 1second is default
>> option priority *.psiblast:3,*.seq:2,*:1
>> end-volume
>>
>>
>>
>> Dan Parsons
>>
>>
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
>