[Gluster-users] Giving up [ was: Re: read-subvolume]

[HL]

Been there ...
here is my 10cent advise
a) Prepare for tomorrow
b) Rest
c) Think
d) Plan
e) act

I am sure it will work for you when calmed

Tech hints.
ifconfig iface mtu 9000 or whatever your nic can afford
Having a 100Mbit is not a good idea.
I 've recently located a dual port 1Gbit nic on ebay for $15 USD
Get them.

And last but not least
In case you happen to have a switch between the nodes make sure that you 
enable jumbo frames on it.
Otherwise you are in DEEP Trouble.

stat' ing a 120G file in my case takes miliseconds not even seconds.

GL
Harry.


On 10/07/2013 02:01 μμ, Allan Latham wrote:
> Hi all
>
> Thanks to all those volunteers who are working to get gluster into a
> state where it can be used for live work.
>
> I understand that you are giving your free time and I very much
> appreciate it on this project and the many others we use for live
> production work.
>
> There seems to be a problem with the way gluster is going.
> For me it would be an ideal solution if it actually worked.
>
> I have a simple scenario and it just simply doesn't work. Reading over
> the network when the file is available locally is plainly wrong. Our
> application cannot take the performance hit nor the extra network traffic.
>
> I would suggest:
>
> 1. get a simple minimalist configuration working - 2 hosts and
> replication only.
> 2. make it bomb-proof.
> 2a. it must cope with network failures, random reboots etc.
> 2b. if it stops it has to auto-recover quickly.
> 2c. if it can't it needs thorough documentation and adequate logs so a
> reasonable sysop can rescue it.
> 2d. it needs a fast validation scanner which verifies that data is where
> it should be and is identical everywhere (md5sum).
> 3. make it efficient (read local whenever possible - use rsync
> techniques - remove scalability obstacles so it doesn't get
> exponentially slower as more files are replicated)
> 4. when that works expand to multiple hosts and clever distribution
> techniques.
> (repeat items 2 and 3 in the more complex environment)
>
> If it doesn't work rock solid in a simple scenario it will never work in
> a large scale cluster.
>
> Until point 3 is reached I cannot use it - which is a great
> disappointment for me as well as the good guys doing the development.
>
> Good luck and thanks again
>
> Allan
>
>
> On 04/07/13 13:10, Allan Latham wrote:
>> Hi all
>>
>> Does anyone use read-subvolume?
>>
>> Has anyone tested read-subvolume?
>>
>> Does read-subvolume work in such a way that if the file is present on
>> the local node the local copy is used rather than a remote one?
>>
>> Alternatively is there any way to configure (or patch) gluster to always
>> prefer the local file?
>>
>> I have read everything available and have found no answer.
>>
>> Unison works very well in our environment but is not real time and needs
>> to be run every few minutes and/or be kicked off with inotify.
>>
>> If I could get gluster to always read the local copy it would be a much
>> better drop in replacement.
>>
>> This is a small scale deployment not a massive cluster but I can imagine
>> there are many potential users of gluster in this mode. It should beat
>> unison and similar solutions in every way - but it doesn't because it is
>> reading from the network even when it has a local up-to-date copy. This
>> can't be intended behaviour.
>>
>> So what have I configured wrong?
>>
>> Thanks in advance
>>
>> Allan
>>
>>
>> On 02/07/13 13:38, Allan Latham wrote:
>>> Hi everyone
>>>
>>> I have installed 3.3.1-1 from the Debian repository you provide.
>>>
>>> I am using a simple 2 node cluster and running in replication mode. The
>>> connection between the nodes is limited to 100MB/sec (that's bits not
>>> bytes!). Usage will be mainly for read access and since there is always
>>> a local copy available [ exactly 2 replicas on exactly 2 machines ] I
>>> expect very fast read performance. Writes are low volume and very
>>> infrequent - performance is not an issue.
>>>
>>> Almost everything works as I would expect.
>>>
>>> Write speed is limited to 10Mb (bytes) per second which is what I would
>>> expect and is adequate for the application.
>>>
>>> But read speed is either super fast or 10Mb/sec. i.e. read operations
>>> take place on the local copy or the remote seemingly at random.
>>>
>>> This not the 'small files problem'. I am aware that Gluster must use
>>> network access for stat() etc. This is all about where the data comes
>>> from on a read(). If I do an m5dum on a 200Mb file it takes either half
>>> a second or 18 seconds.
>>>
>>> There is an option read-subvolume.
>>>
>>> I have tried to understand how this works from the documentation
>>> available and from the few examples on the web.
>>>
>>> I have added the option using:
>>>
>>> gluster volume set X read-subvolume Y
>>>
>>> It has no effect even after stopping and starting the volume,
>>> remounting, restarting gluster servers etc.
>>>
>>> What's more I fail to see how this option could ever work at all. The
>>> configuration changes caused by the above command are rolled out to both
>>> nodes - but what is right for one node is exactly the wrong
>>> configuration for the other node.
>>>
>>> Configs attached are in /var/lib/glusterd/vols/shared except
>>> glusterd.vol which is in /etc/glusterfs.
>>>
>>> Here is the output of the mount command filtered to just the glusterfs
>>> mount:
>>>
>>> 10.255.255.1:/shared on /gluster/rw/shared type fuse.glusterfs
>>> (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
>>>
>>> 10.255.255.1 is local to this host.
>>>
>>> I would be very thankful if someone can enlighten me. I am obviously
>>> configuring this wrong. I may have missed something important.
>>>
>>> Best regards to all
>>>
>>> Allan
>>>
>>>
>>>
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users