[Gluster-devel] glusterfs version3 and xen images

Mon Dec 14 07:50:55 UTC 2009

Hi,

i think the algorithms doesnt work yet. I read the pdf -> 
http://ftp.gluster.com/pub/gluster/glusterfs/3.0/LATEST/GlusterFS-3.0.0-Release-Notes.pdf

and this article is very interesting:

The “Diff” algorithm is especially beneficial for situations such as 
running VM images,
where self-heal of a recovering replicated copy of the image will occur 
much faster because
only the changed blocks need to be synchronized.

i use 12 GB (to 30GB) Xen Images and client side replication. If i 
simulate an Hardwareerror on glusterfsserver 1 and after 5 minutes the 
glusterfsserver 1 came back online, glusterfsserver2 starts sync but i 
think not with "diff" algotihm ether with full!! i see the sync if i do 
an iftop on both glusterfsserver.

while syncing you cant do nothing in domU. no "df -h", no "ls" nothing - 
you can do nothing. if the iftop on both glusterfsserver has reached 
e.g. 12GB the domU works further.

And thats the problem. in case of sync the domU must run further. This 
is our destination.

Thank you,

Roland Fischer

Am 12.12.2009 00:02, schrieb Kamal Varma:
> Hi Roland,
>
> The algorithms are automatically selected based on file size.
>
> May I ask what algorithm would you like to select/override to? and why (whats your use case)
>
> Thanks,
>
> Kamal
>
> ----- "Roland Fischer"<roland.fischer at xidras.com>  wrote:
>
>    
>> Hallo,
>>
>> i have tried the new version 3 of glusterfs with xen.
>>
>> we have 2 gfsserver and 2 xen server. we use client-side-replication
>> with domUs.
>>
>> i need to know if our configuration is good to use with xen domUs:
>>
>> server:
>> # export-domU-images-server_repl
>> # gfs-01-01 /GFS/domU-images
>> # gfs-01-02 /GFS/domU-images
>>
>> volume posix
>> type storage/posix
>> option directory /GFS/domU-images
>> end-volume
>>
>> volume locks
>> type features/locks
>> subvolumes posix
>> end-volume
>>
>> volume domU-images
>> type performance/io-threads
>> option thread-count 16 # default is 16
>> subvolumes locks
>> end-volume
>>
>> volume server
>> type protocol/server
>> option transport-type tcp
>> option auth.addr.domU-images.allow 192.168.11.*,127.0.0.1
>> option transport.socket.listen-port 6997
>> subvolumes domU-images
>> end-volume
>>
>> and client:
>> volume gfs-01-01
>> type protocol/client
>> option transport-type tcp
>> option remote-host gfs-01-01
>> option transport.socket.nodelay on
>> option remote-port 6997
>> option remote-subvolume domU-images
>> option ping-timeout 7
>> end-volume
>>
>> volume gfs-01-02
>> type protocol/client
>> option transport-type tcp
>> option remote-host gfs-01-02
>> option transport.socket.nodelay on
>> option remote-port 6997
>> option remote-subvolume domU-images
>> option ping-timeout 7
>> end-volume
>>
>> volume gfs-replicate
>> type cluster/replicate
>> subvolumes gfs-01-01 gfs-01-02
>> end-volume
>>
>> volume writebehind
>> type performance/write-behind
>> option cache-size 16MB
>> subvolumes gfs-replicate
>> end-volume
>>
>> volume readahead
>> type performance/read-ahead
>> option page-count 16 # cache per file = (page-count x page-size)
>> subvolumes writebehind
>> end-volume
>>
>> volume iocache
>> type performance/io-cache
>> option cache-size 1GB
>> option cache-timeout 1
>> subvolumes readahead
>> end-volume
>>
>>
>> i start a domU and simulate an crash on gfs-01-02 (rcnetwork stop;
>> sleep
>> 150; rcnetwork start). domU runs further without any problems.
>>
>> Client Log:
>> [2009-12-10 16:34:16] E
>> [client-protocol.c:415:client_ping_timer_expired] gfs-01-02: Server
>> xxx.xxx.xxx.xxx:6997 has not responded in the last 7 seconds,
>> disconnecting.
>> [2009-12-10 16:34:16] E [saved-frames.c:165:saved_frames_unwind]
>> gfs-01-02: forced unwinding frame type(1) op(GETXATTR)
>> [2009-12-10 16:34:16] E [saved-frames.c:165:saved_frames_unwind]
>> gfs-01-02: forced unwinding frame type(2) op(PING)
>> [2009-12-10 16:34:16] N [client-protocol.c:6972:notify] gfs-01-02:
>> disconnected
>> [2009-12-10 16:34:38] E [socket.c:760:socket_connect_finish]
>> gfs-01-02:
>> connection to xxx.xxx.xxx.xxx:6997 failed (No route to host)
>> [2009-12-10 16:34:38] E [socket.c:760:socket_connect_finish]
>> gfs-01-02:
>> connection to xxx.xxx.xxx.xxx:6997 failed (No route to host)
>>
>> network on gfs-01-02 started
>>
>> Client Log:
>> [2009-12-10 16:35:15] N [client-protocol.c:6224:client_setvolume_cbk]
>>
>> gfs-01-02: Connected to xxx.xxx.xxx.xxx:6997, attached to remote
>> volume
>> 'domU-images'.
>> [2009-12-10 16:35:18] N [client-protocol.c:6224:client_setvolume_cbk]
>>
>> gfs-01-02: Connected to xxx.xxx.xxx.xxx:6997, attached to remote
>> volume
>> 'domU-images'.
>> [2009-12-10 16:35:20] E
>> [afr-self-heal-common.c:1186:sh_missing_entries_create] gfs-replicate:
>>
>> no missing files - /vm_disks/virt-template. proceeding to metadata
>> check
>>
>> everything looks good - sync is started from gfs-01-01 to gfs-01-02
>> but
>> the whole image were transfered.
>>
>> if we do a ls -la in domU while the transfer the prompt in domU
>> disappears. After the sync is ready the prompt in domU appears and we
>>
>> can work further.
>>
>> my question: is this a normal behavoir? in
>> http://ftp.gluster.com/pub/gluster/glusterfs/3.0/LATEST/GlusterFS-3.0.0-Release-Notes.pdf
>>
>> we have read:
>>
>> 2.1) Choice of self-heal algorithms
>>
>> During self-heal of file contents, GlusterFS will now dynamically
>> choose
>> between two
>> algorithms based on file size:
>>
>> a) "Full" algorithm – this algorithm copies the entire file data in
>> order to heal the out-ofsync
>> copy. This algorithm is used when a file has to be created from
>> scratch on a
>> server.
>>
>> b) "Diff" algorithm – this algorithm compares blocks present on both
>> servers and copies
>> only those blocks that are different from the correct copy to the
>> out-of-sync copy. This
>> algorithm is used when files have to be re-built partially.
>>
>> The “Diff” algorithm is especially beneficial for situations such as
>> running VM images,
>> where self-heal of a recovering replicated copy of the image will
>> occur
>> much faster because
>> only the changed blocks need to be synchronized.
>>
>> can we change the self-heal algorithmus in config file?
>>
>> Thank you very much
>>
>> Roland Fischer
>>
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>      
>    

-- 

Roland Fischer
Technik

________________________________________________________________________________

Xidras GmbH
Stockern 47
3744 Stockern
Austria

Tel:     0043 2983 201 30 5 02
Fax:     0043 2983 201 30 5 02 9
Email:   roland.fischer at xidras.com
Web:     http://www.xidras.com

FN 317036 f | Landesgericht Krems | ATU64485024

________________________________________________________________________________

VERTRAULICHE INFORMATIONEN!
Diese eMail enthält vertrauliche Informationen und ist nur für den berechtigten Empfänger bestimmt. Wenn diese eMail nicht für Sie bestimmt ist, bitten wir Sie,diese eMail an uns zurückzusenden und anschließend auf Ihrem Computer und Mail-Server zu löschen. Solche eMails und Anlagen dürfen Sie weder nutzen, noch verarbeiten oder Dritten zugänglich machen, gleich in welcher Form.
Wir danken für Ihre Kooperation!

CONFIDENTIAL!
This email contains confidential information and is intended for the authorised recipient only. If you are not an authorised recipient, please return the email to us and then delete it from your computer and mail-server. You may neither use nor edit any such emails including attachments, nor make them accessible to third parties in any manner whatsoever.
Thank you for your cooperation
________________________________________________________________________________