[Gluster-users] Question on the rename atomicity behavior in gluster

[Mohammed Rafi K C]

On 03/23/2016 04:10 AM, Rama Shenai wrote:
> Hi, We had some questions with respect to expectations of atomicity of
> rename in gluster.
>
> To elaborate : 
>
> We have setup with two machines (lets call them M1 and M2) which
> essentially access a file (F) on a gluster volume (mounted by M1 and M2)
> A program does the following steps sequentially on each of the two
> machines (M1 & M2) in an infinite loop
>  1) Opens the file F in O_RDWR|O_EXCL mode, reads some data and closes (F)
>  2)  Renames some other file F' => F
>  
> Periodically either M1 or M2 sees a "Stalefile handle error" when it
> tries to read the file (step (1)) after opening the file in
> O_RDWR|O_EXCL (the open is successful)
>
> The specific error reported the client volume logs
>  (/var/log/glusterfs/mnt-repos-volume1.log)
> [2016-03-21 16:53:17.897902] I [dht-rename.c:1344:dht_rename]
> 0-volume1-dht: renaming master.lock
> (hash=volume1-replicate-0/cache=volume1-replicate-0) => master
> (hash=volume1-replicate-0/cache=<nul>)
> [2016-03-21 16:53:18.735090] W
> [client-rpc-fops.c:504:client3_3_stat_cbk] 0-volume1-client-0: remote
> operation failed: Stale file handle

Hi Rama,

ESTALE error in rename normally generated when either the source file is
not resolvable (deleted or inaccessible) or when the parent of
destination is not resolvable. It can happen when let's say file F' was
present when your application did a lookup before rename, but if it is
got renamed by Node M1 before M2 could rename it. Basically a race
between two rename on the same file can result in ESTALE for either of one.

To confirm this, Can you please paste the log message from brick
"0-volume1-client-0". You can find out the brick name from the graph.

Also if you can share the program or snippet that used to reproduce this
issue, that would be great.

Rafi KC



>
> We see no error when: have two processes of the above program running
> on the same machine (say on M1) accessing the file F on the gluster
> volume, for which we want to understand the expectations of atomicity
> in gluster specifically specifically for rename, and if the above is a
> bug.
>
> Also  glusterfs --version => glusterfs 3.6.9 built on Mar  2 2016 18:21:14
>
> We also would like to know if there any parameter in the one
> translators  that we can tweak to prevent this problem
>
> Any help or insights here is appreciated
>
> Thanks
> Rama
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160323/a8011d84/attachment.html>