[Gluster-devel] 3.6.1 issue

Mon Dec 22 16:21:15 UTC 2014

That did not fix the issue (see below).  I also have run into another 
possibly related issue.  After untarring the boost directory and 
compiling the software, I cannot delete the source directory structure.  
It says "direct not empty".

corvidpost5:temp3/gfs> \rm -r boost_1_57_0
rm: cannot remove `boost_1_57_0/libs/numeric/odeint/test': Directory not 
empty
corvidpost5:temp3/gfs> cd boost_1_57_0/libs/numeric/odeint/test/
corvidpost5:odeint/test> ls -al
total 0
drwxr-x--- 3 dfrobins users  94 Dec 20 01:51 ./
drwx------ 3 dfrobins users 100 Dec 20 01:51 ../

cluster.read-hash-mode to 2 results

corvidpost5:TankExamples/DakotaList> ls -al
total 5
drwxr-x--- 2 dfrobins users  166 Dec 22 11:16 ./
drwxr-x--- 6 dfrobins users  445 Dec 22 11:16 ../
lrwxrwxrwx 1 dfrobins users   25 Dec 22 11:16 EvalTank.py -> 
../tank_model/EvalTank.py*
---------- 1 dfrobins users    0 Dec 22 11:16 FEMTank.py
-rwx--x--- 1 dfrobins users  734 Nov  7 11:05 RunTank.sh*
-rw------- 1 dfrobins users 1432 Nov  7 11:05 dakota_PandL_list.in
-rw------- 1 dfrobins users 1860 Nov  7 11:05 dakota_Ponly_list.in

gluster> volume info homegfs

Volume Name: homegfs
Type: Distributed-Replicate
Volume ID: 1e32672a-f1b7-4b58-ba94-58c085e59071
Status: Started
Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: gfsib01a.corvidtec.com:/data/brick01a/homegfs
Brick2: gfsib01b.corvidtec.com:/data/brick01b/homegfs
Brick3: gfsib01a.corvidtec.com:/data/brick02a/homegfs
Brick4: gfsib01b.corvidtec.com:/data/brick02b/homegfs
Brick5: gfsib02a.corvidtec.com:/data/brick01a/homegfs
Brick6: gfsib02b.corvidtec.com:/data/brick01b/homegfs
Brick7: gfsib02a.corvidtec.com:/data/brick02a/homegfs
Brick8: gfsib02b.corvidtec.com:/data/brick02b/homegfs
Options Reconfigured:
cluster.read-hash-mode: 2
performance.stat-prefetch: off
performance.io-thread-count: 32
performance.cache-size: 128MB
performance.write-behind-window-size: 128MB
server.allow-insecure: on
network.ping-timeout: 10
storage.owner-gid: 100
geo-replication.indexing: off
geo-replication.ignore-pid-check: on
changelog.changelog: on
changelog.fsync-interval: 3
changelog.rollover-time: 15
server.manage-gids: on

------ Original Message ------
From: "Vijay Bellur" <vbellur at redhat.com>
To: "David F. Robinson" <david.robinson at corvidtec.com>
Cc: "Justin Clift" <justin at gluster.org>; "Gluster Devel" 
<gluster-devel at gluster.org>
Sent: 12/22/2014 9:23:44 AM
Subject: Re: [Gluster-devel] 3.6.1 issue

>On 12/21/2014 11:10 PM, David F. Robinson wrote:
>>So for now it is up to all of the individual users to know they cannot 
>>use tar without the -P switch if they are accessing a data storage 
>>system that uses gluster?
>>
>
>Setting volume option cluster.read-hash-mode to 2 could help here. Can 
>you please check if this resolves the problem without -P switch?
>
>-Vijay
>
>
>>>On Dec 21, 2014, at 12:30 PM, Vijay Bellur <vbellur at redhat.com> 
>>>wrote:
>>>
>>>>On 12/20/2014 12:09 PM, David F. Robinson wrote:
>>>>Seems to work with "-xPf". I obviously couldn't check all of the 
>>>>files,
>>>>but the two specific ones that I noted in my original email do not 
>>>>show
>>>>any problems when using -P...
>>>
>>>This is related to the way tar extracts symbolic links by default & 
>>>its interaction with GlusterFS. In a nutshell the following steps are 
>>>involved in creation of symbolic links on the destination:
>>>
>>>a) Create an empty regular placeholder file with permission bits set 
>>>to 0 and the name being that of the symlink source file.
>>>
>>>b) Record the device, inode numbers and the mtime of the placeholder 
>>>file through stat.
>>>
>>>c) After the first pass of extraction is complete, there is a second 
>>>pass involved to set right symbolic links. In this phase a stat is 
>>>performed on the placeholder file. If all attributes recorded in b) 
>>>are in sync with the latest information from stat buf, only then the 
>>>placeholder is unlinked and a new symbolic link is created. If any 
>>>attribute is out of sync, the unlink and creation of symbolic link do 
>>>not happen.
>>>
>>>In the case of replicated GlusterFS volumes, the mtimes can vary 
>>>across nodes during the creation of placeholder files. If the stat 
>>>calls in steps b) and c) land on different nodes, then there is a 
>>>very good likelihood that tar would skip creation of symbolic links 
>>>and leave behind the placeholder files.
>>>
>>>A little more detail about this particular implementation behavior of 
>>>symlinks for tar can be found at [1].
>>>
>>>To overcome this behavior, we can make use of the P switch with tar 
>>>command during extraction which will create the link file directly 
>>>and not go ahead with the above set of steps.
>>>
>>>Keeping timestamps in sync across the cluster will help to an extent 
>>>in preventing this situation. There are ongoing refinements in 
>>>replicate's selection of read-child which will help in addressing 
>>>this problem.
>>>
>>>-Vijay
>>>
>>>[1] http://lists.debian.org/debian-user/2003/03/msg03249.html
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20141222/bb45ca21/attachment-0001.html>