<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:DengXian;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:DengXian;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal;
font-family:DengXian;
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal-compose;
font-family:DengXian;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ZH-CN" link="#0563C1" vlink="#954F72" style="text-justify-trim:punctuation">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi, Gluster Expert,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Sorry for wrong code “open(argv[1], O_CREAT|O_RDWR|S_IRUSR|S_IWUSR) “in previous mail, after correct it , the issues seems GONE.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">But I still confuse why wrong code not failed in EXT4 file system.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">After I check tcpdump packet via network, it show that in CREATE process, the mode of file sometime became to “ Mode: 0101000,
<span style="background:yellow;mso-highlight:yellow">S_ISVTX, Reserved</span> </span>
“<span lang="EN-US">, with this mode, the file could be seen with “ls” command, but can’t seen by “hexdump”command even with root privilege, but if run this wrong program in ext4 file system, this MODE can’t be occurred.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">So I suggest maybe Gluster could do some inprovment for this MODE , so that can avoid thiss can’t access file with wrong API called.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">@Sami, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">After I check the manual of open API, the correct format should be
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> int open(const char *pathname, int flags);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> int open(const char *pathname, int flags, mode_t mode);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">for your test code, I suppose the correct call should be “open(argv[1], O_CREAT|O_RDWR<span style="background:yellow;mso-highlight:yellow">,</span>S_IRUSR|S_IWUSR) “<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">after I correct it , c and C++ code all PASS in 3.6 and 3.12.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Best Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">George<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" align="left" style="text-align:left"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Lian, George (NSB - CN/Hangzhou)
<br>
<b>Sent:</b> Wednesday, December 27, 2017 5:23 PM<br>
<b>To:</b> Gluster-devel@gluster.org<br>
<b>Cc:</b> Zhong, Hua (NSB - CN/Hangzhou) <hua.zhong@nokia-sbell.com>; Venetjoki, Sami (Nokia - FI/Espoo) <sami.venetjoki@nokia.com>; Li, Deqian (NSB - CN/Hangzhou) <deqian.li@nokia-sbell.com><br>
<b>Subject:</b> Sometimes read failed just after write before file closed when try 100 times in different new files<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" align="left" style="text-align:left"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Hi, Gluster expert,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I has just raise a bug with <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1529237">
https://bugzilla.redhat.com/show_bug.cgi?id=1529237</a> on gluster community.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I also attached the reproduce source code and shell script here FYI.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">From my investigation , I suppose the issue maybe caused by
</span>“<span lang="EN-US">cache in client is invalid but the file still not been persistented in server brick</span>”<span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">But I can</span>’<span lang="EN-US">t find the root cause from the debug log which attached in
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1529237">https://bugzilla.redhat.com/show_bug.cgi?id=1529237</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">From the following log I abstract client log from the attachment in the Bugzilla, I highlight it with
<span style="background:yellow;mso-highlight:yellow">YELLOW</span> which in read phase, you can see
<span style="background:yellow;mso-highlight:yellow">both volume(test-client-0 and test-client-1) are failed</span>
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I just suppose it maybe some abnormal when it in read phase, but I can</span>’<span lang="EN-US">t any usefully log in server brick log from myself, so your helps is highly appreciated!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332524] T [MSGID: 0] [afr-inode-read.c:286:afr_fstat_cbk] 12-stack-trace: stack-address: 0x7fb5ec010680, test-replicate-0 returned 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332724] T [MSGID: 0] [syncop.c:1715:syncop_fgetxattr] 12-stack-trace: stack-address: 0x7fb5fc0618d0, winding from test-dht to test-replicate-0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332768] D [MSGID: 0] [afr-read-txn.c:220:afr_read_txn] 12-test-replicate-0: 70e0f7ee-2f0a-4364-a38e-96e7031eee7b: generation now vs cached: 2, 2<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332791] T [MSGID: 0] [afr-inode-read.c:1728:afr_fgetxattr_wind] 12-stack-trace: stack-address: 0x7fb5fc0618d0, winding from test-replicate-0 to test-client-0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332817] T [rpc-clnt.c:1496:rpc_clnt_record] 12-test-client-0: Auth Info: pid: 5257, uid: 0, gid: 0, owner: 0000000000000000<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332837] T [rpc-clnt.c:1353:rpc_clnt_record_build_header] 12-rpc-clnt: Request fraglen 140, payload: 64, rpc hdr: 76<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.332862] T [rpc-clnt.c:1699:rpc_clnt_submit] 12-rpc-clnt: submitted request (XID: 0x2681 Program: GlusterFS 3.3, ProgVers: 330, Proc: 35) to rpc-transport (test-client-0)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.333943] T [rpc-clnt.c:675:rpc_clnt_reply_init] 12-test-client-0: received rpc message (RPC XID: 0x2681 Program: GlusterFS 3.3, ProgVers: 330, Proc: 35) from rpc-transport (test-client-0)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334013] D [MSGID: 0] [<span style="background:yellow;mso-highlight:yellow">client-rpc-fops.c:1143:client3_3_fgetxattr_cbk] 12-test-client-0: remote operation failed: No data available</span><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334036] D [MSGID: 0] [client-rpc-fops.c:1151:client3_3_fgetxattr_cbk] 12-stack-trace: stack-address: 0x7fb5fc0618d0, test-client-0 returned -1 error:
<span style="background:yellow;mso-highlight:yellow">No data available [No data available]</span><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334114] T [MSGID: 0] [afr-common.c:1306:afr_inode_refresh_subvol_with_fstat] 12-stack-trace: stack-address: 0x7fb5fc0618d0, winding from test-replicate-0 to test-client-0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334174] T [rpc-clnt.c:1496:rpc_clnt_record] 12-test-client-0: Auth Info: pid: 5257, uid: 0, gid: 0, owner: 0000000000000000<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334196] T [rpc-clnt.c:1353:rpc_clnt_record_build_header] 12-rpc-clnt: Request fraglen 336, payload: 260, rpc hdr: 76<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334229] T [rpc-clnt.c:1699:rpc_clnt_submit] 12-rpc-clnt: submitted request (XID: 0x2682 Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (test-client-0)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334280] T [MSGID: 0] [afr-common.c:1306:afr_inode_refresh_subvol_with_fstat] 12-stack-trace: stack-address: 0x7fb5fc0618d0, winding from test-replicate-0 to test-client-1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334311] T [rpc-clnt.c:1496:rpc_clnt_record] 12-test-client-1: Auth Info: pid: 5257, uid: 0, gid: 0, owner: 0000000000000000<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334330] T [rpc-clnt.c:1353:rpc_clnt_record_build_header] 12-rpc-clnt: Request fraglen 336, payload: 260, rpc hdr: 76<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334354] T [rpc-clnt.c:1699:rpc_clnt_submit] 12-rpc-clnt: submitted request (XID: 0x252c Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (test-client-1)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334904] T [rpc-clnt.c:675:rpc_clnt_reply_init] 12-test-client-1: received rpc message (RPC XID: 0x252c Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) from rpc-transport (test-client-1)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.334983] T [MSGID: 0] [client-rpc-fops.c:1462:client3_3_fstat_cbk] 12-stack-trace: stack-address: 0x7fb5fc0618d0, test-client-1 returned 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335128] T [rpc-clnt.c:675:rpc_clnt_reply_init] 12-test-client-0: received rpc message (RPC XID: 0x2682 Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) from rpc-transport (test-client-0)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335165] T [MSGID: 0] [client-rpc-fops.c:1462:client3_3_fstat_cbk] 12-stack-trace: stack-address: 0x7fb5fc0618d0, test-client-0 returned 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335193] T [MSGID: 0] [afr-inode-read.c:1728:afr_fgetxattr_wind] 12-stack-trace: stack-address: 0x7fb5fc0618d0, winding from test-replicate-0 to test-client-1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335224] T [rpc-clnt.c:1496:rpc_clnt_record] 12-test-client-1: Auth Info: pid: 5257, uid: 0, gid: 0, owner: 0000000000000000<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335253] T [rpc-clnt.c:1353:rpc_clnt_record_build_header] 12-rpc-clnt: Request fraglen 140, payload: 64, rpc hdr: 76<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335284] T [rpc-clnt.c:1699:rpc_clnt_submit] 12-rpc-clnt: submitted request (XID: 0x252d Program: GlusterFS 3.3, ProgVers: 330, Proc: 35) to rpc-transport (test-client-1)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2017-12-27 06:21:04.335870] T [rpc-clnt.c:675:rpc_clnt_reply_init] 12-test-client-1: received rpc message (RPC XID: 0x252d Program: GlusterFS 3.3, ProgVers: 330, Proc: 35) from rpc-transport (test-client-1)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="background:yellow;mso-highlight:yellow">[2017-12-27 06:21:04.335902] D [MSGID: 0] [client-rpc-fops.c:1143:client3_3_fgetxattr_cbk] 12-test-client-1: remote operation failed: No data available<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="background:yellow;mso-highlight:yellow">[2017-12-27 06:21:04.335922] D [MSGID: 0] [client-rpc-fops.c:1151:client3_3_fgetxattr_cbk] 12-stack-trace: stack-address: 0x7fb5fc0618d0, test-client-1 returned -1 error:
No data available [No data available]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="background:yellow;mso-highlight:yellow">[2017-12-27 06:21:04.335950] D [MSGID: 0] [afr-inode-read.c:1720:afr_fgetxattr_wind] 12-stack-trace: stack-address: 0x7fb5fc0618d0, test-replicate-0 returned -1 error: No
data available [No data available]</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Best Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">George<o:p></o:p></span></p>
</div>
</body>
</html>