<div dir="ltr">Hi David,<div><br></div><div>adding again gluster-devel.</div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 9, 2018 at 4:15 PM, David Spisla <span dir="ltr"><<a href="mailto:david.spisla@iternity.com" target="_blank">david.spisla@iternity.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="DE" link="blue" vlink="purple">
<div class="m_-1186107839114350758WordSection1">
<p class="MsoNormal"><span>Hello Xavi, <u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Von:</b> Xavi Hernandez [mailto:<a href="mailto:jahernan@redhat.com" target="_blank">jahernan@redhat.com</a>] <br>
<b>Gesendet:</b> Dienstag, 9. Januar 2018 09:48<br>
<b>An:</b> David Spisla <<a href="mailto:spisla80@googlemail.com" target="_blank">spisla80@googlemail.com</a>><br>
<b>Cc:</b> <a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a><br>
<b>Betreff:</b> Re: [Gluster-devel] Simulating some kind of "virtual file"<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi David,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div><span class="">
<p class="MsoNormal">On Tue, Jan 9, 2018 at 9:09 AM, David Spisla <<a href="mailto:spisla80@googlemail.com" target="_blank">spisla80@googlemail.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Dear Gluster Devels,<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">at the moment I do some Xlator stuff and I want to know if there is a way to simulate the existing of a file to the client. It should be a kind of "virtual file". Here are more details.<u></u><u></u></p>
</div>
<p class="MsoNormal">1. Client lookup for a file e.g. "apple.test". This file does not exist in the backend
<u></u><u></u></p>
</div>
<p class="MsoNormal">$ ls -l apple.test<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">ls: cannot access apple.test: No such file or directory<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Normally the system will not find that file<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">2. In the backend I have a real file called e.g. "apple". Now there is a Xlator which manipulates the lookup request and is looking for the file "apple" instead of "apple.test". Gluster finds the file "apple" and the client will get a
message from gluster that there is a file called "apple.test" with the attributes of the file "apple" (maybe we can manipulate that attributes too).<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</span><div><span class="">
<p class="MsoNormal">Intercepting lookup is fine to be able to manipulate "virtual files", however it's not enough to completely operate on virtual files. You basically need to intercept all file operations that work on a loc or are path based and do the translation.
You also need to intercept answers from readdir and readdirp to do the reverse transformation so that the user sees the virtual name and not the real name stored on the bricks.<u></u><u></u></p>
</span><p class="MsoNormal"><b><i>[David Spisla] Yes, this is a very good hint. </i></b><u></u><u></u></p>
</div><span class="">
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">$ ls -l apple.test<br>
-rw-r--r-- 1 davids davids 0 Jan 5 15:42 apple.test<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">My first idea is, to have a special lookup and lookup_cbk in some Xlator in the server stack. Or it is better to have this Xlator in the Client Stack?<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</span><div><span class="">
<p class="MsoNormal">That depends on what you are really trying to do. If you don't need any information or coordination with other bricks, you can safely create the xlator in the server stack.<u></u><u></u></p>
</span><p class="MsoNormal"><b><i>[David Spisla] At the moment I do it in the worm xlator. It seems to be no problem.</i></b><u></u><u></u></p>
</div><span class="">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<div>
<p class="MsoNormal">The lookup function has a parameter called "loc_t *loc". In a first test I tried to manipulate loc->name and loc-path. If I manipulate loc->path I got an error and my volume crashed.<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">You should be able to modify the loc without causing any crash. However there are some details you must be aware of:<u></u><u></u></p>
</div>
</span><div><span class="">
<ul type="disc">
<li class="MsoNormal" style="margin-left:0cm">
loc->path and/or loc->name can be NULL in some cases.<u></u><u></u></li><li class="MsoNormal" style="margin-left:0cm">
If loc->path and loc->name are not NULL, loc->name always points to a substring of loc->path. It's not allocated independently (and so it must not be freed).<u></u><u></u></li><li class="MsoNormal" style="margin-left:0cm">
If you change loc->path, you also need to change loc->name (to point to the basename of loc->path)<u></u><u></u></li><li class="MsoNormal" style="margin-left:0cm">
You shouldn't change the contents of loc->path directly. It's better to allocate a new string with the modified path and assign it to loc->path (you need to free the old value of loc->path to avoid memory leaks).<u></u><u></u></li></ul>
</span><p class="MsoNormal"><b><i>[David Spisla] I tried this:<u></u><u></u></i></b></p>
<p class="MsoNormal"><i>char *new_path = malloc(1+len_path-5);<u></u><u></u></i></p>
<p class="MsoNormal"><i>memcpy(new_path, loc->path, len_path-5);<u></u><u></u></i></p>
<p class="MsoNormal"><i>new_path[strlen(new_path)] = '\0';<u></u><u></u></i></p>
<p class="MsoNormal"><i>loc->name = new_path + (len_path - len_name);</i></p></div></div></div></div></div></div></div></blockquote><div><br></div><div>First of all, you should always use memory allocation functions from gluster. This includes GF_MALLOC(), gf_strdup(), gf_asprintf() and several other variants. You can look at libglusterfs/src/mem-pool.h to see all available options.</div><div><br></div><div>The second problem I see is that memcpy() doesn't write a terminating null character, so when you compute strlen() afterwards, it will return invalid length, or even try to access invalid memory, causing a crash.</div><div><br></div><div>You should do something like this (assuming both loc->path and loc->name are not NULL and skipping many necessary checks):</div><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">len_path = strlen(loc->path);</font></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">len_name = strlen(loc->name);</font></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">new_path = GF_MALLOC(len_path - 4, gf_common_mt_char);</font></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">memcpy(new_path, loc->path, len_path - 5);</font></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">new_path[len_path - 5] = 0;</font></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">loc->name = new_path + len_path - len_name;</font></div></div></div></blockquote><div style="font-family:monospace,monospace"><font face="monospace, monospace"><br></font></div><font face="arial, helvetica, sans-serif">This should work fine.</font><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Xavi<br></font><div class="gmail_extra"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="DE" link="blue" vlink="purple"><div class="m_-1186107839114350758WordSection1"><div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div><div><div><p class="MsoNormal"><i><u></u><u></u></i></p>
<p class="MsoNormal"><i><u></u> <u></u></i></p>
<p class="MsoNormal">So, if I do this command:<u></u><u></u></p>
<p class="MsoNormal">$ ls -l /test/dir/test1.txt.test<u></u><u></u></p>
<p class="MsoNormal">Sometimes it is working but sometimes I got strange outputs in the lookup_function<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:5.25pt">
[2018-01-09 14:55:33.179568] I [worm.c:564:worm_lookup] 0-gv0-worm: checking if *.test<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:5.25pt">
[2018-01-09 14:55:33.179583] I [worm.c:584:worm_lookup] 0-gv0-worm: is *.test<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:5.25pt">
[2018-01-09 14:55:33.179596] I [worm.c:585:worm_lookup] 0-gv0-worm: /test/dir/test1.txtn_US.<span style="font-family:"Tahoma",sans-serif">�</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-left:5.25pt">
[2018-01-09 14:55:33.179610] I [worm.c:592:worm_lookup] 0-gv0-worm: Length of path: 24<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:5.25pt">
[2018-01-09 14:55:33.179623] I [worm.c:594:worm_lookup] 0-gv0-worm: Length of name: 14<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:5.25pt">
[2018-01-09 14:55:33.179635] I [worm.c:595:worm_lookup] 0-gv0-worm: /test/dir/test1.txt.test<u></u><u></u></p>
<p class="MsoNormal">[2018-01-09 14:55:33.179648] I [worm.c:596:worm_lookup] 0-gv0-worm: test1.txtn_US.<span style="font-family:"Tahoma",sans-serif">�<u></u><u></u></span></p>
<p class="MsoNormal">But it should be test1.txt.
<u></u><u></u></p>
<div><span class="">
<p class="MsoNormal">I think this should be enough to correctly change loc->path and loc->name.<u></u><u></u></p>
</span><p class="MsoNormal"><b><i>[David Spisla] I am hanging around with that. I want to assign a new pointer to loc->path via „loc->path = new_path“, but that causes a memory crash:<u></u><u></u></i></b></p>
<p class="MsoNormal"><b><i><u></u> <u></u></i></b></p>
<p class="MsoNormal">[2018-01-09 15:14:02.340676] E [mem-pool.c:307:__gf_free] (-->/lib64/libglusterfs.so.0(<wbr>args_wipe+0x12) [0x7f925414bb72] -->/lib64/libglusterfs.so.0(<wbr>loc_wipe+0x27) [0x7f92540d2027] -->/lib64/libglusterfs.so.0(__<wbr>gf_free+0xac) [0x7f92540fc6cc]
) 0-: Assertion failed: GF_MEM_HEADER_MAGIC == header->magic<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">What ist he best way to allocate a new pointer to loc->path?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Regards<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></p><span class="HOEnZb"><font color="#888888">
<p class="MsoNormal">David<u></u><u></u></p>
</font></span></div>
</div><span class="">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Xavi<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Any hints?<u></u><u></u></p>
</div>
<p class="MsoNormal">Regards<u></u><u></u></p>
</div>
<p class="MsoNormal"><span class="m_-1186107839114350758hoenzb"><span style="color:#888888">David Spisla
</span></span><u></u><u></u></p>
</div>
<p class="MsoNormal"><br>
______________________________<wbr>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@gluster.org" target="_blank">Gluster-devel@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-devel</a><u></u><u></u></p>
</blockquote>
</span></div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div>
</div>
</blockquote></div><br></div></div></div>