[Gluster-devel] Simulating some kind of "virtual file"

Tue Jan 9 22:01:44 UTC 2018

Hi David,

adding again gluster-devel.

On Tue, Jan 9, 2018 at 4:15 PM, David Spisla <david.spisla at iternity.com>
wrote:

> Hello Xavi,
>
>
>
> *Von:* Xavi Hernandez [mailto:jahernan at redhat.com]
> *Gesendet:* Dienstag, 9. Januar 2018 09:48
> *An:* David Spisla <spisla80 at googlemail.com>
> *Cc:* gluster-devel at gluster.org
> *Betreff:* Re: [Gluster-devel] Simulating some kind of "virtual file"
>
>
>
> Hi David,
>
>
>
> On Tue, Jan 9, 2018 at 9:09 AM, David Spisla <spisla80 at googlemail.com>
> wrote:
>
> Dear Gluster Devels,
>
> at the moment I do some Xlator stuff and I want to know if there is a way
> to simulate the existing of a file to the client. It should be a kind of
> "virtual file". Here are more details.
>
> 1. Client lookup for a file e.g. "apple.test". This file does not exist in
> the backend
>
> $ ls -l apple.test
>
> ls: cannot access apple.test: No such file or directory
>
>
>
> Normally the system will not find that file
>
>
>
> 2. In the backend I have a real file called  e.g. "apple". Now there is a
> Xlator which manipulates the lookup request and is looking for the file
> "apple" instead of "apple.test". Gluster finds the file "apple" and the
> client will get a message from gluster that there is a file called
> "apple.test" with the attributes of the file "apple" (maybe we can
> manipulate that attributes too).
>
>
>
> Intercepting lookup is fine to be able to manipulate "virtual files",
> however it's not enough to completely operate on virtual files. You
> basically need to intercept all file operations that work on a loc or are
> path based and do the translation. You also need to intercept answers from
> readdir and readdirp to do the reverse transformation so that the user sees
> the virtual name and not the real name stored on the bricks.
>
> *[David Spisla] Yes, this is a very good hint. *
>
>
>
>
>
> $ ls -l apple.test
> -rw-r--r-- 1 davids davids 0 Jan  5 15:42 apple.test
>
>
>
> My first idea is, to have a special lookup and lookup_cbk in some Xlator
> in the server stack. Or it is better to have this Xlator in the Client
> Stack?
>
>
>
> That depends on what you are really trying to do. If you don't need any
> information or coordination with other bricks, you can safely create the
> xlator in the server stack.
>
> *[David Spisla] At the moment I do it in the worm xlator. It seems to be
> no problem.*
>
>
>
> The lookup function has a parameter called "loc_t *loc". In a first test I
> tried to manipulate loc->name and loc-path. If I manipulate loc->path I got
> an error and my volume crashed.
>
>
>
> You should be able to modify the loc without causing any crash. However
> there are some details you must be aware of:
>
>    - loc->path and/or loc->name can be NULL in some cases.
>    - If loc->path and loc->name are not NULL, loc->name always points to
>    a substring of loc->path. It's not allocated independently (and so it must
>    not be freed).
>    - If you change loc->path, you also need to change loc->name (to point
>    to the basename of loc->path)
>    - You shouldn't change the contents of loc->path directly. It's better
>    to allocate a new string with the modified path and assign it to loc->path
>    (you need to free the old value of loc->path to avoid memory leaks).
>
> *[David Spisla] I tried this:*
>
> *char *new_path = malloc(1+len_path-5);*
>
> *memcpy(new_path, loc->path, len_path-5);*
>
> *new_path[strlen(new_path)] = '\0';*
>
> *loc->name = new_path + (len_path - len_name);*
>

First of all, you should always use memory allocation functions from
gluster. This includes GF_MALLOC(), gf_strdup(), gf_asprintf() and several
other variants. You can look at libglusterfs/src/mem-pool.h to see all
available options.

The second problem I see is that memcpy() doesn't write a terminating null
character, so when you compute strlen() afterwards, it will return invalid
length, or even try to access invalid memory, causing a crash.

You should do something like this (assuming both loc->path and loc->name
are not NULL and skipping many necessary checks):

len_path = strlen(loc->path);

len_name = strlen(loc->name);

new_path = GF_MALLOC(len_path - 4, gf_common_mt_char);

memcpy(new_path, loc->path, len_path - 5);

new_path[len_path - 5] = 0;

loc->name = new_path + len_path - len_name;

This should work fine.

Xavi

>
>
> So, if I do this command:
>
> $ ls -l /test/dir/test1.txt.test
>
> Sometimes it is working but sometimes I got strange outputs in the
> lookup_function
>
> [2018-01-09 14:55:33.179568] I [worm.c:564:worm_lookup] 0-gv0-worm:
> checking if *.test
>
> [2018-01-09 14:55:33.179583] I [worm.c:584:worm_lookup] 0-gv0-worm: is
> *.test
>
> [2018-01-09 14:55:33.179596] I [worm.c:585:worm_lookup] 0-gv0-worm:
> /test/dir/test1.txtn_US.�
>
> [2018-01-09 14:55:33.179610] I [worm.c:592:worm_lookup] 0-gv0-worm: Length
> of path: 24
>
> [2018-01-09 14:55:33.179623] I [worm.c:594:worm_lookup] 0-gv0-worm: Length
> of name: 14
>
> [2018-01-09 14:55:33.179635] I [worm.c:595:worm_lookup] 0-gv0-worm:
> /test/dir/test1.txt.test
>
> [2018-01-09 14:55:33.179648] I [worm.c:596:worm_lookup] 0-gv0-worm:
> test1.txtn_US.�
>
> But it should be test1.txt.
>
> I think this should be enough to correctly change loc->path and loc->name.
>
> *[David Spisla] I am hanging around with that. I want to assign a new
> pointer to loc->path via „loc->path = new_path“, but that causes a memory
> crash:*
>
>
>
> [2018-01-09 15:14:02.340676] E [mem-pool.c:307:__gf_free]
> (-->/lib64/libglusterfs.so.0(args_wipe+0x12) [0x7f925414bb72]
> -->/lib64/libglusterfs.so.0(loc_wipe+0x27) [0x7f92540d2027]
> -->/lib64/libglusterfs.so.0(__gf_free+0xac) [0x7f92540fc6cc] ) 0-:
> Assertion failed: GF_MEM_HEADER_MAGIC == header->magic
>
>
>
> What ist he best way to allocate a new pointer to loc->path?
>
>
>
> Regards
>
> David
>
>
>
> Xavi
>
>
>
>
>
> Any hints?
>
> Regards
>
> David Spisla
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20180109/13696115/attachment-0001.html>