[Gluster-users] trying to avoid a penalty for renaming every file

[Joe Julian]

The best slide of all from the Gluster Developer Summit

https://twitter.com/JoeCyberGuru/status/784310321980674048


On 10/27/2016 11:52 AM, Christian Rice wrote:
> We have a gluster pool where our developers are writing large files 
> with the suffix .COPYING, then removing that suffix when the copy is 
> done (renaming the file), so consumers of the data know the file is 
> safe to read.
>
> Back in 2014, Jeff Darcy wrote the below in a thread.  I guess I am 
> basically asking if this is still the supported/encouraged approach, 
> and how do I implement this when I’m using gluster’s NFS?
>
> "That said, there's also a trick you can use to avoid creation of a
> linkfile.  Other tools, such as rsync and our own object interface,
> use the same write-then-rename idiom.  To serve them, there's an
> option called extra-hash-regex that can be used to place files on the
> "right" brick according to their final name even though they're created
> with another.  Unfortunately, specifying that option via the command line
> doesn't seem to work (it creates a malformed volfile) so you have to
> mount a bit differently.  For example:
>
>    glusterfs --volfile-server=a_server --volfile-id=a_volume \
> --xlator-option a_volume-dht.extra_hash_regex='(.*+)tmp' \
>    /a/mountpoint
>
> The important part is that second line.  That causes any file with a
> "tmp" suffix to be hashed and placed as though only the part in the
> first parenthesized part of the regex (i.e. without the "tmp") was
> there.  Therefore, creating "xxxtmp" and then renaming it to "xxx" is
> the same as just creating "xxx" in the first place as far as linkfiles
> etc. are concerned.  Note that the excluded part can be anything that
> a regex can match, including a unique random number.  If I recall,
> rsync uses temp files something like this:
>
>    fubar = .fubar.NNNNNN (where NNNNNNN is a random number)
>
> I know this probably seems a little voodoo-ish, but with a little bit
> of experimentation to find the right regex you should be able to avoid
> those dreaded linkfiles altogether."
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161027/f36be574/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot from 2016-10-27 13-22-04.png
Type: image/png
Size: 114132 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161027/f36be574/attachment.png>