[Gluster-users] Advice for building samba-glusterfs-vfs

Thu Oct 3 23:21:02 UTC 2013

Thanks for the pointers Dave.   That got me to compiling the module.

I'm still using the RPM-installed GlusterFS and Samba, but have
compiled/installed matching source versions into a --prefix=/localdev
folder and used that to build the samba-glusterfs-vfs module with
relevant paths during the "./configure" step.

The module was copied to /usr/lib64/samba/vfs, vfs_glusterfs.so was
symlinked to glusterfs.so, and appears to be working.

Relevant smb.conf

[sharename]
        vfs object = glusterfs
        glusterfs:volume = volumename
        path = /

Testing so far has been much faster for read/browse operations, so
that much is good.

However, bugs that already make it unusable for me at the moment:

* renaming files and folders from a Windows host gives a "does not
exist" error in the Explorer GUI, and a "duplicate filename exists"
error from cmd.exe.

* Brick logs fill up with 4 lines of "getxattr failed" messages for
each file or folder read, and this is with all GlusterFS error log
levels already set to "critical".  With a network full of users, I'm
going to fill disks with logs in minutes.  I had seen this reported in
3.3.1 and marked as fixed in 3.4.0, but it appears to still be around.
 I have XFS formatted disk backing each brick with 512 byte inodes and
default mount options.  I don't wish to turn on user_xattr, as I want
to just discard any of that sort of data coming from Mac and Windows
hosts, as it's unnecessary for our production data.  But at the same
time, I don't want GlusterFS telling me it's an error when it's not
there (INFO level would suffice, so I can ignore that in my logs).

-Dan

On 3 October 2013 10:59, David Gibbons <david.c.gibbons at gmail.com> wrote:
> Dan,
>
> I had the same trouble yesterday. As it happens, I created a doc to help
> script installs of future nodes. I did not snip out the portions that apply
> to what you're looking for, but the script below works for me.
>
> The biggest issue was that some modules were apparently installed in lib and
> the build process was looking for them in lib64. In any event, the vfs
> module builds, installs and runs cleanly after this.
>
> ** The big win for me was finding this command, that allowed me to figure
> out where it was looking for modules in the wrong lib directory.
> ldd /usr/local/samba/lib/vfs/glusterfs.so
>
> I'm sure there is any easier way to to do this :)...
>
> Cheers,
> Dave
>
>
> --
>
> #!/bin/bash
>
> yum groupinstall "Development Tools"
>
> yum install git openssl-devel wget
>
> yum install libtalloc libtdb
>
>
> # set up gluster
>
> cd /usr/src && git clone https://github.com/gluster/glusterfs.git
>
> cd /usr/src/glusterfs && ./autogen.sh && ./configure && make
>
> make install
>
>
> # set up samba 3.6.9
>
> cd /usr/src && wget http://ftp.samba.org/pub/samba/stable/samba-3.6.9.tar.gz
> && tar -zxvf samba-3.6.9.tar.gz
>
> cd /usr/src/samba-3.6.9/source3 && ./configure && make
>
> make install
>
>
> ln -s /usr/local/samba/lib/libwbclient.so.0 /usr/lib64/libwbclient.so.0
>
>
> # then install the RPM samba version
>
> yum install samba
>
>
> # set up vfs_glusterfs
>
> cd /usr/src && git clone
> git://forge.gluster.org/samba-glusterfs/samba-glusterfs-vfs.git
>
>
> ln -s /usr/local/include/glusterfs /usr/include/glusterfs
>
>
> cd /usr/src/samba-vfs/glusterfs
>
> ./configure --with-samba-source=/usr/src/samba-3.6.9/source3
>
>
> ln -s /usr/local/samba/lib/vfs/glusterfs.so
> /usr/lib64/samba/vfs/glusterfs.so
>
>
> # link the other modules
>
> ln -s /usr/local/lib/libgfapi.so /usr/lib64/
>
> ln -s /usr/local/lib/libgfapi.so.0 /usr/lib64/
>
> ln -s /usr/local/lib/libgfapi.la /usr/lib64/
>
> ln -s /usr/local/lib/libglusterfs.la /usr/lib64/
>
> ln -s /usr/local/lib/libglusterfs.so /usr/lib64/
>
> ln -s /usr/local/lib/libglusterfs.so.0 /usr/lib64/
>
> ln -s /usr/local/lib/libglusterfs.so.0.0.0 /usr/lib64/
>
> EOF
>
>
> On Tue, Oct 1, 2013 at 10:38 PM, Dan Mons <dmons at cuttingedge.com.au> wrote:
>>
>> Hi folks,
>>
>> I've got CentOS6.4 with Samba 3.6.9 installed from the standard CentOS
>> repos via yum.  I also have GlusterFS 3.4.0 GA installed from RPMs
>> direct from gluster.org.
>>
>> I'm trying to build the glusterfs VFS module for Samba to take
>> advantage of libgfapi for our Windows users, and migrate them off the
>> current Samba-on-FUSE setup we have currently.
>>
>> I've downloaded the appropriate source trees for all projects
>> (GlusterFS from gluster.org, Samba from the matching CentOS6 SRPM, and
>> samba-glusterfs-vfs from the git repo), but am facing troubles early
>> on just finding appropriate headers.
>>
>> [root at bne-gback000 samba-glusterfs-vfs]# find
>> /usr/local/src/glusterfs-3.4.0 -type f -name glfs.h
>> /usr/local/src/glusterfs-3.4.0/api/src/glfs.h
>>
>> [root at bne-gback000 samba-glusterfs-vfs]# ./configure
>> --with-glusterfs=/usr/local/src/glusterfs-3.4.0
>> *snip*
>> checking api/glfs.h usability... no
>> checking api/glfs.h presence... no
>> checking for api/glfs.h... no
>> Cannot find api/glfs.h. Please specify --with-glusterfs=dir if necessary
>>
>> If I install glusterfs-api-devel-3.4.0-8.el6.x86_64.rpm, I need to
>> copy /usr/include/glusterfs/api/glfs.h to /usr/include for it to be
>> found (even using --with-glusterfs= doesn't work), and then I get
>> further errors about not being able to link to glfs_init:
>>
>> [root at bne-gback000 samba-glusterfs-vfs]# rpm -ivh
>> /tmp/glusterfs-api-devel-3.4.0-8.el6.x86_64.rpm
>> [root at bne-gback000 samba-glusterfs-vfs]# cp
>> /usr/include/glusterfs/api/glfs.h /usr/include/
>> [root at bne-gback000 samba-glusterfs-vfs]# ./configure
>> *snip*
>> checking api/glfs.h usability... yes
>> checking api/glfs.h presence... yes
>> checking for api/glfs.h... yes
>> checking for glfs_init... no
>> Cannot link to gfapi (glfs_init). Please specify --with-glusterfs=dir
>> if necessary
>>
>> If anyone can point me in the right direction, that would be greatly
>> appreciated.
>>
>> Cheers,
>>
>> -Dan
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>

-- 
Dan Mons
R&D SysAdmin
Cutting Edge
http://cuttingedge.com.au