[Gluster-devel] 回复： 回复： 回复：   glfs_creat this method hang up

Niels de Vos ndevos at redhat.com
Tue Sep 2 07:13:01 UTC 2014 

On Mon, Sep 01, 2014 at 02:48:36PM +0530, Soumya Koduri wrote:
> Hi Lixiaopo,
> 
> Looks like there is some issue/disconnect on the brick-server side. I am not
> much familiar with that code base. So have requested Pranith to look into
> the same. He is currently busy with few blocker issues, and so may take a
> little while to debug this.
> 
> Meanwhile I request people familiar with brick server code base to take a
> look at the logs.

I suggest you look at the user that runs the process which does the
glfs_creat(). If the user is unprivileged (not root), the process will
use source ports > 1024. By default, glusterd and the brick processes
deny these connections.

http://www.gluster.org/community/documentation/index.php/Libgfapi_with_qemu_libvirt
explains how to configure the Gluster side of the processes. You will
need to look into these particular items:

- Tuning glusterd to accept requests from QEMU
- Tuning glusterfsd to accept requests from QEMU

HTH,
Niels


> 
> Thanks,
> Soumya
> 
> On 09/01/2014 02:31 PM, ABC-new wrote:
> >Hi Soumya,‍
> >        May be description about log is not clearly.
> >        The logs "/var/log/glusterfs/*" list:
> >        cli.log,glustershd.log,nfs.log
> >The logs "/var/log/glusterfs/bricks/*" list:‍
> >        data-dr.log
> >The logs "/var/run/gluster/*.dump.timestamp‍" is:‍
> >data-dr.2432.dump.1409561882‍
> >        The java api that use uuid as file name works fine.
> >
> >Thank you in advance.
> >Lixiaopo
> >
> >------------------ 原始邮件 ------------------
> >*发件人:* "Soumya Koduri";<skoduri at redhat.com>;
> >*发送时间:* 2014年8月28日(星期四) 下午5:18
> >*收件人:* "ABC-new"<360532762 at qq.com>; "Pranith Kumar
> >Karampuri"<pkarampu at redhat.com>;
> >*抄送:* "Gluster Devel"<gluster-devel at gluster.org>;
> >*主题:* Re: 回复： 回复：   [Gluster-devel] glfs_creat this
> >method hang up
> >
> >Hi Lixiaopo,
> >
> >The logs which we are interested in are "/var/log/glusterfs/*" and
> >"/var/log/glusterfs/bricks/*" .
> >
> >To generate statedump, run the below command (when there is a hang while
> >running the glusterfs_example)-
> >gluster volume statedump <volname> (i.e, in your case 'gluster volume
> >statedump dr')
> >
> >The statedump info will then be copied to
> >"/var/run/gluster/*.dump.timestamp" files.
> >
> >Please share those files as well.
> >
> >Thanks,
> >Soumya
> >
> >
> >
> >On 08/28/2014 02:05 PM, ABC-new wrote:
> > > Thanks for your reply.
> > > [root at tsung150 gluster]# ps -ef|grep glusterfs
> > > root     11703     1  0 Aug21 ?        00:00:10 /usr/sbin/glusterfsd -s
> > > 192.168.108.150 --volfile-id dr.192.168.108.150.data-dr -p
> > > /var/lib/glusterd/vols/dr/run/192.168.108.150-data-dr.pid -S
> > > /var/run/0ad8935acf60ed0598b0a693a69f0e22.socket --brick-name /data/dr
> > > -l /var/log/glusterfs/bricks/data-dr.log --xlator-option
> > > *-posix.glusterd-uuid=c445c335-1d7e-4753-bd13-a83c4877083a --brick-port
> > > 49153 --xlator-option dr-server.listen-port=49153
> > > root     11712     1  0 Aug21 ?        00:00:08 /usr/sbin/glusterfs -s
> > > localhost --volfile-id gluster/nfs -p /var/lib/glusterd/nfs/run/nfs.pid
> > > -l /var/log/glusterfs/nfs.log -S
> > > /var/run/47d6f6e52026b112710ede46f4a73e11.socket
> > > root     11721     1  0 Aug21 ?        00:00:11 /usr/sbin/glusterfs -s
> > > localhost --volfile-id gluster/glustershd -p
> > > /var/lib/glusterd/glustershd/run/glustershd.pid -l
> > > /var/log/glusterfs/glustershd.log -S
> > > /var/run/9c6e4b112c349a36f3097585a2b3f773.socket --xlator-option
> > > *replicate*.node-uuid=c445c335-1d7e-4753-bd13-a83c4877083a
> > > root     16907 17242  0 16:31 pts/0    00:00:00 grep glusterfs‍
> > >
> > > Thank you,
> > > Lixiaopo
> > >
> > >
> > >
> > > ------------------ 原始邮件 ------------------
> > > *发件人:* "Soumya Koduri";<skoduri at redhat.com>;
> > > *发送时间:*
> >2014年8月28日(星期四) 下午4:11
> > > *收件人:* "ABC-new"<360532762 at qq.com>; "Pranith Kumar
> > > Karampuri"<pkarampu at redhat.com>;
> > > *抄送:* "Gluster Devel"<gluster-devel at gluster.org>;
> > > *主题:* Re: 回复：   [Gluster-devel] glfs_creat
> >this method hang up
> > >
> > > Thanks for the bt. Looks like brick process isn't responding here.
> > > Please collect logs and statedump info of the brick process while there
> > > is a hang.
> > >
> > > To generate statedump, refer to the below link -
> > >
> >https://github.com/gluster/glusterfs/blob/master/doc/debugging/statedump.md
> > >
> > > Thanks,
> > > Soumya
> > >
> > > On 08/28/2014 11:37 AM, ABC-new wrote:
> > >  >
> > >  > while hang,stack info:
> > >  >
> > >  >   Program received signal SIGINT, Interrupt.
> > >  > 0x0000003e1380b43c in pthread_cond_wait@@GLIBC_2.3.2 () from
> > > /lib64/libpthread.so.0
> > >  > Missing separate debuginfos, use: debuginfo-install
> > > glibc-2.12-1.107.el6.x86_64 glusterfs-3.4.0.57rhs-1.el6_5.x86_64
> > > glusterfs-api-3.4.0.57rhs-1.el6_5.x86_64
> > > glusterfs-libs-3.4.0.57rhs-1.el6_5.x86_64 keyutils-libs-1.4-4.el6.x86_64
> > > krb5-libs-1.10.3-10.el6.x86_64 libcom_err-1.41.12-14.el6.x86_64
> > > libselinux-2.0.94-5.3.el6.x86_64 openssl-1.0.1e-16.el6_5.14.x86_64
> > > uuid-1.6.1-10.el6.x86_64 zlib-1.2.3-29.el6.x86_64
> > >  > (gdb) bt
> > >  > #0  0x0000003e1380b43c in pthread_cond_wait@@GLIBC_2.3.2 () from
> > > /lib64/libpthread.so.0
> > >  > #1  0x00007ffff7b9903b in syncop_lookup () from
> > > /usr/lib64/libglusterfs.so.0
> > >  > #2  0x00007ffff7ddfd59 in glfs_first_lookup_safe () from
> > > /usr/lib64/libgfapi.so.0
> > >  > #3  0x00007ffff7ddfde7 in __glfs_first_lookup () from
> > > /usr/lib64/libgfapi.so.0
> > >  > #4  0x00007ffff7ddfe66 in __glfs_active_subvol () from
> > > /usr/lib64/libgfapi.so.0
> > >  > #5  0x00007ffff7de010f in glfs_active_subvol () from
> > > /usr/lib64/libgfapi.so.0
> > >  > #6  0x00007ffff7ddd0ff in glfs_creat () from /usr/lib64/libgfapi.so.0
> > >  > #7  0x00000000004014c2 in main (argc=1, argv=0x7fffffffe6f8) at
> > > glfs_example.c:80
> > >  > thank you.
> > >  > ‍Best Regards,
> > >  > Lixiaopo
> > >  >
> > >  >
> > >  > ------------------
> >原始邮件 ------------------
> > >  > *发件人:*
> >"360532762";<360532762 at qq.com>;
> > >  > *发送时间:*
> > >
> >2014年8月28日(星期四)
> >中午1:01
> > >  > *收件人:* "Soumya
> >Koduri"<skoduri at redhat.com>; "Pranith
> > > Kumar
> > >  > Karampuri"<pkarampu at redhat.com>;
> > >  > *抄送:* "Gluster
> >Devel"<gluster-devel at gluster.org>;
> > >  > *主题:* Re:  [Gluster-devel] glfs_creat this
> >method hang up
> > >  >
> > >  > Hi~:
> > >  > Soumya,‍
> > >  > the glusterfs_example.c is placed in directory /usr/local/glusterfs.
> > >  > I want to the file name generated by uuid.but now have not to do that
> > >  > yet,the variable filename is hard
> > >  > code.e.g."/2fcdec2e-688c-4077-bf96-4a42963dcffc.MOV‍".
> > >  > result is ok.
> > >  >
> > >  >   [root at localhost glusterfs]# pwd
> > >  > /usr/include/glusterfs
> > >  > [root at localhost glusterfs]# gcc -o glusterfs_example
> >glusterfs_example.c
> > >  > -lgfapi
> > >  > [root at localhost glusterfs]# ./glusterfs_example dr 192.168.108.150
> > >  > glfs_init: returned 0
> > >  > ----/2fcdec2e-688c-4077-bf96-4a42963dcffc.MOV----
> > >  > /2fcdec2e-688c-4077-bf96-4a42963dcffc.MOV: (0x20241e0) Success
> > >  > /2fcdec2e-688c-4077-bf96-4a42963dcffc.MOV: (0x20241e0) Structure needs
> > >  > cleaning
> > >  > read 32, hi there‍
> > >  >
> > >  > keep the source code not changed and then  I add "-luuid" to gcc ,
> > >  > compiled result is ok, but while run,hanging occured.
> > >  >
> > >  >   [root at localhost glusterfs]# gcc -o glusterfs_example
> > >  > glusterfs_example.c -lgfapi -luuid
> > >  > [root at localhost glusterfs]# ./glusterfs_example dr 192.168.108.151
> > >  > glfs_init: returned 0
> > >  > ----/2fcdec2e-688c-4077-bf96-4a42963dcffc.MOV----
> > >  > <------------------------------hanging  here
> > > ----------------------------->
> > >  >
> > >  >   source code :
> > >  >   #include <stdio.h>
> > >  > #include <stdlib.h>
> > >  > #include <errno.h>
> > >  > #include "api/glfs.h"
> > >  > #include "api/glfs-handles.h"
> > >  > #include <string.h>
> > >  > #include <time.h>
> > >  >
> > >  > int
> > >  > main (int argc, char *argv[])
> > >  > {
> > >  >    glfs_t    *fs2 = NULL;
> > >  >    int        ret = 0;
> > >  >    glfs_fd_t *fd = NULL;
> > >  >    glfs_fd_t *fd2 = NULL;
> > >  >    char       readbuf[32];
> > >  >    char       writebuf[32];
> > >  >    char      *filename = "/2fcdec2e-688c-4077-bf96-4a42963dcffc.MOV";
> > >  >
> > >  >    if (argc != 3) {
> > >  >      printf ("Expect following args\n\t%s <volname> <hostname>\n",
> > > argv[0]);
> > >  >      return -1;
> > >  >    }
> > >  >
> > >  >    fs2 = glfs_new (argv[1]);
> > >  >    if (!fs2) {
> > >  >      fprintf (stderr, "glfs_new: returned NULL\n");
> > >  >      return 1;
> > >  >    }
> > >  >    ret = glfs_set_volfile_server (fs2, "tcp", argv[2], 24007);
> > >  >    ret = glfs_set_logging (fs2, "/dev/stderr", 1);
> > >  >    ret = glfs_init (fs2);
> > >  >    fprintf (stderr, "glfs_init: returned %d\n", ret);
> > >  >
> > >  >    printf("----%s----\n", filename);
> > >  >    fd = glfs_creat (fs2, filename, O_RDWR, 0644);
> > >  >    fprintf (stderr, "%s: (%p) %s\n", filename, fd, strerror (errno));
> > >  >
> > >  >    fd2 = glfs_open (fs2, filename, O_RDWR);
> > >  >    fprintf (stderr, "%s: (%p) %s\n", filename, fd, strerror (errno));
> > >  >
> > >  >    sprintf (writebuf, "hi there\n");
> > >  >    ret = glfs_write (fd, writebuf, 32, 0);
> > >  >
> > >  >    glfs_lseek (fd2, 0, SEEK_SET);
> > >  >    ret = glfs_read (fd2, readbuf, 32, 0);
> > >  >    printf ("read %d, %s", ret, readbuf);
> > >  >
> > >  >    glfs_close (fd);
> > >  >    glfs_close (fd2);
> > >  >
> > >  >     glfs_fini (fs2);
> > >  >
> > >  >    return ret;
> > >  > }‍
> > >  >
> > >  > Thanks,
> > >  > Lixiaopo
> > >  > ------------------ Original ------------------
> > >  > *From: * "Soumya Koduri";<skoduri at redhat.com>;
> > >  > *Date: * Wed, Aug 27, 2014 07:42 PM
> > >  > *To: * "Pranith Kumar Karampuri"<pkarampu at redhat.com>;
> > >  > "ABC-new"<360532762 at qq.com>;
> > >  > *Cc: * "Gluster Devel"<gluster-devel at gluster.org>;
> > >  > *Subject: * Re: [Gluster-devel] glfs_creat this method hang up
> > >  >
> > >  > Could you please share your glusterfs_example code and the steps you
> > >  > have used to compile it and execute the binary? Would like to
> >check how
> > >  > the gfapi header files are linked.
> > >  >
> > >  > Thanks,
> > >  > Soumya
> > >  >
> > >  > On 08/27/2014 03:22 PM, Pranith Kumar Karampuri wrote:
> > >  >  > Guys who work with glfs_*, could you guys reply to this question.
> > >  >  >
> > >  >  > Pranith
> > >  >  > On 08/27/2014 03:16 PM, ABC-new wrote:
> > >  >  >> hi~:
> > >  >  >>
> > >  >  >>        while i run the glusterfs example via libgfapi, gcc -c
> > >  >  >> glusterfs_example -o glfs -luuid
> > >  >  >>        the method glfs_creat hang up.
> > >  >  >>
> > >  >  >>        I want to generate the uuid for file name.
> > >  >  >>
> > >  >  >>
> > >  >  >>    thank you.
> > >  >  >
> > >  >  > _______________________________________________
> > >  >  > Gluster-devel mailing list
> > >  >  > Gluster-devel at gluster.org
> > >  >  > http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel

More information about the Gluster-devel mailing list