[Gluster-devel] Help needed: NFS Debugging for Glusto tests and Glusto help in general

[Nigel Babu]

On Mon, Nov 07, 2016 at 12:18:41PM +0530, shwetha wrote:
> On 11/07/2016 11:55 AM, Nigel Babu wrote:
> > Hello,
k >
> > I've been working on getting the Glusto tests to work and it appears that we're
> > stuck in a situation which Shewtha and Jonathan haven't been able to fully
> > arrive at a solution. Here are the two problems:
> >
> > 1. Originally, we ran into issues with NFS with an error that looked like this:
> >
> >          if 'nfs' in cls.mount_type:
> >              cmd = "showmount -e localhost"
> >              _, _, _ = g.run(cls.mnode, cmd)
> >
> >              cmd = "showmount -e localhost | grep %s" % cls.volname
> >              ret, _, _ = g.run(cls.mnode, cmd)
> > >            assert (ret == 0), "Volume %s not exported" % cls.volname
> > E           AssertionError: Volume testvol_replicated not exported
> > E           assert 1 == 0
> >
> > bvt/test_bvt_lite_and_plus.py:88: AssertionError
> >
> > Entries in glustomain.log:
> >
> > 2016-11-07 06:17:06,007 INFO (run) root at 172.19.2.69 (cp): gluster volume info | egrep "^Brick[0-9]+" | grep -v "ss_brick"
> > 2016-11-07 06:17:06,058 ERROR (get_servers_used_bricks_dict) error in getting bricklist using gluster v info
> > 2016-11-07 06:17:06,059 INFO (run) root at 172.19.2.69 (cp): gluster volume info testvol_replicated --xml
> > 2016-11-07 06:17:06,111 INFO (run) root at 172.19.2.69 (cp): gluster volume create testvol_replicated replica 3       172.19.2.69:/mnt/testvol_replicated_brick0 172.19.2.15:/mnt/testvol_replicated_brick1 172.19.2.3
> > 8:/mnt/testvol_replicated_brick2 --mode=script force
> > 2016-11-07 06:17:08,272 INFO (run) root at 172.19.2.69 (cp): gluster volume start testvol_replicated --mode=script
> > 2016-11-07 06:17:19,066 INFO (run) root at 172.19.2.69 (cp): gluster volume info testvol_replicated
> > 2016-11-07 06:17:19,125 INFO (run) root at 172.19.2.69 (cp): gluster vol status testvol_replicated
> > 2016-11-07 06:17:19,189 INFO (run) root at 172.19.2.69 (cp): showmount -e localhost
> > 2016-11-07 06:17:19,231 INFO (run) root at 172.19.2.69 (cp): showmount -e localhost | grep testvol_replicated
> > 2016-11-07 06:17:19,615 INFO (main) Ending glusto via main()
> > 2016-11-07 06:20:23,713 INFO (main) Starting glusto via main()
> >
> > Today I tried to comment out the NFS bits and run the test again. Here's what
> > that got me:
> 1)  You haven't commented out 'NFS bits' properly . 'showmount -e' will be
> executed only when the 'nfs' protocol is selected as mount protocol to run
> the tests. Ensure that you properly remove it from @runs_on decorator.

This is from before the NFS bits were commented out.

>
> 2) As i had told you before when we were discussing in IRC, the volume is
> not exported because gluster-nfs is not running.  Create a volume manually
> in your setup and set 'nfs.disable' volume option to 'off' and start the
> volume . Check if nfs process gets started and see if the volume you created
> gets exported with 'showmount -e localhost' command. If this works then we
> need to explicitly set this volume option in test case for running tests on
> 'nfs' protocol.

See Kaushal's reply. It looks like it's confirmed that we need to set
nfs.disable to off.

> >
> >          # Setup Volume
> >          ret = setup_volume(mnode=cls.mnode,
> >                             all_servers_info=cls.all_servers_info,
> >                             volume_config=cls.volume, force=True)
> > >        assert (ret == True), "Setup volume %s failed" % cls.volname
> > E       AssertionError: Setup volume testvol_distributed-replicated failed
> > E       assert False == True
> >
> > bvt/test_bvt_lite_and_plus.py:73: AssertionError
> >
> > Entries in glustomain.log:
> > 2016-11-07 06:20:34,994 INFO (run) root at 172.19.2.69 (cp): gluster volume info | egrep "^Brick[0-9]+" | grep -v "ss_brick"
> > 2016-11-07 06:20:35,048 ERROR (form_bricks_list) Not enough bricks available for creating the bricks
> > 2016-11-07 06:20:35,049 ERROR (setup_volume) Number_of_bricks is greater than the unused bricks on servers
> >
> > Does this make sense to anyone in terms of whether it's an error at Glusto-end
> > or an error in Gluster that's being caught?
> 3) Volume creation will fail if you do not have enough number of bricks on
> servers to create the volume.
>
> Please refer to the example config file for the volume_types section and
> it's default values.
>
> For example: in case of 'replicated' volume, we have replica_count as 3.
> Hence from all the servers collectively you definitely need 3 brick mounts.
>
> Ensure that you have enough bricks to run all volume types otherwise test
> cases will fail with the same error.

Aha, I hadn't realized this would be the case. I'll change my provisioning to
have more bricks.


--
nigelb