[Gluster-users] Gluster not recognizing available space

[Pat Haley]

Hi,

Some additional data


[root at mseas-data save]# gluster volume info

Volume Name: gdata
Type: Distribute
Volume ID: eccc3a90-212d-4563-ae8d-10a77758738d
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: gluster-0-0:/mseas-data-0-0
Brick2: gluster-0-1:/mseas-data-0-1
Brick3: gluster-data:/data



[root at mseas-data save]# gluster volume status
Status of volume: gdata
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick gluster-0-0:/mseas-data-0-0                       24009   Y 
27006
Brick gluster-0-1:/mseas-data-0-1                       24009   Y       7063
Brick gluster-data:/data                                24010   N       8007
NFS Server on localhost                                 38467   Y       8013
NFS Server on gluster-0-1                               38467   Y 
10228
NFS Server on 10.1.1.10                                 38467   Y       3867


Noticing that the brick gluster-data:/data was appearing as N
in the "online" column, I tried (1) detaching gluster-data (using
gluster peer detach gluster-data issued from gluster-0-0), (2) removing
/var/lib/glusterd, (3) restarting glusterd on gluster-data,
(4) reattaching /gluster-data (using gluster peer probe gluster-data
issued from gluster-0-0) then (5) restart glusterd one more time on all
3 bricks.  The brick gluster-data:/data still appears as N in the Online
column.

[root at mseas-data save]# gluster peer status
Number of Peers: 2

Hostname: gluster-0-1
Uuid: 393fc4a6-1573-4564-971e-1b1aec434167
State: Peer in Cluster (Connected)

Hostname: 10.1.1.10
Uuid: 3619440a-4ca3-4151-b62e-d4d6bf2e0c03
State: Peer in Cluster (Connected)

(similarly from the other bricks)

Ping works between all bricks too.

> 
> Hi,
> 
> We are using gluster to present 3 bricks as a single name space.
> We appear to have a situation in which gluster thinks there
> is no disk space when there is actually plenty. I have restarted
> the glusterd deamons on all three bricks and I still get the
> following message
> 
> /bin/cp: cannot create regular file 
> `./Bottom_Gravity_Current_25/344.mat': No space left on device
> 
> This is a 6Mbyte file.  The total space available on
> gluster is 3.6T
> 
> Filesystem            Size  Used Avail Use% Mounted on
> mseas-data:/gdata      55T   51T  3.6T  94% /gdata
> 
> Also, no single brick is full:
> 
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/mapper/the_raid-lv_data
>                        15T   14T  804G  95% /data
> 
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sdb1              21T   18T  2.1T  90% /mseas-data-0-0
> 
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/sdb1              21T   20T  784G  97% /mseas-data-0-1
> 
> What should we do to fix this problem or look at to diagnose
> this problem?
> 
> Thanks.
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users


-- 

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley                          Email:  phaley at mit.edu
Center for Ocean Engineering       Phone:  (617) 253-6824
Dept. of Mechanical Engineering    Fax:    (617) 253-8125
MIT, Room 5-213                    http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA  02139-4301