[Gluster-users] Gluster not recognizing available space

[Pat Haley]

Hi Joe,

Sorry to take so long in responding, but we had another emergency
that took all my time...

The subsampled brick log file from gluster-0-1 is available at

http://mseas.mit.edu/download/phaley/GlusterUsers/gluster-0-1/bricks/mseas-data-0-1.log.1

The df results on gluster-0-1 are

[root at nas-0-1 glusterfs]# df /mseas-data-0-1/
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb1            21484242304 20662833060 821409244  97% /mseas-data-0-1

[root at nas-0-1 glusterfs]# df -i /mseas-data-0-1/
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sdb1            3290028528 4391552 3285636976    1% /mseas-data-0-1

I've also included the same time period log files from gluster-0-1 and
the other bricks in

http://mseas.mit.edu/download/phaley/GlusterUsers/gluster-data/
http://mseas.mit.edu/download/phaley/GlusterUsers/gluster-0-0/
http://mseas.mit.edu/download/phaley/GlusterUsers/gluster-0-1/

Thanks for the help!

Pat

> 1st, I don't think we need log data from November. Let's try to share 
> logs that are limited to an example of the error you're trying to get 
> help with. A 40Mb client log is unnecessary when it's only the last 5k 
> that's relevant.
> 
> 2nd, your "server" logs are mostly more client logs, with the exception 
> of etc-glusterfs-glusterd.vol.log. The brick logs that might actually 
> show what's happening are absent. (/var/log/glusterfs/bricks/*)
> 
> The last line of the client log does show that 
> gluster-0-1:/mseas-data-0-1 is reporting that it's out of space. On 
> gluster-0-1 please share the log /var/log/glusterfs/bricks/gluster-0-1 
> but limit it to no more than an hour surrounding 2014-01-21 11:50:16. 
> Also on that server, please share the result of "df /gluster-0-1 ; df -i 
> /gluster-0-1".
> 
> On 01/21/2014 10:58 PM, Pat Haley wrote:
>>
>> Hi Vijay,
>>
>> I've put the log files in
>>
>> http://mseas.mit.edu/download/phaley/GlusterUsers/server_logs/
>> http://mseas.mit.edu/download/phaley/GlusterUsers/client_logs/
>>
>>> On 01/21/2014 10:24 PM, Pat Haley wrote:
>>>>
>>>> Hi Joe,
>>>>
>>>> The peer status on all 3 showed the
>>>> proper connections.  Doing the killall
>>>> and restart on all three bricks fixed
>>>> the N in the Online column.  I then
>>>> did have to remount the gluster
>>>> filesystem on the client
>>>>
>>>> Unfortunately my original problem remains.
>>>> I'm still getting "no space on left on device"
>>>> when I try to write even a small file to
>>>> the gluster filesystem.
>>>>
>>>> What should I look at next?
>>>>
>>>
>>> Would it be possible to share the brick and client log files?
>>>
>>> Thanks,
>>> Vijay
>>>
>>>
>>>
>>
>>
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users


-- 

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley                          Email:  phaley at mit.edu
Center for Ocean Engineering       Phone:  (617) 253-6824
Dept. of Mechanical Engineering    Fax:    (617) 253-8125
MIT, Room 5-213                    http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA  02139-4301