[Gluster-devel] RE: Help needed

Fri Jul 18 10:14:24 UTC 2008

Hi Krishna,

Thanks for your concern. 

We found that one server GFS-ACT-ST-007 is giving error "VFS: file-max limit
315868 reached". We've kept that server for fsck.

Dose any one know why this server reaching VFS: file-max limit every
frequently.

We had faced same problem with this server.

Rohan

-----Original Message-----
From: krishna.zresearch at gmail.com [mailto:krishna.zresearch at gmail.com] On
Behalf Of Krishna Srinivas
Sent: Friday, July 18, 2008 3:08 PM
To: Rohan
Cc: Amar S. Tumballi; Gluster-devel at nongnu.org
Subject: Re: [Gluster-devel] RE: Help needed

Rohan

Is this problem still there? can it be reproduced? can you give
us access to your machine to debug?

Krishna

On Thu, Jul 17, 2008 at 12:59 PM, Rohan <rohan.thale at moneycontrol.com>
wrote:
> 2008-07-17 20:54:14 E [afr.c:2177:afr_open] AFR-G3-B4: self heal failed,
> returning EIO
> 2008-07-17 20:54:14 E [unify.c:827:unify_open_cbk] bricks: Open success on
> namespace, failed on child node
> 2008-07-17 20:54:14 E [fuse-bridge.c:692:fuse_fd_cbk] glusterfs-fuse:
> 533995: (12)
> /mailbox/0/0/spam at in.com/mails/f2a205a8e4d754fe3f8311f6f958a94da22d9977 =>
> -1 (5)
> 2008-07-17 20:54:14 W [client-protocol.c:280:client_protocol_xfer]
> GFS-ACT-ST-007-4: attempting to pipeline request type(1) op(34) with
> handshake
> 2008-07-17 20:54:55 W [client-protocol.c:205:call_bail] GFS-ACT-ST-007-2:
> activating bail-out. pending frames = 1. last sent = 2008-07-17 20:54:14.
> last received = 2008-07-17 20:54:14 transport-timeout = 40
> 2008-07-17 20:54:55 C [client-protocol.c:212:call_bail] GFS-ACT-ST-007-2:
> bailing transport
> 2008-07-17 20:54:55 W [client-protocol.c:4777:client_protocol_cleanup]
> GFS-ACT-ST-007-2: cleaning up state in transport object 0x16e13ea0
> 2008-07-17 20:54:55 E [client-protocol.c:4827:client_protocol_cleanup]
> GFS-ACT-ST-007-2: forced unwinding frame type(2) op(5)
reply=@0x2aaabbc02b50
> 2008-07-17 20:54:55 E [client-protocol.c:4239:client_lock_cbk]
> GFS-ACT-ST-007-2: no proper reply from server, returning ENOTCONN
> 2008-07-17 20:54:55 E [afr.c:1909:afr_selfheal_lock_cbk] AFR-G3-B2:
>
(path=/mailbox/0/0/spam at in.com/mails/2e90b5b2e407829e95246df6052250ac21e2cdc
> 9o child=GFS-ACT-ST-007-2) op_ret=-1 op_errno=107
> 2008-07-17 20:54:55 E [afr.c:2177:afr_open] AFR-G3-B2: self heal failed,
> returning EIO
> 2008-07-17 20:54:55 E [unify.c:827:unify_open_cbk] bricks: Open success on
> namespace, failed on child node
> 2008-07-17 20:54:55 E [fuse-bridge.c:692:fuse_fd_cbk] glusterfs-fuse:
> 534020: (12)
> /mailbox/0/0/spam at in.com/mails/2e90b5b2e407829e95246df6052250ac21e2cdc9o
=>
> -1 (5)
> 2008-07-17 20:54:55 W [client-protocol.c:280:client_protocol_xfer]
> GFS-ACT-ST-007-2: attempting to pipeline request type(1) op(34) with
> handshake
> 2008-07-17 20:55:45 W [client-protocol.c:205:call_bail] GFS-ACT-ST-007-2:
> activating bail-out. pending frames = 1. last sent = 2008-07-17 20:54:55.
> last received = 2008-07-17 20:54:55 transport-timeout = 40
> 2008-07-17 20:55:45 C [client-protocol.c:212:call_bail] GFS-ACT-ST-007-2:
> bailing transport
> 2008-07-17 20:55:45 W [client-protocol.c:4777:client_protocol_cleanup]
> GFS-ACT-ST-007-2: cleaning up state in transport object 0x16e13ea0
> 2008-07-17 20:55:45 E [client-protocol.c:4827:client_protocol_cleanup]
> GFS-ACT-ST-007-2: forced unwinding frame type(2) op(5)
reply=@0x2aaabf1f6260
> 2008-07-17 20:55:45 E [client-protocol.c:4239:client_lock_cbk]
> GFS-ACT-ST-007-2: no proper reply from server, returning ENOTCONN
> 2008-07-17 20:55:45 E [afr.c:1909:afr_selfheal_lock_cbk] AFR-G3-B2:
>
(path=/mailbox/0/0/spam at in.com/mails/bc11dddb59644779d99bfdf39b7e56f226068a4
> 3 child=GFS-ACT-ST-007-2) op_ret=-1 op_errno=107
> 2008-07-17 20:55:45 E [afr.c:2177:afr_open] AFR-G3-B2: self heal failed,
> returning EIO
> 2008-07-17 20:55:45 E [unify.c:827:unify_open_cbk] bricks: Open success on
> namespace, failed on child node
> 2008-07-17 20:55:45 E [fuse-bridge.c:692:fuse_fd_cbk] glusterfs-fuse:
> 534035: (12)
> /mailbox/0/0/spam at in.com/mails/bc11dddb59644779d99bfdf39b7e56f226068a43 =>
> -1 (5)
> 2008-07-17 20:55:45 W [client-protocol.c:280:client_protocol_xfer]
> GFS-ACT-ST-007-2: attempting to pipeline request type(1) op(34) with
> handshake
> 2008-07-17 20:56:34 W [client-protocol.c:205:call_bail] GFS-ACT-ST-007-1:
> activating bail-out. pending frames = 1. last sent = 2008-07-17 20:55:46.
> last received = 2008-07-17 20:55:46 transport-timeout = 40
> 2008-07-17 20:56:34 C [client-protocol.c:212:call_bail] GFS-ACT-ST-007-1:
> bailing transport
> 2008-07-17 20:56:34 W [client-protocol.c:4777:client_protocol_cleanup]
> GFS-ACT-ST-007-1: cleaning up state in transport object 0x16e0d020
> 2008-07-17 20:56:34 E [client-protocol.c:4827:client_protocol_cleanup]
> GFS-ACT-ST-007-1: forced unwinding frame type(2) op(5)
reply=@0x2aaabbbfc200
> 2008-07-17 20:56:34 E [client-protocol.c:4239:client_lock_cbk]
> GFS-ACT-ST-007-1: no proper reply from server, returning ENOTCONN
> 2008-07-17 20:56:34 E [afr.c:1909:afr_selfheal_lock_cbk] AFR-G3-B1:
>
(path=/mailbox/0/0/spam at in.com/mails/d7ac63f0c00cc54887cc2181cca1c7fcc18f2c5
> 1 child=GFS-ACT-ST-007-1) op_ret=-1 op_errno=107
> 2008-07-17 20:56:34 E [afr.c:2177:afr_open] AFR-G3-B1: self heal failed,
> returning EIO
> 2008-07-17 20:56:34 E [unify.c:827:unify_open_cbk] bricks: Open success on
> namespace, failed on child node
> 2008-07-17 20:56:34 E [fuse-bridge.c:692:fuse_fd_cbk] glusterfs-fuse:
> 534062: (12)
> /mailbox/0/0/spam at in.com/mails/d7ac63f0c00cc54887cc2181cca1c7fcc18f2c51 =>
> -1 (5)
> 2008-07-17 20:56:34 W [client-protocol.c:280:client_protocol_xfer]
> GFS-ACT-ST-007-1: attempting to pipeline request type(1) op(34) with
> handshake
>
> -----Original Message-----
> From: krishna.zresearch at gmail.com [mailto:krishna.zresearch at gmail.com] On
> Behalf Of Krishna Srinivas
> Sent: Thursday, July 17, 2008 12:48 PM
> To: Rohan
> Cc: Amar S. Tumballi; Gluster-devel at nongnu.org
> Subject: Re: [Gluster-devel] RE: Help needed
>
> Rohan,
> Can access be given to these machines? Does the log have
> any useful information?
> Krishna
>
> On Thu, Jul 17, 2008 at 12:41 PM, Rohan <rohan.thale at moneycontrol.com>
> wrote:
>> No these are not symlinks.
>>
>>
>>
>>  _____
>>
>> From: amarts at gmail.com [mailto:amarts at gmail.com] On Behalf Of Amar S.
>> Tumballi
>> Sent: Thursday, July 17, 2008 12:34 PM
>> To: Rohan
>> Cc: Raghavendra G; Gluster-devel at nongnu.org
>> Subject: Re: [Gluster-devel] RE: Help needed
>>
>>
>>
>>
>> Can you make sure these are not symlinks? i have seen that if you issue
>> open() over symlink, it tries to open the target, and it may not exits,
so
>> returns ENOENT.
>>
>>
>>
>>
>> head: cannot open
>>
>
`/gfs-data1/mailbox/0/0/spam at in.com/mails/fc2080d273fb767df2660b52cd69b67eb1
>> <http://spam@in.com/mails/fc2080d273fb767df2660b52cd69b67eb18e56f6o>
>> 8e56f6o' for reading: No such file or directory
>>
>> head: cannot open
>>
>
`/gfs-data1/mailbox/0/0/spam at in.com/mails/6f444176af4a238239360800172a6a1a19
>> <http://spam@in.com/mails/6f444176af4a238239360800172a6a1a193afbef>
>> 3afbef' for reading: No such file or directory
>>
>> head: cannot open
>>
>
`/gfs-data1/mailbox/0/0/spam at in.com/mails/6bd73dbf7db1cf0c6acf56fa18d4f6fe63
>> <http://spam@in.com/mails/6bd73dbf7db1cf0c6acf56fa18d4f6fe63f950eco>
>> f950eco' for reading: No such file or directory
>>
>> head: cannot open
>>
>
`/gfs-data1/mailbox/0/0/spam at in.com/mails/3fec9cef25a2f43b1b1f0a712375dc23de
>> <http://spam@in.com/mails/3fec9cef25a2f43b1b1f0a712375dc23de30fb28o>
>> 30fb28o' for reading: No such file or directory
>>
>> head: cannot open
>>
>
`/gfs-data1/mailbox/0/0/spam at in.com/mails/0fd0ba2a4d8c52de7c52b09abbfec726c6
>> <http://spam@in.com/mails/0fd0ba2a4d8c52de7c52b09abbfec726c6e4e67bo>
>> e4e67bo' for reading: No such file or directory
>>
>> head: cannot open
>>
>
`/gfs-data1/mailbox/0/0/spam at in.com/mails/52fae85ff4366d7d8492d5154ead0659fd
>> <http://spam@in.com/mails/52fae85ff4366d7d8492d5154ead0659fd37f48do>
>> 37f48do' for reading: No such file or directory
>>
>>
>>
>>
>>
>> Why "No such file or directory" these errors are coming? These files are
> not
>> present in file system. Why find command finding them?
>>
>>
>>
>> Your help is really appreciated.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Rohan
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
>>
>>
>>
>> --
>> Amar Tumballi
>> Gluster/GlusterFS Hacker
>> [bulde on #gluster/irc.gnu.org]
>> http://www.zresearch.com - Commoditizing Super Storage!
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
>
>