[Gluster-users] Too many open files

[Pranith Kumar Karampuri]

On Fri, Jul 29, 2016 at 8:49 AM, Pranith Kumar Karampuri <
pkarampu at redhat.com> wrote:

>
>
> On Fri, Jul 29, 2016 at 8:43 AM, Gmail <b.s.mikhael at gmail.com> wrote:
>
>> I’ve already done that in the previous e-mail.
>>
>
> Those were state dump files, the command failed and was saying check logs
> for details, so was asking for logfiles as well.
>

I went through the statedump files and don't see any open-fds. Could you
see if there are a lot of files opened in the output of 'ls -l
/proc/<brick-pid-where-you-saw-the-failures>/fd'. Please attach this output
as well. Are you still seeing this issue? Or does it happen only when you
run the application?


>
>>
>> *— Bishoy*
>>
>> On Jul 28, 2016, at 3:20 PM, Gmail <b.s.mikhael at gmail.com> wrote:
>>
>> <glusterdump.glusterd.dump.1469742056>
>> <glusterdump.glusterfs.dump.1469742040>
>>
>> On Jul 27, 2016, at 4:54 PM, Pranith Kumar Karampuri <pkarampu at redhat.com>
>> wrote:
>>
>>
>>
>> On Wed, Jul 27, 2016 at 11:26 PM, Gmail <b.s.mikhael at gmail.com> wrote:
>>
>>> Hello All,
>>>
>>> I keep getting the following errors accompanied by the bricks in the
>>> same replica group going offline when trying to sync files between two
>>> volumes located in two different geographic locations using Csync2.
>>>
>>> [2016-07-27 17:07:06.701575] E [MSGID: 113015]
>>> [posix.c:1011:posix_opendir] 0-gvol001-posix: opendir failed on
>>> /zfsvol/brick02/gvol001/.glusterfs/f2/28/f228f015-2bf4-43a3-bfa6-803f282f13af/man3
>>> [Too many open files]
>>> [2016-07-27 17:07:06.701643] E [MSGID: 115056]
>>> [server-rpc-fops.c:690:server_opendir_cbk] 0-gvol001-server: 46179928:
>>> OPENDIR
>>> <gfid:25c95c97-d5a4-415c-baed-b87b847b2ec1>/perl-5.18.1/cpan/Test-Harness/blib/man3
>>> (10bc989e-395f-44ec-911e-746335f567a0) ==> (Too many open files) [Too many
>>> open files]
>>> [2016-07-27 17:07:06.702574] E [MSGID: 113099]
>>> [posix-helpers.c:389:_posix_xattr_get_set] 0-gvol001-posix: Opening file
>>> /zfsvol/brick02/gvol001/.glusterfs/7f/af/7faf6eff-4036-4603-a974-7e95c8fdd568
>>> failed [Too many open files]
>>> [2016-07-27 17:07:06.709581] E [MSGID: 113099]
>>> [posix-helpers.c:389:_posix_xattr_get_set] 0-gvol001-posix: Opening file
>>> /zfsvol/brick02/gvol001/.glusterfs/dd/d2/ddd2c86e-c296-463c-9d03-2da596bb9f88
>>> failed [Too many open files]
>>>
>>
>> 1) May be there is a bug in the brick stack which forgets to close a file.
>> 2) May be Csync2 doesn't close the files after syncing.
>> Could you let us know the version of gluster and also statedump of the
>> bricks using
>> https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/
>>
>> I’ve tried to collect the logs as per the documentation provided, but not
>> all of the logs was successfully collected.
>>
>> # gluster volume statedump gvol001
>> volume statedump: failed: Commit failed on <HOSTNAME>. Please check log
>> file for details.
>>
>>
>>
>>>
>>> *—Bishoy*
>>>
>>> _______________________________________________
>>> Gluster-users mailing list
>>> Gluster-users at gluster.org
>>> http://www.gluster.org/mailman/listinfo/gluster-users
>>>
>>
>>
>>
>> --
>> Pranith
>>
>>
>>
>
>
> --
> Pranith
>



-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160729/33c3160b/attachment.html>