[Gluster-devel] crash with 1.4.0tla846
Dan Parsons
dparsons at nyip.net
Wed Jan 14 20:11:12 UTC 2009
And while we're waiting for my next test to complete, here's my client
config and server config files:
client:
volume unify-switch-ns
type protocol/client
option transport-type tcp
option remote-host 10.8.101.51
option remote-subvolume posix-unify-switch-ns
end-volume
#volume distfs01-ns-readahead
# type performance/read-ahead
# option page-size 1MB
# option page-count 8
# subvolumes distfs01-ns-brick
#end-volume
#volume unify-switch-ns
# type performance/write-behind
# option block-size 1MB
# option cache-size 3MB
# subvolumes distfs01-ns-readahead
#end-volume
volume distfs01-unify
type protocol/client
option transport-type tcp
option remote-host 10.8.101.51
option remote-subvolume posix-unify
end-volume
volume distfs02-unify
type protocol/client
option transport-type tcp
option remote-host 10.8.101.52
option remote-subvolume posix-unify
end-volume
volume distfs03-unify
type protocol/client
option transport-type tcp
option remote-host 10.8.101.53
option remote-subvolume posix-unify
end-volume
volume distfs04-unify
type protocol/client
option transport-type tcp
option remote-host 10.8.101.54
option remote-subvolume posix-unify
end-volume
volume distfs01-stripe
type protocol/client
option transport-type tcp
option remote-host 10.8.101.51
option remote-subvolume posix-stripe
end-volume
volume distfs02-stripe
type protocol/client
option transport-type tcp
option remote-host 10.8.101.52
option remote-subvolume posix-stripe
end-volume
volume distfs03-stripe
type protocol/client
option transport-type tcp
option remote-host 10.8.101.53
option remote-subvolume posix-stripe
end-volume
volume distfs04-stripe
type protocol/client
option transport-type tcp
option remote-host 10.8.101.54
option remote-subvolume posix-stripe
end-volume
volume stripe0
type cluster/stripe
option block-size *.jar,*.pin:1MB,*:2MB
subvolumes distfs01-stripe distfs02-stripe distfs03-stripe distfs04-
stripe
end-volume
volume dht0
type cluster/dht
subvolumes distfs01-unify distfs02-unify distfs03-unify distfs04-unify
end-volume
volume unify
type cluster/unify
option namespace unify-switch-ns
option self-heal off
option scheduler switch
# send *.phr/psq/pnd etc to stripe0, send the rest to hash
# extensions have to be *.foo* and not simply *.foo or rsync's tmp
file naming will prevent files from being matched
option scheduler.switch.case
*.phr*:stripe0;*.psq*:stripe0;*.pnd*:stripe0;*.psd*:stripe0;*.pin*:stripe0;*.nsi*:stripe0;*.nin*:stripe0;*.nsd*:stripe0;*.nhr*:stripe0;*.nsq*:stripe0;*.tar*:stripe0;*.tar.gz*:stripe0;*.jar*:stripe0;*.img*:stripe0;*.perf*:stripe0;*.tgz*:stripe0;*.fasta*:stripe0;*.huge*:stripe0
subvolumes stripe0 dht0
end-volume
volume ioc
type performance/io-cache
subvolumes unify
option cache-size 1500MB
end-volume
#volume filter
# type features/filter
# option fixed-uid 0
# option fixed-gid 900
# subvolumes ioc
#end-volume
server:
volume posix-unify
type storage/posix
option directory /distfs-storage-space/glusterfs/unify
# the below line is here to make the output of 'df' accurate, as both
volumes are served from the same local drive
option export-statfs-size off
end-volume
volume posix-stripe
type storage/posix
option directory /distfs-storage-space/glusterfs/stripe
end-volume
volume posix-unify-switch-ns
type storage/posix
option directory /distfs-storage-space/glusterfs/unify-switch-
ns
end-volume
volume server
type protocol/server
option transport-type tcp
option auth.addr.posix-unify.allow 10.8.101.*
option auth.addr.posix-stripe.allow 10.8.101.*
option auth.addr.posix-unify-switch-ns.allow 10.8.101.*
subvolumes posix-unify posix-stripe posix-unify-switch-ns
end-volume
Dan Parsons
On Jan 14, 2009, at 11:42 AM, Anand Avati wrote:
> Dan,
> for the purpose of debugging, can you do 'option self-heal off' in
> unify and try again (with a fresh log file)?
>
> thanks,
> avati
>
> 2009/1/15 Dan Parsons <dparsons at nyip.net>:
>> Just to clarify, the below problems occurred on tla 846, which is
>> to my
>> knowledge, the latest.
>>
>>
>> Dan Parsons
>>
>>
>> On Jan 14, 2009, at 11:24 AM, Dan Parsons wrote:
>>
>>> Every time I try to do a big rsync operation, the glusterfs client
>>> is
>>> crashing on me. First there are a ton of self-heal messages, one per
>>> directory that rsync is examining, and then eventually a segfault.
>>> I tried
>>> using tla814 (the officially released rc7) but that broke even
>>> faster and
>>> with different error messages.
>>>
>>> ANY help on this would be appreciated, I thought I was nearing the
>>> end of
>>> this project :-(
>>>
>>> OS: CentOS 5.2
>>> Kernel: 2.6.23.14
>>> Fuse: 2.7.3glfs10
>>>
>>> Errors:
>>> (imagine a million more of the below line)
>>> 2009-01-14 11:20:11 W [unify-self-heal.c:
>>> 593:unify_sh_checksum_cbk] unify:
>>> Self-heal triggered on directory /bio/db/hmm/panther/books/
>>> PTHR10210/SF3
>>> pending frames:
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>> frame : type(1) op(LOOKUP)
>>>
>>> patchset: glusterfs--mainline--3.0--patch-846
>>> signal received: 11
>>> configuration details:argp 1
>>> backtrace 1
>>> db.h 1
>>> dlfcn 1
>>> fdatasync 1
>>> libpthread 1
>>> llistxattr 1
>>> setfsid 1
>>> spinlock 1
>>> epoll.h 1
>>> xattr.h 1
>>> st_atim.tv_nsec 1
>>> package-string: glusterfs 1.4.0tla846
>>> /lib64/libc.so.6[0x359ba301b0]
>>> /lib64/libc.so.6[0x359ba71865]
>>> /lib64/libc.so.6(__libc_malloc+0x7d)[0x359ba72efd]
>>> /lib64/libc.so.6(__strdup+0x22)[0x359ba78092]
>>> /usr/local/lib/libglusterfs.so.0(loc_copy+0x40)[0x2b63d6dbebc0]
>>>
>>> /usr/local/lib/glusterfs/1.4.0tla846/xlator/cluster/
>>> unify.so(unify_lookup+0x85)[0x2b63d76b0bb5]
>>>
>>> /usr/local/lib/glusterfs/1.4.0tla846/xlator/performance/io-
>>> cache.so(ioc_lookup+0xc0)[0x2b63d78bd0e0]
>>>
>>> /usr/local/lib/glusterfs/1.4.0tla846/xlator/features/
>>> filter.so(filter_lookup+0xb5)[0x2b63d7ac5135]
>>> /usr/local/lib/glusterfs/1.4.0tla846/xlator/mount/
>>> fuse.so[0x2b63d7cd2175]
>>> /usr/local/lib/glusterfs/1.4.0tla846/xlator/mount/
>>> fuse.so[0x2b63d7cd4c91]
>>> /lib64/libpthread.so.0[0x359ce06307]
>>> /lib64/libc.so.6(clone+0x6d)[0x359bad1ded]
>>> ---------
>>>
>>> Client config:
>>>
>>> volume unify-switch-ns
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.51
>>> option remote-subvolume posix-unify-switch-ns
>>> end-volume
>>>
>>> #volume distfs01-ns-readahead
>>> # type performance/read-ahead
>>> # option page-size 1MB
>>> # option page-count 8
>>> # subvolumes distfs01-ns-brick
>>> #end-volume
>>>
>>> #volume unify-switch-ns
>>> # type performance/write-behind
>>> # option block-size 1MB
>>> # option cache-size 3MB
>>> # subvolumes distfs01-ns-readahead
>>> #end-volume
>>>
>>> volume distfs01-unify
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.51
>>> option remote-subvolume posix-unify
>>> end-volume
>>>
>>> volume distfs02-unify
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.52
>>> option remote-subvolume posix-unify
>>> end-volume
>>>
>>> volume distfs03-unify
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.53
>>> option remote-subvolume posix-unify
>>> end-volume
>>>
>>> volume distfs04-unify
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.54
>>> option remote-subvolume posix-unify
>>> end-volume
>>>
>>> volume distfs01-stripe
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.51
>>> option remote-subvolume posix-stripe
>>> end-volume
>>>
>>> volume distfs02-stripe
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.52
>>> option remote-subvolume posix-stripe
>>> end-volume
>>>
>>> volume distfs03-stripe
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.53
>>> option remote-subvolume posix-stripe
>>> end-volume
>>>
>>> volume distfs04-stripe
>>> type protocol/client
>>> option transport-type tcp
>>> option remote-host 10.8.101.54
>>> option remote-subvolume posix-stripe
>>> end-volume
>>>
>>> volume stripe0
>>> type cluster/stripe
>>> option block-size *.jar,*.pin:1MB,*:2MB
>>> subvolumes distfs01-stripe distfs02-stripe distfs03-stripe
>>> distfs04-stripe
>>> end-volume
>>>
>>> volume dht0
>>> type cluster/dht
>>> subvolumes distfs01-unify distfs02-unify distfs03-unify
>>> distfs04-unify
>>> end-volume
>>>
>>> volume unify
>>> type cluster/unify
>>> option namespace unify-switch-ns
>>> option scheduler switch
>>> # send *.phr/psq/pnd etc to stripe0, send the rest to hash
>>> # extensions have to be *.foo* and not simply *.foo or rsync's tmp
>>> file
>>> naming will prevent files from being matched
>>> option scheduler.switch.case
>>> *.phr*:stripe0;*.psq*:stripe0;*.pnd*:stripe0;*.psd*:stripe0;*.pin*:stripe0;*.nsi*:stripe0;*.nin*:stripe0;*.nsd*:stripe0;*.nhr*:stripe0;*.nsq*:stripe0;*.tar*:stripe0;*.tar.gz*:stripe0;*.jar*:stripe0;*.img*:stripe0;*.perf*:stripe0;*.tgz*:stripe0;*.fasta*:stripe0;*.huge*:stripe0
>>> subvolumes stripe0 dht0
>>> end-volume
>>>
>>> volume ioc
>>> type performance/io-cache
>>> subvolumes unify
>>> option cache-size 1500MB
>>> end-volume
>>>
>>> volume filter
>>> type features/filter
>>> option fixed-uid 0
>>> option fixed-gid 900
>>> subvolumes ioc
>>> end-volume
>>>
>>>
>>>
>>> Server config:
>>>
>>> volume posix-unify
>>> type storage/posix
>>> option directory /distfs-storage-space/glusterfs/unify
>>> # the below line is here to make the output of 'df' accurate, as
>>> both
>>> volumes are served from the same local drive
>>> option export-statfs-size off
>>> end-volume
>>>
>>> volume posix-stripe
>>> type storage/posix
>>> option directory /distfs-storage-space/glusterfs/stripe
>>> end-volume
>>>
>>> volume posix-unify-switch-ns
>>> type storage/posix
>>> option directory /distfs-storage-space/glusterfs/unify-switch-
>>> ns
>>> end-volume
>>>
>>> volume server
>>> type protocol/server
>>> option transport-type tcp
>>> option auth.addr.posix-unify.allow 10.8.101.*
>>> option auth.addr.posix-stripe.allow 10.8.101.*
>>> option auth.addr.posix-unify-switch-ns.allow 10.8.101.*
>>> subvolumes posix-unify posix-stripe posix-unify-switch-ns
>>> end-volume
>>>
>>> Dan Parsons
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at nongnu.org
>>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>>
>>
>>
>>
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>>
>
More information about the Gluster-devel
mailing list