[Gluster-users] glusterfs missing files on ls
Xavier Hernandez
xhernandez at datalab.es
Fri May 31 12:50:11 UTC 2013
Al 31/05/13 11:48, En/na Stefano Sinigardi ha escrit:
> Dear Xavier,
> I realized that the volume was not build properly when doing the first
> analyses suggested by Davide, but I'm sure that this is not the
> problem and so I quickly dismissed it. Also, we need a replica but not
> so strictly, maybe in the future with the next volume I'll build it
> properly. Anyway, yes, the volume got birth on "pedrillo" with a
> replica-2 and the next day was expanded on "osmino", again with
> replica-2, just by adding bricks and doing a rebalance, that was just
> tried. I'm saying "tried" because it got "stuck", consuming a lot of
> RAM (almost all, 16 GB), and it was counting million of files that I
> think don't even exist on the volume, so I stopped it. Do you think
> that it might be worth restarting?
> The brick logs are there, just before your nice reply asking for them.
> They're all the same, except for the order of the connection, all seem
> very good.
Sorry I don't know what I saw. Yes, the logs are there... :p
> Yes, you're right, many files are missing through the mounpoint, I
> just choose one. But then again, what do you think about the fact that
> ls hides them but calling them from the mount point are not counted as
> "file not found"? "leggi_particelle" was working calling it from the
> bricks and from the mountpoint, even if ls didn't show it in the
> mountpoint...
It can happen. When you list a directory, a file may not be shown if it
has some problem. However it can be reached if you access it directly (I
have seen this before).
> And here it is all the other info that you were asking: first of all a
> collection of ls of the folder on all the bricks of the volume (10)
> /storage/1/data/stefano/leggi_particelle:
> total 20
> drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./
> drwxr-xr-x 14 stefano user 4096 May 28 04:58 ../
> -rwxr-xr-x 2 stefano user 286 Feb 25 17:24 Espec.plt*
> lrwxrwxrwx 2 stefano user 53 Feb 13 11:30 parametri.cpp ->
> /tamino/stefano/codice/leggi_particelle/parametri.cpp
> drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/
> /storage/2/data/stefano/leggi_particelle:
> total 20
> drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./
> drwxr-xr-x 14 stefano user 4096 May 28 04:58 ../
> -rwxr-xr-x 2 stefano user 286 Feb 25 17:24 Espec.plt*
> lrwxrwxrwx 2 stefano user 53 Feb 13 11:30 parametri.cpp ->
> /tamino/stefano/codice/leggi_particelle/parametri.cpp
> drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/
> /storage/5/data/stefano/leggi_particelle:
> total 892
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./
> drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../
> lrwxrwxrwx 2 stefano user 50 Apr 11 19:20 filtro.cpp ->
> /tamino/stefano/codice/leggi_particelle/filtro.cpp
> lrwxrwxrwx 2 stefano user 70 Apr 11 19:20
> leggi_binario_ALaDyn_fortran.h ->
> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.h
> -rwxr-xr-x 2 stefano user 705045 May 22 18:24 leggi_particelle*
> -rwxr-xr-x 2 stefano user 61883 Dec 16 17:20 leggi_particelle.old01*
> -rwxr-xr-x 2 stefano user 106014 Apr 11 19:20 leggi_particelle.old03*
> ---------T 2 root root 0 May 24 17:16 parametri.cpp
> drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/
> /storage/6/data/stefano/leggi_particelle:
> total 892
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./
> drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../
> lrwxrwxrwx 2 stefano user 50 Apr 11 19:20 filtro.cpp ->
> /tamino/stefano/codice/leggi_particelle/filtro.cpp
> lrwxrwxrwx 2 stefano user 70 Apr 11 19:20
> leggi_binario_ALaDyn_fortran.h ->
> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.h
> -rwxr-xr-x 2 stefano user 705045 May 22 18:24 leggi_particelle*
> -rwxr-xr-x 2 stefano user 61883 Dec 16 17:20 leggi_particelle.old01*
> -rwxr-xr-x 2 stefano user 106014 Apr 11 19:20 leggi_particelle.old03*
> ---------T 2 root root 0 May 24 17:16 parametri.cpp
> drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/
> /storage/arc1/data/stefano/leggi_particelle:
> total 144
> drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./
> drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../
> lrwxrwxrwx 2 stefano user 53 Feb 22 19:40 binnaggio.cpp ->
> /tamino/stefano/codice/leggi_particelle/binnaggio.cpp
> -rwxr-xr-x 2 stefano user 350 Feb 25 17:24 Etheta.plt*
> lrwxrwxrwx 2 stefano user 72 Mar 22 2012
> leggi_binario_ALaDyn_fortran.cpp ->
> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.cpp
> lrwxrwxrwx 2 stefano user 55 Mar 22 2012 leggi_campi.cpp ->
> /tamino/stefano/codice/leggi_particelle/leggi_campi.cpp
> lrwxrwxrwx 2 stefano user 60 Apr 11 19:20 leggi_particelle.cpp ->
> /tamino/stefano/codice/leggi_particelle/leggi_particelle.cpp
> -rwxr-xr-x 2 stefano user 97536 Mar 25 12:46 leggi_particelle.old02*
> -rwxr-xr-x 2 stefano user 923 May 12 12:42 plot_den.plt*
> lrwxrwxrwx 2 stefano user 54 Mar 22 2012 swap_tools.cpp ->
> /tamino/stefano/codice/leggi_particelle/swap_tools.cpp
> drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/
> -rwxr-xr-x 2 stefano user 309 Feb 25 17:24 xpx.plt*
> /storage/arc2/data/stefano/leggi_particelle:
> total 144
> drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./
> drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../
> lrwxrwxrwx 2 stefano user 53 Feb 22 19:40 binnaggio.cpp ->
> /tamino/stefano/codice/leggi_particelle/binnaggio.cpp
> -rwxr-xr-x 2 stefano user 350 Feb 25 17:24 Etheta.plt*
> lrwxrwxrwx 2 stefano user 72 Mar 22 2012
> leggi_binario_ALaDyn_fortran.cpp ->
> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.cpp
> lrwxrwxrwx 2 stefano user 55 Mar 22 2012 leggi_campi.cpp ->
> /tamino/stefano/codice/leggi_particelle/leggi_campi.cpp
> lrwxrwxrwx 2 stefano user 60 Apr 11 19:20 leggi_particelle.cpp ->
> /tamino/stefano/codice/leggi_particelle/leggi_particelle.cpp
> -rwxr-xr-x 2 stefano user 97536 Mar 25 12:46 leggi_particelle.old02*
> -rwxr-xr-x 2 stefano user 923 May 12 12:42 plot_den.plt*
> lrwxrwxrwx 2 stefano user 54 Mar 22 2012 swap_tools.cpp ->
> /tamino/stefano/codice/leggi_particelle/swap_tools.cpp
> drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/
> -rwxr-xr-x 2 stefano user 309 Feb 25 17:24 xpx.plt*
> /storageOsmino/1/data/stefano/leggi_particelle:
> total 24
> drwxr-xr-x 3 stefano user 4096 May 29 05:00 ./
> drwxr-xr-x 14 stefano user 4096 May 28 04:55 ../
> ---------T 2 root root 0 May 29 05:00 leggi_particelle
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/
> /storageOsmino/2/data/stefano/leggi_particelle:
> total 24
> drwxr-xr-x 3 stefano user 4096 May 29 05:00 ./
> drwxr-xr-x 14 stefano user 4096 May 28 04:55 ../
> ---------T 2 root root 0 May 29 05:00 leggi_particelle
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/
> /storageOsmino/4/data/stefano/leggi_particelle:
> total 16
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./
> drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/
> /storageOsmino/5/data/stefano/leggi_particelle:
> total 16
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./
> drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../
> drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/
> =======================================
> =======================================
> and these are the attributes of the folders in every brick
> # file: storage/1/data/stefano/leggi_particelle
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x00000001000000000000000033333332
> # file: storage/2/data/stefano/leggi_particelle
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x00000001000000000000000033333332
> # file: storage/5/data/stefano/leggi_particelle
> trusted.afr.data-client-2=0x000000000000000000000000
> trusted.afr.data-client-3=0x000000000000000000000000
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x00000001000000003333333366666665
> # file: storage/6/data/stefano/leggi_particelle
> trusted.afr.data-client-2=0x000000000000000000000000
> trusted.afr.data-client-3=0x000000000000000000000000
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x00000001000000003333333366666665
> # file: storage/arc1/data/stefano/leggi_particelle
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x0000000100000000ccccccccffffffff
> # file: storage/arc2/data/stefano/leggi_particelle
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x0000000100000000ccccccccffffffff
> # file: storageOsmino/1/data/stefano/leggi_particelle
> trusted.afr.data-client-6=0x000000000000000000000000
> trusted.afr.data-client-7=0x000000000000000000000000
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x00000001000000006666666699999998
> # file: storageOsmino/2/data/stefano/leggi_particelle
> trusted.afr.data-client-6=0x000000000000000000000000
> trusted.afr.data-client-7=0x000000000000000000000000
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x00000001000000006666666699999998
> # file: storageOsmino/4/data/stefano/leggi_particelle
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x000000010000000099999999cccccccb
> # file: storageOsmino/5/data/stefano/leggi_particelle
> trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105
> trusted.glusterfs.dht=0x000000010000000099999999cccccccb
> ===========================================
> Something is not consistent here and I can see it by myself...
> I hope to be forgiven for all these details and spamming in the
> mailing list. I'm just arrived and I'm already filling it up... Sorry.
> But also thanks a lot for your help!
> Stefano
>
I don't see anything incorrect here. It's very weird. Have you modified
any file manually ?
Can you look at .glusterfs/b6/2a/b62a16f0-bdb9-4e3f-8563-ccfb278c2105 on
each brick ? is it a valid symbolic link pointing to
data/stefano/leggi_particelle ?
Check also .glusterfs/88/3c/883c343b-9366-478d-a660-843da8f6b87c.
Also make sure if the file 'leggi_particelle' is present in any other brick.
Isn't there anything different in the logs of first and second bricks ?
I don't see what can be happening...
Xavi
More information about the Gluster-users
mailing list