[Gluster-users] Crash of brick during data transfer

[Heiko Schröter]

Hello,

during a data transfer job the underlying raid6 crashed. We could restore the XFS file system to some extend and now do have a lot of files in "lost+found".
These have to be restored i.e moved manually from "lost+found" to their correct locations.

What would be the correct method to move these files "gluster aware" ?
What is the correct procedure to heal a system after such a crash ?
The docs do only handle a crash where the underlying file system could be completely restored.

Thanks and Regards
Heiko


rd34 ~ # gluster volume info

Volume Name: our_data
Type: Distribute
Status: Started
Number of Bricks: 12
Transport-type: tcp
Bricks:
Brick1: rd29:/data
Brick2: rd34:/data
Brick3: rd25:/data <-- crashed
Brick4: rd28:/data
Brick5: rd24:/data
Brick6: rd26:/data
Brick7: rd27:/data
Brick8: rd21:/data
Brick9: rd20:/data
Brick10: rd22:/data
Brick11: rd23:/data
Brick12: rd30:/data
Options Reconfigured:
cluster.min-free-disk: 500GB
network.ping-timeout: 24
nfs.disable: off
nfs.export-dir: /data
nfs.export-volumes: on

rd34 ~ # gluster peer status
Number of Peers: 11

Hostname: rd24                                                                                           
Uuid: db74d69f-44af-4d20-bb26-104de8d272fc
State: Peer in Cluster (Connected)

Hostname: rd21
Uuid: 230d4fb6-6092-4c14-93ee-e5a45c99d3ac
State: Peer in Cluster (Connected)

Hostname: rd28
Uuid: c29ac356-9caf-4d84-bd08-28d37aa71099
State: Peer in Cluster (Connected)

Hostname: rd26
Uuid: 7792b011-d748-4630-be44-44ff5fbf67e5
State: Peer in Cluster (Connected)

Hostname: rd29
Uuid: c0ad7e6f-492f-408f-ba41-3dda8cb15ec9
State: Peer in Cluster (Connected)

Hostname: rd27
Uuid: cd34e500-d174-42ad-90d1-063b159c6f73
State: Peer in Cluster (Connected)

Hostname: rd25
Uuid: e8d95290-1339-4d00-b0e9-bf810d9bdc26
State: Peer in Cluster (Disconnected)

Hostname: rd20
Uuid: 7bcefbf4-5c7d-4ab3-978a-d0aa0ac48cb2
State: Peer in Cluster (Connected)

Hostname: rd22
Uuid: 68df3b44-738f-4b95-8ead-3972dc2f89b0
State: Peer in Cluster (Connected)

Hostname: rd23
Uuid: 117b4b4b-336b-4935-aaf0-75a4e3ae16bf
State: Peer in Cluster (Connected)

Hostname: rd30
Uuid: d001b1ee-4e5e-4c65-a4f1-38fe37461851
State: Peer in Cluster (Connected)