[Gluster-users] Glusterfs rsync: mkstemp "" > failed: Input/output error (5)

Leung, Alex (398C) alex.leung at jpl.nasa.gov
Thu Nov 3 22:12:18 UTC 2016 

[root at pdsraid13 ~]# gluster volume info

Volume Name: pdsclust
Type: Disperse
Volume ID: 02629f52-cfe1-4542-8581-21d25e254d39
Status: Started
Number of Bricks: 1 x (4 + 2) = 6
Transport-type: tcp
Bricks:
Brick1: pdsraid4-gb:/data/gfs
Brick2: pdsraid8-gb:/data/gfs
Brick3: pdsraid10-gb:/data/gfs
Brick4: pdsraid12-gb:/data/gfs
Brick5: pdsraid14-gb:/data/gfs
Brick6: pdsraid13-gb:/data/gfs
Options Reconfigured:
performance.readdir-ahead: on
[root at pdsraid13 ~]#


gluster> volume status
Status of volume: pdsclust
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick pdsraid4-gb:/data/gfs                 49152     0          Y       13473
Brick pdsraid8-gb:/data/gfs                 49152     0          Y       25425
Brick pdsraid10-gb:/data/gfs                49152     0          Y       2818
Brick pdsraid12-gb:/data/gfs                49152     0          Y       17399
Brick pdsraid14-gb:/data/gfs                49152     0          Y       30226
Brick pdsraid13-gb:/data/gfs                49152     0          Y       21097
NFS Server on localhost                     2049      0          Y       21104
Self-heal Daemon on localhost               N/A       N/A        Y       21110
NFS Server on pdsraid10-gb                  2049      0          Y       2808
Self-heal Daemon on pdsraid10-gb            N/A       N/A        Y       2813
NFS Server on pdsraid12-gb                  2049      0          Y       22741
Self-heal Daemon on pdsraid12-gb            N/A       N/A        Y       22746
NFS Server on pdsraid8-gb                   2049      0          Y       30464
Self-heal Daemon on pdsraid8-gb             N/A       N/A        Y       30469
NFS Server on pdsraid4-gb                   2049      0          Y       18835
Self-heal Daemon on pdsraid4-gb             N/A       N/A        Y       18840
NFS Server on pdsraid14-gb                  2049      0          Y       3203
Self-heal Daemon on pdsraid14-gb            N/A       N/A        Y       3211

Task Status of Volume pdsclust
------------------------------------------------------------------------------
There are no active volume tasks
gluster>

From: "Leung, Alex (398C)" <alex.leung at jpl.nasa.gov>
Date: Thursday, November 3, 2016 at 3:07 PM
To: Nithya Balachandran <nbalacha at redhat.com>
Cc: gluster-users <gluster-users at gluster.org>, "Leung, Alex (398C)" <alex.leung at jpl.nasa.gov>
Subject: Glusterfs rsync: mkstemp "" > failed: Input/output error (5)

Here is the  /var/log/glusterfs/export-gfs.log

> Does anyone has any idea to troubleshoot the following problem?
>
>
>
> Alex
>
>
>

Can you please provide the gluster client logs (in /var/log/glusterfs) and
the gluster volume info?

Regards,
Nithya

>
>
> [root@*pdsimg-6* alex]# rsync -av pdsraid1:/export/pdsdata1/mro/
> safed/rsds/09000_09999_ops-120210/crism/ops_crism_vnir_09000_09999/ .
>
>
>
> root at pdsraid1's password:
>
> receiving incremental file list
>
> ./
>
> 4A_03_0000B42000_01.DAT
>
> 4A_03_0000B42000_01.DAT.hdr
>
> 4A_03_0000B42100_01.DAT
>
> 4A_03_0000B42100_01.DAT.hdr
>
> 4A_03_0000B42200_01.DAT
>
> 4A_03_0000B42200_01.DAT.hdr
>
> rsync: rename "/export/gfs/alex/.4A_03_0000B42200_01.DAT.hdr.wxXFyY" ->
> "4A_03_0000B42200_01.DAT.hdr": Input/output error (5)
>
> 4A_03_0000B42300_01.DAT
>
> 4A_03_0000B42300_01.DAT.hdr
>
> 4A_03_0000B42400_01.DAT
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61900_01.DAT.hdr":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42400_01.DAT.hdr.APw96Y"
> failed: Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61A00_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61A00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61B00_01.DAT":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42500_01.DAT.JIEZjw" failed:
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61B00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61C00_01.DAT":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42500_01.DAT.hdr.YynNY3"
> failed: Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61C00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61D00_01.DAT":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42600_01.DAT.BVuNVB" failed:
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61D00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61E00_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61E00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42600_01.DAT.hdr.Ub0zT9"
> failed: Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61F00_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B61F00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42800_01.DAT.LRlMRH" failed:
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62000_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62000_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62200_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62200_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62300_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62300_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62400_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62400_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62500_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62500_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62600_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62600_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62700_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62700_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62900_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62900_01.DAT.hdr":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62A00_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62A00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42800_01.DAT.hdr.SLIPgh"
> failed: Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62B00_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62B00_01.DAT.hdr":
> Input/output error (5)
>
> rsync: mkstemp "/export/gfs/alex/.4A_03_0000B42900_01.DAT.JZja8Q" failed:
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62C00_01.DAT":
> Input/output error (5)
>
> rsync: recv_generator: failed to stat
> "/export/gfs/alex/4A_03_0000B62C00_01.DAT.hdr":
> Input/output error (5)
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161103/ddb72040/attachment.html>

More information about the Gluster-users mailing list