[Gluster-users] nfs test dd cp tar

Ted Miller tmiller at hcjb.org
Fri Jul 26 16:30:08 UTC 2013


On 7/24/2013 4:10 PM, Heiko L. wrote:
> Hallo
>
>
> I tried glusterfs-3.3.2 dd cp tar.
> - dd: ok
> - cp: ok
> - tar: fail (filesize=0)
>
>
> Whats going wrong?
I am fairly new to digging this deep into Linux file systems, but I will put 
my understanding into writing.  I believe dd fails because glusterfs is 
neither a file nor a block device.  dd basically interacts with files.  It 
can also interact with disks and partitions because, under the /dev/ 
directory structure, entire file systems can be read and written to as though 
they were one big file.

Gluster is different.  It has no file system of its own--it borrows the file 
system from whatever directory you tell it to use for its brick.  When there 
are redundant copies of a brick, those bricks are not all arranged the same, 
if you dig down far enough.  In fact, if I understand things correctly, on 
one computer the files may be occupying an entire 500GB hard drive formatted 
with XFS.  On another computer, that same brick may occupy a /gluster 
subdirectory on a 2TB drive which contains the entire root file system on an 
ext4 drive.  The way those files are stored on those two drives will be 
different in many, many ways, but gluster is fine with that. However, that 
makes it impossible to use dd to duplicate a "drive" like you can with other 
file systems.

If you want to back up the file system, it will have to be by handling it as 
a bunch of files, not as a block device.  If you want to restore the files to 
glusterfs again, you will need to make sure that your backup includes ALL the 
extended attributes.  I don't know exactly what commands it takes to do that, 
but I do know that without the extended attributes glusterfs won't be able to 
figure out if those files are the same as ones already on the system.

One other approach: if you are using identical partitions on all bricks to 
hold the file system (ext4 or XFS) that glusterfs is using, you can do a dd 
backup of the ext4 or XFS file system, and all the glusterfs information will 
be there.  If you do everything right, you can restore that backup back to a 
hard drive, mount the hard drive in the brick's location, and glusterfs will 
see it (assuming glusterfs configuration has not changed) and glusterfs will 
welcome the new brick back into the volume group.

As I said, I am new to this, so I hope someone will correct any mistakes in 
my explanation.
Ted Miller

>
> regards Heiko
>
>
>
> details
>
> - test10 cp
> root at z1-gluster:/mnt/gv3/test# sleep 10;date; cp -p /etc/profile .
> Wed Jul 24 19:05:14 UTC 2013
> root at z1-gluster:/mnt/gv3/test#   sleep 10;date; cp -p /etc/profile test1
> Wed Jul 24 19:05:24 UTC 2013
> root at z1-gluster:/mnt/gv3/test#   sleep 10;date; cp -p test1 test2
> Wed Jul 24 19:05:35 UTC 2013
> root at z1-gluster:/mnt/gv3/test# ls -l
> total 4
> -rw-r--r--   1 root     sys         1570 Jul 19 19:39 profile
> -rw-r--r--   1 root     sys         1570 Jul 19 19:39 test1
> -rw-r--r--   1 root     sys         1570 Jul 19 19:39 test2
> drwxr-xr-x  14 root     sys           14 Jul 20 11:05 var
> root at z1-gluster:/mnt/gv3/test#
>
>    -> ok
>
> -----------------------------------------------
>
> - test11 dd
>
> testfile=testfile
> testdir=.
> sz=1
> time dd if=/dev/urandom bs=1024k count=$sz of=$testdir/${testfile}_${sz} 2>&1 | grep -v rec
>
> root at z1-gluster:/mnt/gv3/test# time dd if=/dev/urandom bs=1024k count=$sz of=$testdir/${testfile}_${sz} 2>&1 | grep -v rec
>
> real    0m1.083s
> user    0m0.002s
> sys     0m0.276s
> root at z1-gluster:/mnt/gv3/test# ls -l ${testfile}*
> -rw-r--r--   1 root     root     1048576 Jul 24 19:23 testfile_1
> -rw-r--r--   1 root     root     10485760 Jul 24 19:24 testfile_10
>
>   -> ok
>
> -----------------------------------------------
> - test12 tar
>
> root at z1-gluster:/mnt/gv3/test# src="profile test1 test2"
> root at z1-gluster:/mnt/gv3/test# dst=test
>
> profile
> tar: profile: Cannot open: I/O error
> test1
> tar: test1: Cannot open: I/O error
> test2
> tar: test2: Cannot open: I/O error
> tar: Exiting with failure status due to previous errors
>
> root at z1-gluster:/mnt/gv3/test# ls -l $src $dst
> -rw-r--r--   1 root     sys         1570 Jul 19 19:39 profile
> -rw-r--r--   1 root     sys         1570 Jul 19 19:39 test1
> -rw-r--r--   1 root     sys         1570 Jul 19 19:39 test2
>
> test:
> total 0
> -rw-------   1 root     root           0 Jul 24 19:55 profile
> -rw-------   1 root     root           0 Jul 24 19:55 test1
> -rw-------   1 root     root           0 Jul 24 19:55 test2
>
>   -> fail
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
>

-- 
"He is no fool who gives what he cannot keep, to gain what he cannot lose." - - Jim Elliot
For more information about Jim Elliot and his unusual life, see http://www.christianliteratureandliving.com/march2003/carolyn.html.

Ted Miller
Design Engineer
HCJB Global Technology Center
2830 South 17th St
Elkhart, IN  46517
574--970-4272 my desk
574--970-4252 receptionist





More information about the Gluster-users mailing list