[Gluster-users] Getting glusterfs to expand volume size to brick size
Artem Russakovskii
archon810 at gmail.com
Mon Apr 16 23:47:01 UTC 2018
Hi Nithya,
I'm on Gluster 4.0.1.
I don't think the bricks were smaller before - if they were, maybe 20GB
because Linode's minimum is 20GB, then I extended them to 25GB, resized
with resize2fs as instructed, and rebooted many times over since. Yet,
gluster refuses to see the full disk size.
Here's the status detail output:
gluster volume status dev_apkmirror_data detail
Status of volume: dev_apkmirror_data
------------------------------------------------------------
------------------
Brick : Brick pylon:/mnt/pylon_block1/dev_apkmirror_data
TCP Port : 49152
RDMA Port : 0
Online : Y
Pid : 1263
File System : ext4
Device : /dev/sdd
Mount Options : rw,relatime,data=ordered
Inode Size : 256
Disk Space Free : 23.0GB
Total Disk Space : 24.5GB
Inode Count : 1638400
Free Inodes : 1625429
------------------------------------------------------------
------------------
Brick : Brick pylon:/mnt/pylon_block2/dev_apkmirror_data
TCP Port : 49153
RDMA Port : 0
Online : Y
Pid : 1288
File System : ext4
Device : /dev/sdc
Mount Options : rw,relatime,data=ordered
Inode Size : 256
Disk Space Free : 24.0GB
Total Disk Space : 25.5GB
Inode Count : 1703936
Free Inodes : 1690965
------------------------------------------------------------
------------------
Brick : Brick pylon:/mnt/pylon_block3/dev_apkmirror_data
TCP Port : 49154
RDMA Port : 0
Online : Y
Pid : 1313
File System : ext4
Device : /dev/sde
Mount Options : rw,relatime,data=ordered
Inode Size : 256
Disk Space Free : 23.0GB
Total Disk Space : 24.5GB
Inode Count : 1638400
Free Inodes : 1625433
What's interesting here is that the gluster volume size is exactly 1/3 of
the total (8357M * 3 = 25071M). Yet, each block device is separate, and the
total storage available is 25071M on each brick.
The fstab is as follows:
/dev/disk/by-id/scsi-0Linode_Volume_pylon_block1 /mnt/pylon_block1 ext4
defaults 0 2
/dev/disk/by-id/scsi-0Linode_Volume_pylon_block2 /mnt/pylon_block2 ext4
defaults 0 2
/dev/disk/by-id/scsi-0Linode_Volume_pylon_block3 /mnt/pylon_block3 ext4
defaults 0 2
localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data1 glusterfs
defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0
localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data2 glusterfs
defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0
localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data3 glusterfs
defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0
localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data_ganesha nfs4
defaults,_netdev,bg,intr,soft,timeo=5,retrans=5,actimeo=10,retry=5 0 0
The latter entry is for an nfs ganesha test, in case it matters (which,
btw, fails miserably with all kinds of stability issues about broken pipes).
Note: this is a test server, so all 3 bricks are attached and mounted on
the same server.
Sincerely,
Artem
--
Founder, Android Police <http://www.androidpolice.com>, APK Mirror
<http://www.apkmirror.com/>, Illogical Robot LLC
beerpla.net | +ArtemRussakovskii
<https://plus.google.com/+ArtemRussakovskii> | @ArtemR
<http://twitter.com/ArtemR>
On Sun, Apr 15, 2018 at 10:56 PM, Nithya Balachandran <nbalacha at redhat.com>
wrote:
> What version of Gluster are you running? Were the bricks smaller earlier?
>
> Regards,
> Nithya
>
> On 15 April 2018 at 00:09, Artem Russakovskii <archon810 at gmail.com> wrote:
>
>> Hi,
>>
>> I have a 3-brick replicate volume, but for some reason I can't get it to
>> expand to the size of the bricks. The bricks are 25GB, but even after
>> multiple gluster restarts and remounts, the volume is only about 8GB.
>>
>> I believed I could always extend the bricks (we're using Linode block
>> storage, which allows extending block devices after they're created), and
>> gluster would see the newly available space and extend to use it.
>>
>> Multiple Google searches, and I'm still nowhere. Any ideas?
>>
>> df | ack "block|data"
>> Filesystem 1M-blocks
>> Used Available Use% Mounted on
>> /dev/sdd 25071M
>> 1491M 22284M 7% /mnt/pylon_block1
>> /dev/sdc 26079M
>> 1491M 23241M 7% /mnt/pylon_block2
>> /dev/sde 25071M
>> 1491M 22315M 7% /mnt/pylon_block3
>> localhost:/dev_apkmirror_data 8357M
>> 581M 7428M 8% /mnt/dev_apkmirror_data1
>> localhost:/dev_apkmirror_data 8357M
>> 581M 7428M 8% /mnt/dev_apkmirror_data2
>> localhost:/dev_apkmirror_data 8357M
>> 581M 7428M 8% /mnt/dev_apkmirror_data3
>>
>>
>>
>> gluster volume info
>>
>> Volume Name: dev_apkmirror_data
>> Type: Replicate
>> Volume ID: cd5621ee-7fab-401b-b720-08863717ed56
>> Status: Started
>> Snapshot Count: 0
>> Number of Bricks: 1 x 3 = 3
>> Transport-type: tcp
>> Bricks:
>> Brick1: pylon:/mnt/pylon_block1/dev_apkmirror_data
>> Brick2: pylon:/mnt/pylon_block2/dev_apkmirror_data
>> Brick3: pylon:/mnt/pylon_block3/dev_apkmirror_data
>> Options Reconfigured:
>> disperse.eager-lock: off
>> cluster.lookup-unhashed: auto
>> cluster.read-hash-mode: 0
>> performance.strict-o-direct: on
>> cluster.shd-max-threads: 12
>> performance.nl-cache-timeout: 600
>> performance.nl-cache: on
>> cluster.quorum-count: 1
>> cluster.quorum-type: fixed
>> network.ping-timeout: 5
>> network.remote-dio: enable
>> performance.rda-cache-limit: 256MB
>> performance.parallel-readdir: on
>> network.inode-lru-limit: 500000
>> performance.md-cache-timeout: 600
>> performance.cache-invalidation: on
>> performance.stat-prefetch: on
>> features.cache-invalidation-timeout: 600
>> features.cache-invalidation: on
>> performance.io-thread-count: 32
>> server.event-threads: 4
>> client.event-threads: 4
>> performance.read-ahead: off
>> cluster.lookup-optimize: on
>> performance.client-io-threads: on
>> performance.cache-size: 1GB
>> transport.address-family: inet
>> performance.readdir-ahead: on
>> nfs.disable: on
>> cluster.readdir-optimize: on
>>
>>
>> Thank you.
>>
>> Sincerely,
>> Artem
>>
>> --
>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
>> <http://www.apkmirror.com/>, Illogical Robot LLC
>> beerpla.net | +ArtemRussakovskii
>> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR
>> <http://twitter.com/ArtemR>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180416/b108df76/attachment.html>
More information about the Gluster-users
mailing list