[Gluster-users] Getting glusterfs to expand volume size to brick size
Nithya Balachandran
nbalacha at redhat.com
Tue Apr 17 04:29:15 UTC 2018
Ok, it looks like the same problem.
@Amar, this fix is supposed to be in 4.0.1. Is it possible to regenerate
the volfiles to fix this?
Regards,
Nithya
On 17 April 2018 at 09:57, Artem Russakovskii <archon810 at gmail.com> wrote:
> pylon:/var/lib/glusterd/vols/dev_apkmirror_data # ack shared-brick-count
> dev_apkmirror_data.pylon.mnt-pylon_block3-dev_apkmirror_data.vol
> 3: option shared-brick-count 3
>
> dev_apkmirror_data.pylon.mnt-pylon_block2-dev_apkmirror_data.vol
> 3: option shared-brick-count 3
>
> dev_apkmirror_data.pylon.mnt-pylon_block1-dev_apkmirror_data.vol
> 3: option shared-brick-count 3
>
>
> Sincerely,
> Artem
>
> --
> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
> <http://www.apkmirror.com/>, Illogical Robot LLC
> beerpla.net | +ArtemRussakovskii
> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR
> <http://twitter.com/ArtemR>
>
> On Mon, Apr 16, 2018 at 9:22 PM, Nithya Balachandran <nbalacha at redhat.com>
> wrote:
>
>> Hi Artem,
>>
>> Was the volume size correct before the bricks were expanded?
>>
>> This sounds like [1] but that should have been fixed in 4.0.0. Can you
>> let us know the values of shared-brick-count in the files in
>> /var/lib/glusterd/vols/dev_apkmirror_data/ ?
>>
>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1541880
>>
>> On 17 April 2018 at 05:17, Artem Russakovskii <archon810 at gmail.com>
>> wrote:
>>
>>> Hi Nithya,
>>>
>>> I'm on Gluster 4.0.1.
>>>
>>> I don't think the bricks were smaller before - if they were, maybe 20GB
>>> because Linode's minimum is 20GB, then I extended them to 25GB, resized
>>> with resize2fs as instructed, and rebooted many times over since. Yet,
>>> gluster refuses to see the full disk size.
>>>
>>> Here's the status detail output:
>>>
>>> gluster volume status dev_apkmirror_data detail
>>> Status of volume: dev_apkmirror_data
>>> ------------------------------------------------------------
>>> ------------------
>>> Brick : Brick pylon:/mnt/pylon_block1/dev_apkmirror_data
>>> TCP Port : 49152
>>> RDMA Port : 0
>>> Online : Y
>>> Pid : 1263
>>> File System : ext4
>>> Device : /dev/sdd
>>> Mount Options : rw,relatime,data=ordered
>>> Inode Size : 256
>>> Disk Space Free : 23.0GB
>>> Total Disk Space : 24.5GB
>>> Inode Count : 1638400
>>> Free Inodes : 1625429
>>> ------------------------------------------------------------
>>> ------------------
>>> Brick : Brick pylon:/mnt/pylon_block2/dev_apkmirror_data
>>> TCP Port : 49153
>>> RDMA Port : 0
>>> Online : Y
>>> Pid : 1288
>>> File System : ext4
>>> Device : /dev/sdc
>>> Mount Options : rw,relatime,data=ordered
>>> Inode Size : 256
>>> Disk Space Free : 24.0GB
>>> Total Disk Space : 25.5GB
>>> Inode Count : 1703936
>>> Free Inodes : 1690965
>>> ------------------------------------------------------------
>>> ------------------
>>> Brick : Brick pylon:/mnt/pylon_block3/dev_apkmirror_data
>>> TCP Port : 49154
>>> RDMA Port : 0
>>> Online : Y
>>> Pid : 1313
>>> File System : ext4
>>> Device : /dev/sde
>>> Mount Options : rw,relatime,data=ordered
>>> Inode Size : 256
>>> Disk Space Free : 23.0GB
>>> Total Disk Space : 24.5GB
>>> Inode Count : 1638400
>>> Free Inodes : 1625433
>>>
>>>
>>>
>>> What's interesting here is that the gluster volume size is exactly 1/3
>>> of the total (8357M * 3 = 25071M). Yet, each block device is separate, and
>>> the total storage available is 25071M on each brick.
>>>
>>> The fstab is as follows:
>>> /dev/disk/by-id/scsi-0Linode_Volume_pylon_block1 /mnt/pylon_block1 ext4
>>> defaults 0 2
>>> /dev/disk/by-id/scsi-0Linode_Volume_pylon_block2 /mnt/pylon_block2 ext4
>>> defaults 0 2
>>> /dev/disk/by-id/scsi-0Linode_Volume_pylon_block3 /mnt/pylon_block3 ext4
>>> defaults 0 2
>>>
>>> localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data1 glusterfs
>>> defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0
>>> localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data2 glusterfs
>>> defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0
>>> localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data3 glusterfs
>>> defaults,_netdev,fopen-keep-cache,direct-io-mode=enable 0 0
>>> localhost:/dev_apkmirror_data /mnt/dev_apkmirror_data_ganesha nfs4
>>> defaults,_netdev,bg,intr,soft,timeo=5,retrans=5,actimeo=10,retry=5 0 0
>>>
>>> The latter entry is for an nfs ganesha test, in case it matters (which,
>>> btw, fails miserably with all kinds of stability issues about broken pipes).
>>>
>>> Note: this is a test server, so all 3 bricks are attached and mounted on
>>> the same server.
>>>
>>>
>>> Sincerely,
>>> Artem
>>>
>>> --
>>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
>>> <http://www.apkmirror.com/>, Illogical Robot LLC
>>> beerpla.net | +ArtemRussakovskii
>>> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR
>>> <http://twitter.com/ArtemR>
>>>
>>> On Sun, Apr 15, 2018 at 10:56 PM, Nithya Balachandran <
>>> nbalacha at redhat.com> wrote:
>>>
>>>> What version of Gluster are you running? Were the bricks smaller
>>>> earlier?
>>>>
>>>> Regards,
>>>> Nithya
>>>>
>>>> On 15 April 2018 at 00:09, Artem Russakovskii <archon810 at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have a 3-brick replicate volume, but for some reason I can't get it
>>>>> to expand to the size of the bricks. The bricks are 25GB, but even after
>>>>> multiple gluster restarts and remounts, the volume is only about 8GB.
>>>>>
>>>>> I believed I could always extend the bricks (we're using Linode block
>>>>> storage, which allows extending block devices after they're created), and
>>>>> gluster would see the newly available space and extend to use it.
>>>>>
>>>>> Multiple Google searches, and I'm still nowhere. Any ideas?
>>>>>
>>>>> df | ack "block|data"
>>>>> Filesystem
>>>>> 1M-blocks Used Available Use% Mounted on
>>>>> /dev/sdd
>>>>> 25071M 1491M 22284M 7% /mnt/pylon_block1
>>>>> /dev/sdc
>>>>> 26079M 1491M 23241M 7% /mnt/pylon_block2
>>>>> /dev/sde
>>>>> 25071M 1491M 22315M 7% /mnt/pylon_block3
>>>>> localhost:/dev_apkmirror_data
>>>>> 8357M 581M 7428M 8% /mnt/dev_apkmirror_data1
>>>>> localhost:/dev_apkmirror_data
>>>>> 8357M 581M 7428M 8% /mnt/dev_apkmirror_data2
>>>>> localhost:/dev_apkmirror_data
>>>>> 8357M 581M 7428M 8% /mnt/dev_apkmirror_data3
>>>>>
>>>>>
>>>>>
>>>>> gluster volume info
>>>>>
>>>>> Volume Name: dev_apkmirror_data
>>>>> Type: Replicate
>>>>> Volume ID: cd5621ee-7fab-401b-b720-08863717ed56
>>>>> Status: Started
>>>>> Snapshot Count: 0
>>>>> Number of Bricks: 1 x 3 = 3
>>>>> Transport-type: tcp
>>>>> Bricks:
>>>>> Brick1: pylon:/mnt/pylon_block1/dev_apkmirror_data
>>>>> Brick2: pylon:/mnt/pylon_block2/dev_apkmirror_data
>>>>> Brick3: pylon:/mnt/pylon_block3/dev_apkmirror_data
>>>>> Options Reconfigured:
>>>>> disperse.eager-lock: off
>>>>> cluster.lookup-unhashed: auto
>>>>> cluster.read-hash-mode: 0
>>>>> performance.strict-o-direct: on
>>>>> cluster.shd-max-threads: 12
>>>>> performance.nl-cache-timeout: 600
>>>>> performance.nl-cache: on
>>>>> cluster.quorum-count: 1
>>>>> cluster.quorum-type: fixed
>>>>> network.ping-timeout: 5
>>>>> network.remote-dio: enable
>>>>> performance.rda-cache-limit: 256MB
>>>>> performance.parallel-readdir: on
>>>>> network.inode-lru-limit: 500000
>>>>> performance.md-cache-timeout: 600
>>>>> performance.cache-invalidation: on
>>>>> performance.stat-prefetch: on
>>>>> features.cache-invalidation-timeout: 600
>>>>> features.cache-invalidation: on
>>>>> performance.io-thread-count: 32
>>>>> server.event-threads: 4
>>>>> client.event-threads: 4
>>>>> performance.read-ahead: off
>>>>> cluster.lookup-optimize: on
>>>>> performance.client-io-threads: on
>>>>> performance.cache-size: 1GB
>>>>> transport.address-family: inet
>>>>> performance.readdir-ahead: on
>>>>> nfs.disable: on
>>>>> cluster.readdir-optimize: on
>>>>>
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Sincerely,
>>>>> Artem
>>>>>
>>>>> --
>>>>> Founder, Android Police <http://www.androidpolice.com>, APK Mirror
>>>>> <http://www.apkmirror.com/>, Illogical Robot LLC
>>>>> beerpla.net | +ArtemRussakovskii
>>>>> <https://plus.google.com/+ArtemRussakovskii> | @ArtemR
>>>>> <http://twitter.com/ArtemR>
>>>>>
>>>>> _______________________________________________
>>>>> Gluster-users mailing list
>>>>> Gluster-users at gluster.org
>>>>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>>>>
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180417/dff19f38/attachment.html>
More information about the Gluster-users
mailing list