[heketi-devel] Create Volume Build Failed: No space
Andrea Cucciarre'
acucciarre at cloudian.com
Wed May 19 08:56:50 UTC 2021
Hello,
I'm trying to create a large volume but it fails shortly showing "No
space", though there is plenty of space.
The debug logs doesn't say much:
[heketi] INFO 2021/05/19 08:11:17 Allocating brick set #122
[heketi] INFO 2021/05/19 08:11:17 Allocating brick set #123
[heketi] INFO 2021/05/19 08:11:17 Allocating brick set #124
[heketi] INFO 2021/05/19 08:11:17 Allocating brick set #125
[heketi] INFO 2021/05/19 08:11:17 Allocating brick set #126
[heketi] ERROR 2021/05/19 08:11:17
heketi/apps/glusterfs/operations_manage.go:220:glusterfs.AsyncHttpOperation:
Create Volume Build Failed: No space
I'm using the following heketi-cli command:
heketi-cli --user admin --secret xZbphMQHQ volume create --name=cache
--size=95545 --gluster-volume-options="performance.write-behind
off","performance.open-behind off" --replica=3
If I change the volume size to 94800, it works.
It's a 3 cluster nodes, each node have 9 disks 10Tb each and I have set
max_bricks_per_volume to 600 and brick_max_size_gb 10240.
My topology info is the end of the mail.
volume size / (max_bricks_per_volume/ 3 nodes) = 95545/200 = 477.725
brick size
device size = 11175, so 11175/477.725 = 23 bricks per device
9 devices * 23 bricks = 207
So the free space on the devices allow the creation of 207 bricks
(477.725 size each) on every node, I need 200 bricks per node to create
the volume, then why it fails for "No space" ?
Cluster Id: 1def3eb774703edd5dad5687d86ec1f5
File: true
Block: true
Volumes:
Name: heketidbstorage
Size: 2
Id: cc07e3205650842e31324cab359bca43
Cluster Id: 1def3eb774703edd5dad5687d86ec1f5
Mount: 10.50.50.51:heketidbstorage
Mount Options: backup-volfile-servers=10.50.50.52,10.50.50.53
Durability Type: replicate
Replica: 3
Snapshot: Disabled
Bricks:
Id: 7a4004f395cce5fe89f4a0fa5af61549
Path:
/var/lib/heketi/mounts/vg_a81b43597c607f9a3ba54e2552b821e8/brick_7a4004f395cce5fe89f4a0fa5af61549/brick
Size (GiB): 2
Node: e18b193a75bc688ea86659bd1aa0a96c
Device: a81b43597c607f9a3ba54e2552b821e8
Id: 84879c1d761d54788285a8728fe7100f
Path:
/var/lib/heketi/mounts/vg_d25b7a27273da7d25d1e1d8ca60ec922/brick_84879c1d761d54788285a8728fe7100f/brick
Size (GiB): 2
Node: fc13ff5b8a48893e31b1c3da2815ac16
Device: d25b7a27273da7d25d1e1d8ca60ec922
Id: 9d7dd2cf5ae65b87111f4481f32fb7d7
Path:
/var/lib/heketi/mounts/vg_bbd7550f3fdeccf291397337375b0cd2/brick_9d7dd2cf5ae65b87111f4481f32fb7d7/brick
Size (GiB): 2
Node: 7ad303638bf94f974ebae8aaf11ac127
Device: bbd7550f3fdeccf291397337375b0cd2
Nodes:
Node Id: 7ad303638bf94f974ebae8aaf11ac127
State: online
Cluster Id: 1def3eb774703edd5dad5687d86ec1f5
Zone: 1
Management Hostnames: appliance1
Storage Hostnames: 10.50.50.51
Devices:
Id:00599faa322c0ef3cd5de1f29c6fb38e State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdg
Bricks:
Id:35c678931dd899477289034ca9c4be87 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdc
Bricks:
Id:4b21b903a642a65557fee7f38745a393 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdh
Bricks:
Id:718ef6478d1209909e38567a81ee3dc5 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdj
Bricks:
Id:7569be6559a050ae87c18096a94f7ee1 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdk
Bricks:
Id:88e42232a848d4dce05fbc9e6ab5fe24 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdf
Bricks:
Id:a7129a57851de3c8f74f059486ef801d State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sde
Bricks:
Id:bbd7550f3fdeccf291397337375b0cd2 State:online
Size (GiB):11175 Used (GiB):2 Free (GiB):11173
Known Paths: /dev/sdd
Bricks:
Id:9d7dd2cf5ae65b87111f4481f32fb7d7 Size (GiB):2 Path:
/var/lib/heketi/mounts/vg_bbd7550f3fdeccf291397337375b0cd2/brick_9d7dd2cf5ae65b87111f4481f32fb7d7/brick
Id:c5d7fe344f184120594983b305596b6a State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdi
Bricks:
Node Id: e18b193a75bc688ea86659bd1aa0a96c
State: online
Cluster Id: 1def3eb774703edd5dad5687d86ec1f5
Zone: 1
Management Hostnames: appliance2
Storage Hostnames: 10.50.50.52
Devices:
Id:038fb10f2f56c79155b415c2065cc4be State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdf
Bricks:
Id:17987efce61b40397d7809074aacb669 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdk
Bricks:
Id:1ead1ac8442017e28bfd7a2f62bf944c State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdh
Bricks:
Id:271ff7788b1204a15075cb1ce750876a State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sde
Bricks:
Id:63bdcd6738e2f9a0ff9631d6e4608ab4 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdj
Bricks:
Id:7168580b23549f88031328de2d4d82bd State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdc
Bricks:
Id:9a12e60bb5c6793f125eafd81a339a6e State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdi
Bricks:
Id:a81b43597c607f9a3ba54e2552b821e8 State:online
Size (GiB):11175 Used (GiB):2 Free (GiB):11173
Known Paths: /dev/sdd
Bricks:
Id:7a4004f395cce5fe89f4a0fa5af61549 Size (GiB):2 Path:
/var/lib/heketi/mounts/vg_a81b43597c607f9a3ba54e2552b821e8/brick_7a4004f395cce5fe89f4a0fa5af61549/brick
Id:b025f8d107533230de67ed2ce9f75aab State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdg
Bricks:
Node Id: fc13ff5b8a48893e31b1c3da2815ac16
State: online
Cluster Id: 1def3eb774703edd5dad5687d86ec1f5
Zone: 1
Management Hostnames: appliance3
Storage Hostnames: 10.50.50.53
Devices:
Id:29ccf7627df71dd8e8781841c76c3e38 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sde
Bricks:
Id:3c8bd93e84138f4075e4697c5be2a6f1 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdj
Bricks:
Id:42dd1c3486974f3905f41b12890285ae State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdk
Bricks:
Id:8dd5e8558a5669a85e24342661218bb7 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdg
Bricks:
Id:972a57af87a24a3bad819d3b78a7fa35 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdc
Bricks:
Id:bc679d5c2db68f320904bc486da64bac State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdd
Bricks:
Id:c8f540c5837982973624821ed0dd1d61 State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdh
Bricks:
Id:d25b7a27273da7d25d1e1d8ca60ec922 State:online
Size (GiB):11175 Used (GiB):2 Free (GiB):11173
Known Paths: /dev/sdi
Bricks:
Id:84879c1d761d54788285a8728fe7100f Size (GiB):2 Path:
/var/lib/heketi/mounts/vg_d25b7a27273da7d25d1e1d8ca60ec922/brick_84879c1d761d54788285a8728fe7100f/brick
Id:dd67cbd491766a0ce81d7e751e19c79d State:online
Size (GiB):11175 Used (GiB):0 Free (GiB):11175
Known Paths: /dev/sdf
Bricks:
--
Regards
Andrea Cucciarre'
More information about the heketi-devel
mailing list