[heketi-devel] volume provisioning fails with "No space"
Polasek, Robert
RPolasek at ifdsgroup.com
Thu Oct 25 20:59:41 UTC 2018
Hi everybody,
I am testing heketi autoprovisioning of gluster volumes and I am hitting a problem where heketi reports "No space" in the log, even there should be a space.
Would somebody be able to point me what I need to look into or where my problem is? Here are the details of my problem:
I am using a curl to simulate my failing request from kubernetes. As a side note, our kubernetes is old, 1.4 and therefore I cannot
robert at workstation:~/src/ifds/helm/tapremium$ curl -v -X POST -H "Authorization: bearer ****" -H "Content-Type: application/json" POST http://localhost:8080/volumes -d@/tmp/volume.json<mailto:-d@/tmp/volume.json>
=================================================================================================================
This is a content of /tmp/volume.json file
{
"size": 10,
"name": "",
"durability": {
"type": "replicate",
"replicate": {
"replica": 3
},
"disperse": {}
},
"gid": 2000,
"snapshot": {
"enable": false,
"factor": 0
}
}
This is my topology
=================================================================================================================
Cluster Id: 7396c8b03593f9bfa1288cdebd9412ac
Volumes:
Name: vol_4708719dddaacda301e3b71763cb73fa
Size: 10
Id: 4708719dddaacda301e3b71763cb73fa
Cluster Id: 7396c8b03593f9bfa1288cdebd9412ac
Mount: 192.168.48.9:vol_4708719dddaacda301e3b71763cb73fa
Mount Options: backup-volfile-servers=
Durability Type: none
Snapshot: Disabled
Bricks:
Id: 99b33f7f23347c1691e4a93c7220af04
Path: /var/lib/heketi/mounts/vg_05df77bacca68b941f8d3d20d64a1aa2/brick_99b33f7f23347c1691e4a93c7220af04/brick
Size (GiB): 5
Node: 31e1ac3014029ff48398a989c10952b6
Device: 05df77bacca68b941f8d3d20d64a1aa2
Id: acebc8961ffb304fa2b97c201c7af8da
Path: /var/lib/heketi/mounts/vg_05df77bacca68b941f8d3d20d64a1aa2/brick_acebc8961ffb304fa2b97c201c7af8da/brick
Size (GiB): 5
Node: 31e1ac3014029ff48398a989c10952b6
Device: 05df77bacca68b941f8d3d20d64a1aa2
Nodes:
Node Id: 31e1ac3014029ff48398a989c10952b6
State: online
Cluster Id: 7396c8b03593f9bfa1288cdebd9412ac
Zone: 1
Management Hostname: 192.168.48.9
Storage Hostname: 192.168.48.9
Devices:
Id:05df77bacca68b941f8d3d20d64a1aa2 Name:/dev/gluster_vg/heketi_volumeState:online Size (GiB):99 Used (GiB):10 Free (GiB):89
Bricks:
Id:99b33f7f23347c1691e4a93c7220af04 Size (GiB):5 Path: /var/lib/heketi/mounts/vg_05df77bacca68b941f8d3d20d64a1aa2/brick_99b33f7f23347c1691e4a93c7220af04/brick
Id:acebc8961ffb304fa2b97c201c7af8da Size (GiB):5 Path: /var/lib/heketi/mounts/vg_05df77bacca68b941f8d3d20d64a1aa2/brick_acebc8961ffb304fa2b97c201c7af8da/brick
Id:6c0325d31e1b84a4f3aa088ba352e2ec Name:/dev/gluster_vg/heketi_volume2State:online Size (GiB):24 Used (GiB):0 Free (GiB):24
Bricks:
Id:bac78b5e4d558d387218d12fdfb7bfa0 Name:/dev/gluster_vg/heketi_volume1State:online Size (GiB):24 Used (GiB):0 Free (GiB):24
Bricks:
=================================================================================================================
and this is the log output
[negroni] Started POST /volumes
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry.go:101: [e8cb84a8b977a00d77e137a05a5b2530] Replica 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry.go:580: Using the following clusters: [7396c8b03593f9bfa1288cdebd9412ac]
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:43: brick_size = 10485760
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:44: sets = 1
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:45: num_bricks = 3
[heketi] INFO 2018/10/25 14:08:28 Allocating brick set #0
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 0 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:411: expected space needed for amount=10485760 snapFactor=1 : 10539008
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:368: device 6c0325d31e1b84a4f3aa088ba352e2ec[26079232] > required size [10539008] ?
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 1 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:455: Error detected. Cleaning up volume e8cb84a8b977a00d77e137a05a5b2530: Len(0)
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:56: No space, re-trying with smaller brick size
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:43: brick_size = 5242880
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:44: sets = 2
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:45: num_bricks = 6
[heketi] INFO 2018/10/25 14:08:28 Allocating brick set #0
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 0 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:411: expected space needed for amount=5242880 snapFactor=1 : 5271552
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:368: device bac78b5e4d558d387218d12fdfb7bfa0[26079232] > required size [5271552] ?
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 1 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:455: Error detected. Cleaning up volume e8cb84a8b977a00d77e137a05a5b2530: Len(0)
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:56: No space, re-trying with smaller brick size
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:43: brick_size = 2621440
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:44: sets = 4
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:45: num_bricks = 12
[heketi] INFO 2018/10/25 14:08:28 Allocating brick set #0
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 0 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:411: expected space needed for amount=2621440 snapFactor=1 : 2637824
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:368: device 05df77bacca68b941f8d3d20d64a1aa2[94179328] > required size [2637824] ?
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 1 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:455: Error detected. Cleaning up volume e8cb84a8b977a00d77e137a05a5b2530: Len(0)
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:56: No space, re-trying with smaller brick size
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:43: brick_size = 1310720
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:44: sets = 8
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:45: num_bricks = 24
[heketi] INFO 2018/10/25 14:08:28 Allocating brick set #0
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 0 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:411: expected space needed for amount=1310720 snapFactor=1 : 1318912
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/device_entry.go:368: device 05df77bacca68b941f8d3d20d64a1aa2[94179328] > required size [1318912] ?
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/brick_allocate.go:251: 1 / 3
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:455: Error detected. Cleaning up volume e8cb84a8b977a00d77e137a05a5b2530: Len(0)
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:56: No space, re-trying with smaller brick size
[heketi] ERROR 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:37: Minimum brick size limit reached. Out of space.
[heketi] DEBUG 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/volume_entry.go:445: Cluster 7396c8b03593f9bfa1288cdebd9412ac can not accommodate volume (Minimum brick size limit reached. Out of space.), trying next cluster
[heketi] ERROR 2018/10/25 14:08:28 /src/github.com/heketi/heketi/apps/glusterfs/operations_manage.go:89: Create Volume Build Failed: No space
[negroni] Completed 500 Internal Server Error in 3.397009ms
This message is marked Public
Please consider the environment before printing this email and any attachments.
This e-mail and any attachments are intended only for the individual or company to which it is addressed and may contain information which is privileged, confidential and prohibited from disclosure or unauthorized use under applicable law. If you are not the intended recipient of this e-mail, you are hereby notified that any use, dissemination, or copying of this e-mail or the information contained in this e-mail is strictly prohibited by the sender. If you have received this transmission in error, please return the material received to the sender and delete all copies from your system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20181025/79f9935e/attachment-0001.html>
More information about the heketi-devel
mailing list