[heketi-devel] Heketi DB corrupted after reboot

[Vishal Kanojwar]

Hi Team,

While there was an operation going on for the creation of PVC, a reboot of
the Heketi pod has corrupted the DB file.

I cant execute the dump on the DB or do any operation.
Heketi pod is in continuous crashloopback off mode.

Tried by copying the db file on a different container and tried recovering
but in vain.
The environment used

Kubernetes: v1.12.1
Heketi: 9

The error it throws while trying to dump the db is

[root at heketi-7d8d9d6d4b-jxspz heketi]# ./heketi  db dump  --dbfile
/var/lib/heketi/heketi.db  --jsonfile=heketi.json --debug


*[heketi] DEBUG 2019/05/02 12:28:47
heketi/apps/glusterfs/db_operations.go:36:glusterfs.dbDumpInternal.func1:
volume bucketfailed to check db: Unable to check the database: Could not
construct dump from DB: Could not construct dump from DB: Unable to access
list*

when tried with the debug mode the error thrown is

*pgid (274877906944) above high water mark (69)*

Can anyone please suggest how can we go ahead with the recovery of the db.
or there is an alternative where I can generate the JSON from the heketi.db
content.

Thanks,
Vishal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/heketi-devel/attachments/20190502/9012c1ca/attachment.html>