[Gluster-devel] RFC for quota limit exceeded logging

[Anuradha Talur]

Hi all,

This mail is regarding quota logging for the following conditions :

1. Soft-limit alerts - "Usage crossed soft-limit" and "usage above soft-limit"
2. Quota exceeded warning

Issue with the soft-limit alerts logging --
Currently, the soft-limit alerts are logged in the brick logs when a write
is made and soft-limit is exceeded. In a scenario where there are a large 
number of bricks, it is difficult for the system admin to poll all the 
bricks for a soft-limit alert.

Issue with the quota exceeded logging --
When the quota exceeds, op_errno is set to EDQUOT and eventually
"Quota exceeded" logs are being logged in the client logs (nfs.log, fuse mnt log,..)
as of now (Similar to other op_errors logging). 
An issue has been raised stating that the quota hard-limit and soft-limit 
exceeding logs should be logged in same file.

So a need for a unified logging has been proposed. We came up with the 
solution that all the soft-limit alert logs and the quota exceeded logs 
should be logged in quotad.log file.
Resulting in the limit exceeded logs being logged in all the nodes in the
cluster. The sys admin can poll any one node to get the status of soft-limit
and hard-limit.

Approach proposed to achieve this solution --
During the event of soft-limit/hard-limit exceeding, glusterd is informed.
glusterd of the current node should interact with the glusterd of the other 
nodes in the cluster indicating the issue.
All the glusterd's talk to their respective quotad to log the limit exceed
in their logs.

The following bugs are related to the points made here --
https://bugzilla.redhat.com/show_bug.cgi?id=1020816
https://bugzilla.redhat.com/show_bug.cgi?id=1019302


Is this approach recommended?

Thanks,
Anuradha.