[Gluster-devel] Dynamic disabling of eager-locking based on number of fds

[Anand Avati]

On Mon, Feb 11, 2013 at 7:02 PM, Pranith Kumar K <pkarampu at redhat.com>wrote:

> hi,
>       Problem:
>
> When there are multiple fds writing to same file with eager-lock enabled,
> the fd which acquires the eager-lock waits for
> post-op-delay secs before doing the unlock. Because of this all other fds
> opened on the file face extra delay when
> performing writes. Eager-locking, post-op-delay need to be disabled when
> there are multiple fds opened on the file.
>
> Here is the profile info output for the case above:
> Execute the following command on the mount point.
> for n in `seq 1 50` ; do eval "/home/pranithk/workspace/**gerrit-repo/append2log.py
> ./ben.log 10000 0.001 &" ; done ; wait
>
>  %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls
>   Fop
>  ---------   -----------   -----------   -----------   ------------
>  ----
>       0.00       0.00 us       0.00 us       0.00 us             50
> RELEASE
>       0.00       0.00 us       0.00 us       0.00 us             60
>  RELEASEDIR
>       0.00      55.00 us      55.00 us      55.00 us              1
>  GETXATTR
>       0.00      31.50 us      27.00 us      36.00 us              2
>  STATFS
>       0.00      41.00 us      29.00 us      53.00 us              2
> ENTRYLK
>       0.00     198.00 us     198.00 us     198.00 us              1
>  CREATE
>       0.00     124.00 us     108.00 us     140.00 us              2
> READDIR
>       0.00      27.04 us      17.00 us      95.00 us             49
>  OPEN
>       0.00      74.89 us      13.00 us     206.00 us             47
>  STAT
>       0.01      87.02 us      11.00 us     391.00 us             50
> FLUSH
>       0.01     102.43 us      20.00 us     268.00 us             60
> OPENDIR
>       0.02     344.27 us      22.00 us     940.00 us             44
> WRITE
>       0.02     228.80 us      52.00 us     345.00 us             82
>  FXATTROP
>       0.03     199.89 us      19.00 us     404.00 us            120
>  READDIRP
>       0.05      91.41 us      23.00 us     832.00 us            421
>  LOOKUP
>      99.86  632698.45 us      17.00 us 1999724.00 us            126
>  FINODELK
>
> Observe that most of the delay is in FINODELK fop.
>
>       Possible Solution:
>       With the patch: http://review.gluster.org/4468 we started
> maintaining open-fd count in the inode. We need to implement xdata based
> xattr retrieval in write-fop and get open-fd-count in write fop. Remember
> the open-fd-count received from the write-fops and maintain it in
> afr-fd-ctx. If the open-fd count is >1 post-op-delay is immediately
> disabled for that write fop. All write-fops take into consideration this
> count to determine whether to enable eager-lock, post-op-delay for that
> write fop.
>
> Let me know if you foresee any issues with this approach.
>
> https://bugzilla.redhat.com/**show_bug.cgi?id=910217<https://bugzilla.redhat.com/show_bug.cgi?id=910217>is tracking this issue.
>


Ideally you would want open-fd count to be retrieved in all fops, and only
when an eager lock has been acquired. Any fop callback's xattr_rsp
inspection should potentially wake up the sleeping post-op-delay in that
inode (and disable further eager locking temporarily).

Avati
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20130211/aeef027d/attachment-0001.html>