[Bugs] [Bug 1170075] [RFE] : BitRot detection in glusterfs

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1170075



--- Comment #25 from Anand Avati <aavati at redhat.com> ---
COMMIT: http://review.gluster.org/9706 committed in master by Vijay Bellur
(vbellur at redhat.com) 
------
commit 87a9d23627586a5d5cd8a502a08ecbdb6f0f7bb2
Author: Venky Shankar <vshankar at redhat.com>
Date:   Tue Feb 3 19:03:30 2015 +0530

    libglusterfs/rot-buffs: rotational buffers

    This patch introduces rotational buffers aiming at the classic
    multiple producer and multiple consumer problem. A fixed set
    of buffer list is allocated during initialization, where each
    list consist of a list of buffers. Each buffer is an iovec
    pointing to a memory region of fixed allocation size. Multiple
    producers write data to these buffers. A buffer list starts with
    a single buffer (iovec) and allocates more when required (although
    this can be preallocatd in multiples of k).

    rot-buffs allow multiple producers to write data parallely with
    a bit of extra cost of taking locks. Therefore, it's much suited
    for large writes. Multiple producers are allowed to write in the
    buffer parallely by "reserving" write space for selected number
    of bytes and returning pointer to the start of the reserved area.
    The write size is selected by the producer before it starts the
    write (which is often known). Therefore, the write itself need not
    be serialized -- just the space reservation needs to be done safely.

    The other part is when a consumer kicks in to consume what has
    been produced. At this point, a buffer list switch is performed.
    The "current" buffer list pointer is safely pointed to the next
    available buffer list. New writes are now directed to the just
    switched buffer list (the old buffer list is now considered out
    of rotation). Note that the old buffer still may have producers
    in progress (pending writes), so the consumer has to wait till
    the writers are drained. Currently this is the slow path for
    producers (write completion) and needs to be improved.

    Currently, there is special handling for cases where the number
    of consumers match (or exceed) the number of producers, which
    could result in writer starvation. In this scenario, when a
    consumers requests a buffer list for consumption, a check is
    performed for writer starvation and consumption is denied
    until at least another buffer list is ready of the producer
    for writes, i.e., one (or more) consumer(s) completed, thereby
    putting the buffer list back in rotation.

    [
       NOTE:
       I've not performance tested this producer-consumer model
       yet. It's being used in changelog for event notification.
       The list of buffers (iovecs) are directly passed to RPC
       layer.
    ]

    Change-Id: I88d235522b05ab82509aba861374a2312bff57f2
    BUG: 1170075
    Signed-off-by: Venky Shankar <vshankar at redhat.com>
    Reviewed-on: http://review.gluster.org/9706
    Tested-by: Vijay Bellur <vbellur at redhat.com>

-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=p8y2M5o1gC&a=cc_unsubscribe