[Bugs] [Bug 1146470] New: The memories are exhausted quickly when handle the message which has multi fragments in a single record

bugzilla at redhat.com bugzilla at redhat.com
Thu Sep 25 10:03:18 UTC 2014


https://bugzilla.redhat.com/show_bug.cgi?id=1146470

            Bug ID: 1146470
           Summary: The memories are exhausted quickly when handle the
                    message which has multi fragments in a single record
           Product: GlusterFS
           Version: 3.4.5
         Component: rpc
          Keywords: Patch
          Severity: low
          Assignee: gluster-bugs at redhat.com
          Reporter: ndevos at redhat.com
                CC: bugs at gluster.org
        Depends On: 1139598
            Blocks: 1125245 (glusterfs-3.4.6)



+++ This bug was initially created as a clone of Bug #1139598 +++
+++ This bug was initially created as a clone of Bug #1136221 +++
+++                                                           +++
+++ Use this bug to backport the change to release-3.4        +++

Description of problem:
    We construct some rpc messages and send it to the IP and port which
glusterfsd listens, the memory usage goes up quickly until exhausted

Version-Release number of selected component (if applicable):
    3.3.0, 3.4.1, 3.5.0


Steps to Reproduce:
1. Start glusterfs services, and get the IP and port that one glusterfsd
process listens

2. Run the attachement python script, which connects the IP and port and send
four bytes 00 00 00 00 to the glusterfsd process

3. Watch the memory usage of the glusterfsd process. It will grow up quickly

Actual results:
   Memory of the glusterfsd process grows up quickly till exhausted

Expected results:
   Glusterfsd just ignores the messages


Additional info:
   The bug seems in __socket_proto_state_machine, which goes into an infinite
loop to malloc memories when handle the special message. The special message is
"multi fragments in a single record", and some values are not reset when handle
next fragment.

   We tested below fix and it seems work:
          if (!RPC_LASTFRAG (in->fraghdr)) {

+             in->pending_vector = in->vector;
+             in->pending_vector->iov_base =  &in->fraghdr;
+             in->pending_vector->iov_len  = sizeof (in->fraghdr);
              in->record_state = SP_STATE_READING_FRAGHDR;
              break;
           }

--- Additional comment from jiangkai on 2014-09-04 06:35:44 EDT ---

More issues than imaging to handle the "multi fragments in a single record"
message. The proposal is to refuse it:


 if (!RPC_LASTFRAG (in->fraghdr)) {
       gf_log (this->name, GF_LOG_ERROR, "multiple fragments per record not
supported now");
       ret = -1;
       goto out;
 }

--- Additional comment from jiangkai on 2014-09-05 04:45:08 EDT ---

It happens after 3.4;  
3.3.1 reports error messages.

It seems imported by the change id Icd9f256bb2fd8c6266a7abefdff16936b4f8922d to
support SSL

--- Additional comment from Anand Avati on 2014-09-09 04:30:12 EDT ---

REVIEW: http://review.gluster.org/8662 (socket: Fixed parsing RPC records
containing multi fragments) posted (#1) for review on master by Gu Feng
(flygoast at 126.com)

--- Additional comment from Anand Avati on 2014-09-09 12:03:17 CEST ---

REVIEW: http://review.gluster.org/8662 (socket: Fixed parsing RPC records
containing multi fragments) posted (#2) for review on master by Gu Feng
(flygoast at 126.com)

--- Additional comment from Anand Avati on 2014-09-13 18:27:24 CEST ---

REVIEW: http://review.gluster.org/8662 (socket: Fixed parsing RPC records
containing multi fragments) posted (#3) for review on master by Gu Feng
(flygoast at 126.com)

--- Additional comment from Anand Avati on 2014-09-19 14:37:09 CEST ---

COMMIT: http://review.gluster.org/8662 committed in master by Raghavendra G
(rgowdapp at redhat.com) 
------
commit fb6702b7f8ba19333b7ba4af543d908e3f5e1923
Author: Gu Feng <flygoast at 126.com>
Date:   Tue Sep 9 18:00:22 2014 +0800

    socket: Fixed parsing RPC records containing multi fragments

    In __socket_proto_state_machine(), when parsing RPC records containing
    multi fragments, just change the state of parsing process, had not
    processed the memory to coalesce the multi fragments.

    Change-Id: I5583e578603bd7290814a5d26885b31759c73115
    BUG: 1139598
    Signed-off-by: Gu Feng <flygoast at 126.com>
    Reviewed-on: http://review.gluster.org/8662
    Tested-by: Gluster Build System <jenkins at build.gluster.com>
    Reviewed-by: Niels de Vos <ndevos at redhat.com>
    Reviewed-by: Raghavendra G <rgowdapp at redhat.com>
    Tested-by: Raghavendra G <rgowdapp at redhat.com>


Referenced Bugs:

https://bugzilla.redhat.com/show_bug.cgi?id=1125245
[Bug 1125245] GlusterFS 3.4.6 Tracker
https://bugzilla.redhat.com/show_bug.cgi?id=1139598
[Bug 1139598] The memories are exhausted quickly when handle the message
which has multi fragments in a single record
-- 
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=923GE4oE6m&a=cc_unsubscribe


More information about the Bugs mailing list