[Gluster-users] Slow write times to gluster disk

Pranith Kumar Karampuri pkarampu at redhat.com
Fri Jun 23 03:40:49 UTC 2017


On Fri, Jun 23, 2017 at 2:23 AM, Pat Haley <phaley at mit.edu> wrote:

>
> Hi,
>
> Today we experimented with some of the FUSE options that we found in the
> list.
>
> Changing these options had no effect:
>
> gluster volume set test-volume performance.cache-max-file-size 2MB
> gluster volume set test-volume performance.cache-refresh-timeout 4
> gluster volume set test-volume performance.cache-size 256MB
> gluster volume set test-volume performance.write-behind-window-size 4MB
> gluster volume set test-volume performance.write-behind-window-size 8MB
>
>
This is a good coincidence, I am meeting with write-behind
maintainer(+Raghavendra G) today for the same doubt. I think we will have
something by EOD IST. I will update you.

> Changing the following option from its default value made the speed slower
>
> gluster volume set test-volume performance.write-behind off (on by default)
>
> Changing the following options initially appeared to give a 10% increase
> in speed, but this vanished in subsequent tests (we think the apparent
> increase may have been to a lighter workload on the computer from other
> users)
>
> gluster volume set test-volume performance.stat-prefetch on
> gluster volume set test-volume client.event-threads 4
> gluster volume set test-volume server.event-threads 4
>
> Can anything be gleaned from these observations?  Are there other things
> we can try?
>
> Thanks
>
> Pat
>
>
>
> On 06/20/2017 12:06 PM, Pat Haley wrote:
>
>
> Hi Ben,
>
> Sorry this took so long, but we had a real-time forecasting exercise last
> week and I could only get to this now.
>
> Backend Hardware/OS:
>
>    - Much of the information on our back end system is included at the
>    top of  http://lists.gluster.org/pipermail/gluster-users/2017-
>    April/030529.html
>    - The specific model of the hard disks is SeaGate ENTERPRISE CAPACITY
>    V.4 6TB (ST6000NM0024). The rated speed is 6Gb/s.
>    - Note: there is one physical server that hosts both the NFS and the
>    GlusterFS areas
>
> Latest tests
>
> I have had time to run the tests for one of the dd tests you requested to
> the underlying XFS FS.  The median rate was 170 MB/s.  The dd results and
> iostat record are in
>
> http://mseas.mit.edu/download/phaley/GlusterUsers/TestXFS/
>
> I'll add tests for the other brick and to the NFS area later.
>
> Thanks
>
> Pat
>
>
> On 06/12/2017 06:06 PM, Ben Turner wrote:
>
> Ok you are correct, you have a pure distributed volume.  IE no replication overhead.  So normally for pure dist I use:
>
> throughput = slowest of disks / NIC * .6-.7
>
> In your case we have:
>
> 1200 * .6 = 720
>
> So you are seeing a little less throughput than I would expect in your configuration.  What I like to do here is:
>
> -First tell me more about your back end storage, will it sustain 1200 MB / sec?  What kind of HW?  How many disks?  What type and specs are the disks?  What kind of RAID are you using?
>
> -Second can you refresh me on your workload?  Are you doing reads / writes or both?  If both what mix?  Since we are using DD I assume you are working iwth large file sequential I/O, is this correct?
>
> -Run some DD tests on the back end XFS FS.  I normally have /xfs-mount/gluster-brick, if you have something similar just mkdir on the XFS -> /xfs-mount/my-test-dir.  Inside the test dir run:
>
> If you are focusing on a write workload run:
>
> # dd if=/dev/zero of=/xfs-mount/file bs=1024k count=10000 conv=fdatasync
>
> If you are focusing on a read workload run:
>
> # echo 3 > /proc/sys/vm/drop_caches
> # dd if=/gluster-mount/file of=/dev/null bs=1024k count=10000
>
> ** MAKE SURE TO DROP CACHE IN BETWEEN READS!! **
>
> Run this in a loop similar to how you did in:
> http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/dd_testvol_gluster.txt
>
> Run this on both servers one at a time and if you are running on a SAN then run again on both at the same time.  While this is running gather iostat for me:
>
> # iostat -c -m -x 1 > iostat-$(hostname).txt
>
> Lets see how the back end performs on both servers while capturing iostat, then see how the same workload / data looks on gluster.
>
> -Last thing, when you run your kernel NFS tests are you using the same filesystem / storage you are using for the gluster bricks?  I want to be sure we have an apples to apples comparison here.
>
> -b
>
>
>
> ----- Original Message -----
>
> From: "Pat Haley" <phaley at mit.edu> <phaley at mit.edu>
> To: "Ben Turner" <bturner at redhat.com> <bturner at redhat.com>
> Sent: Monday, June 12, 2017 5:18:07 PM
> Subject: Re: [Gluster-users] Slow write times to gluster disk
>
>
> Hi Ben,
>
> Here is the output:
>
> [root at mseas-data2 ~]# gluster volume info
>
> Volume Name: data-volume
> Type: Distribute
> Volume ID: c162161e-2a2d-4dac-b015-f31fd89ceb18
> Status: Started
> Number of Bricks: 2
> Transport-type: tcp
> Bricks:
> Brick1: mseas-data2:/mnt/brick1
> Brick2: mseas-data2:/mnt/brick2
> Options Reconfigured:
> nfs.exports-auth-enable: on
> diagnostics.brick-sys-log-level: WARNING
> performance.readdir-ahead: on
> nfs.disable: on
> nfs.export-volumes: off
>
>
> On 06/12/2017 05:01 PM, Ben Turner wrote:
>
> What is the output of gluster v info?  That will tell us more about your
> config.
>
> -b
>
> ----- Original Message -----
>
> From: "Pat Haley" <phaley at mit.edu> <phaley at mit.edu>
> To: "Ben Turner" <bturner at redhat.com> <bturner at redhat.com>
> Sent: Monday, June 12, 2017 4:54:00 PM
> Subject: Re: [Gluster-users] Slow write times to gluster disk
>
>
> Hi Ben,
>
> I guess I'm confused about what you mean by replication.  If I look at
> the underlying bricks I only ever have a single copy of any file.  It
> either resides on one brick or the other  (directories exist on both
> bricks but not files).  We are not using gluster for redundancy (or at
> least that wasn't our intent).   Is that what you meant by replication
> or is it something else?
>
> Thanks
>
> Pat
>
> On 06/12/2017 04:28 PM, Ben Turner wrote:
>
> ----- Original Message -----
>
> From: "Pat Haley" <phaley at mit.edu> <phaley at mit.edu>
> To: "Ben Turner" <bturner at redhat.com> <bturner at redhat.com>, "Pranith Kumar Karampuri"<pkarampu at redhat.com> <pkarampu at redhat.com>
> Cc: "Ravishankar N" <ravishankar at redhat.com> <ravishankar at redhat.com>, gluster-users at gluster.org,
> "Steve Postma" <SPostma at ztechnet.com> <SPostma at ztechnet.com>
> Sent: Monday, June 12, 2017 2:35:41 PM
> Subject: Re: [Gluster-users] Slow write times to gluster disk
>
>
> Hi Guys,
>
> I was wondering what our next steps should be to solve the slow write
> times.
>
> Recently I was debugging a large code and writing a lot of output at
> every time step.  When I tried writing to our gluster disks, it was
> taking over a day to do a single time step whereas if I had the same
> program (same hardware, network) write to our nfs disk the time per
> time-step was about 45 minutes. What we are shooting for here would be
> to have similar times to either gluster of nfs.
>
> I can see in your test:
> http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/dd_testvol_gluster.txt
>
> You averaged ~600 MB / sec(expected for replica 2 with 10G, {~1200 MB /
> sec} / #replicas{2} = 600).  Gluster does client side replication so with
> replica 2 you will only ever see 1/2 the speed of your slowest part of
> the
> stack(NW, disk, RAM, CPU).  This is usually NW or disk and 600 is
> normally
> a best case.  Now in your output I do see the instances where you went
> down to 200 MB / sec.  I can only explain this in three ways:
>
> 1.  You are not using conv=fdatasync and writes are actually going to
> page
> cache and then being flushed to disk.  During the fsync the memory is not
> yet available and the disks are busy flushing dirty pages.
> 2.  Your storage RAID group is shared across multiple LUNS(like in a SAN)
> and when write times are slow the RAID group is busy serviceing other
> LUNs.
> 3.  Gluster bug / config issue / some other unknown unknown.
>
> So I see 2 issues here:
>
> 1.  NFS does in 45 minutes what gluster can do in 24 hours.
> 2.  Sometimes your throughput drops dramatically.
>
> WRT #1 - have a look at my estimates above.  My formula for guestimating
> gluster perf is: throughput = NIC throughput or storage(whatever is
> slower) / # replicas * overhead(figure .7 or .8).  Also the larger the
> record size the better for glusterfs mounts, I normally like to be at
> LEAST 64k up to 1024k:
>
> # dd if=/dev/zero of=/gluster-mount/file bs=1024k count=10000
> conv=fdatasync
>
> WRT #2 - Again, I question your testing and your storage config.  Try
> using
> conv=fdatasync for your DDs, use a larger record size, and make sure that
> your back end storage is not causing your slowdowns.  Also remember that
> with replica 2 you will take ~50% hit on writes because the client uses
> 50% of its bandwidth to write to one replica and 50% to the other.
>
> -b
>
>
>
>
> Thanks
>
> Pat
>
>
> On 06/02/2017 01:07 AM, Ben Turner wrote:
>
> Are you sure using conv=sync is what you want?  I normally use
> conv=fdatasync, I'll look up the difference between the two and see if
> it
> affects your test.
>
>
> -b
>
> ----- Original Message -----
>
> From: "Pat Haley" <phaley at mit.edu> <phaley at mit.edu>
> To: "Pranith Kumar Karampuri" <pkarampu at redhat.com> <pkarampu at redhat.com>
> Cc: "Ravishankar N" <ravishankar at redhat.com> <ravishankar at redhat.com>,gluster-users at gluster.org,
> "Steve Postma" <SPostma at ztechnet.com> <SPostma at ztechnet.com>, "Ben
> Turner" <bturner at redhat.com> <bturner at redhat.com>
> Sent: Tuesday, May 30, 2017 9:40:34 PM
> Subject: Re: [Gluster-users] Slow write times to gluster disk
>
>
> Hi Pranith,
>
> The "dd" command was:
>
>         dd if=/dev/zero count=4096 bs=1048576 of=zeros.txt conv=sync
>
> There were 2 instances where dd reported 22 seconds. The output from
> the
> dd tests are in
> http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/dd_testvol_gluster.txt
>
> Pat
>
> On 05/30/2017 09:27 PM, Pranith Kumar Karampuri wrote:
>
> Pat,
>           What is the command you used? As per the following output,
>           it
> seems like at least one write operation took 16 seconds. Which is
> really bad.
>          96.39    1165.10 us      89.00 us*16487014.00 us*
>          393212
>          WRITE
>
>
> On Tue, May 30, 2017 at 10:36 PM, Pat Haley <phaley at mit.edu<mailto:phaley at mit.edu> <phaley at mit.edu>> wrote:
>
>
>        Hi Pranith,
>
>        I ran the same 'dd' test both in the gluster test volume and
>        in
>        the .glusterfs directory of each brick.  The median results
>        (12
>        dd
>        trials in each test) are similar to before
>
>          * gluster test volume: 586.5 MB/s
>          * bricks (in .glusterfs): 1.4 GB/s
>
>        The profile for the gluster test-volume is in
>
>        http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/profile_testvol_gluster.txt
>        <http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/profile_testvol_gluster.txt> <http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/profile_testvol_gluster.txt>
>
>        Thanks
>
>        Pat
>
>
>
>
>        On 05/30/2017 12:10 PM, Pranith Kumar Karampuri wrote:
>
>        Let's start with the same 'dd' test we were testing with to
>        see,
>        what the numbers are. Please provide profile numbers for the
>        same. From there on we will start tuning the volume to see
>        what
>        we can do.
>
>        On Tue, May 30, 2017 at 9:16 PM, Pat Haley <phaley at mit.edu
>        <mailto:phaley at mit.edu> <phaley at mit.edu>> wrote:
>
>
>            Hi Pranith,
>
>            Thanks for the tip.  We now have the gluster volume
>            mounted
>            under /home.  What tests do you recommend we run?
>
>            Thanks
>
>            Pat
>
>
>
>            On 05/17/2017 05:01 AM, Pranith Kumar Karampuri wrote:
>
>            On Tue, May 16, 2017 at 9:20 PM, Pat Haley
>            <phaley at mit.edu
>            <mailto:phaley at mit.edu> <phaley at mit.edu>> wrote:
>
>
>                Hi Pranith,
>
>                Sorry for the delay.  I never saw received your
>                reply
>                (but I did receive Ben Turner's follow-up to your
>                reply).  So we tried to create a gluster volume
>                under
>                /home using different variations of
>
>                gluster volume create test-volume
>                mseas-data2:/home/gbrick_test_1
>                mseas-data2:/home/gbrick_test_2 transport tcp
>
>                However we keep getting errors of the form
>
>                Wrong brick type: transport, use
>                <HOSTNAME>:<export-dir-abs-path>
>
>                Any thoughts on what we're doing wrong?
>
>
>            You should give transport tcp at the beginning I think.
>            Anyways, transport tcp is the default, so no need to
>            specify
>            so remove those two words from the CLI.
>
>
>                Also do you have a list of the test we should be
>                running
>                once we get this volume created?  Given the
>                time-zone
>                difference it might help if we can run a small
>                battery
>                of tests and post the results rather than
>                test-post-new
>                test-post... .
>
>
>            This is the first time I am doing performance analysis
>            on
>            users as far as I remember. In our team there are
>            separate
>            engineers who do these tests. Ben who replied earlier is
>            one
>            such engineer.
>
>            Ben,
>                Have any suggestions?
>
>
>                Thanks
>
>                Pat
>
>
>
>                On 05/11/2017 12:06 PM, Pranith Kumar Karampuri
>                wrote:
>
>                On Thu, May 11, 2017 at 9:32 PM, Pat Haley
>                <phaley at mit.edu <mailto:phaley at mit.edu> <phaley at mit.edu>> wrote:
>
>
>                    Hi Pranith,
>
>                    The /home partition is mounted as ext4
>                    /home ext4 defaults,usrquota,grpquota   1 2
>
>                    The brick partitions are mounted ax xfs
>                    /mnt/brick1 xfs defaults 0 0
>                    /mnt/brick2 xfs defaults 0 0
>
>                    Will this cause a problem with creating a
>                    volume
>                    under /home?
>
>
>                I don't think the bottleneck is disk. You can do
>                the
>                same tests you did on your new volume to confirm?
>
>
>                    Pat
>
>
>
>                    On 05/11/2017 11:32 AM, Pranith Kumar Karampuri
>                    wrote:
>
>                    On Thu, May 11, 2017 at 8:57 PM, Pat Haley
>                    <phaley at mit.edu <mailto:phaley at mit.edu> <phaley at mit.edu>>
>                    wrote:
>
>
>                        Hi Pranith,
>
>                        Unfortunately, we don't have similar
>                        hardware
>                        for a small scale test.  All we have is
>                        our
>                        production hardware.
>
>
>                    You said something about /home partition which
>                    has
>                    lesser disks, we can create plain distribute
>                    volume inside one of those directories. After
>                    we
>                    are done, we can remove the setup. What do you
>                    say?
>
>
>                        Pat
>
>
>
>
>                        On 05/11/2017 07:05 AM, Pranith Kumar
>                        Karampuri wrote:
>
>                        On Thu, May 11, 2017 at 2:48 AM, Pat
>                        Haley
>                        <phaley at mit.edu <mailto:phaley at mit.edu> <phaley at mit.edu>>
>                        wrote:
>
>
>                            Hi Pranith,
>
>                            Since we are mounting the partitions
>                            as
>                            the bricks, I tried the dd test
>                            writing
>                            to
>                            <brick-path>/.glusterfs/<file-to-be-removed-after-test>.
>                            The results without oflag=sync were
>                            1.6
>                            Gb/s (faster than gluster but not as
>                            fast
>                            as I was expecting given the 1.2 Gb/s
>                            to
>                            the no-gluster area w/ fewer disks).
>
>
>                        Okay, then 1.6Gb/s is what we need to
>                        target
>                        for, considering your volume is just
>                        distribute. Is there any way you can do
>                        tests
>                        on similar hardware but at a small scale?
>                        Just so we can run the workload to learn
>                        more
>                        about the bottlenecks in the system? We
>                        can
>                        probably try to get the speed to 1.2Gb/s
>                        on
>                        your /home partition you were telling me
>                        yesterday. Let me know if that is
>                        something
>                        you are okay to do.
>
>
>                            Pat
>
>
>
>                            On 05/10/2017 01:27 PM, Pranith Kumar
>                            Karampuri wrote:
>
>                            On Wed, May 10, 2017 at 10:15 PM,
>                            Pat
>                            Haley <phaley at mit.edu
>                            <mailto:phaley at mit.edu> <phaley at mit.edu>> wrote:
>
>
>                                Hi Pranith,
>
>                                Not entirely sure (this isn't my
>                                area of expertise). I'll run
>                                your
>                                answer by some other people who
>                                are
>                                more familiar with this.
>
>                                I am also uncertain about how to
>                                interpret the results when we
>                                also
>                                add the dd tests writing to the
>                                /home area (no gluster, still on
>                                the
>                                same machine)
>
>                                  * dd test without oflag=sync
>                                    (rough average of multiple
>                                    tests)
>                                      o gluster w/ fuse mount :
>                                      570
>                                      Mb/s
>                                      o gluster w/ nfs mount:
>                                      390
>                                      Mb/s
>                                      o nfs (no gluster):  1.2
>                                      Gb/s
>                                  * dd test with oflag=sync
>                                  (rough
>                                    average of multiple tests)
>                                      o gluster w/ fuse mount:
>                                      5
>                                      Mb/s
>                                      o gluster w/ nfs mount:
>                                      200
>                                      Mb/s
>                                      o nfs (no gluster): 20
>                                      Mb/s
>
>                                Given that the non-gluster area
>                                is
>                                a
>                                RAID-6 of 4 disks while each
>                                brick
>                                of the gluster area is a RAID-6
>                                of
>                                32 disks, I would naively expect
>                                the
>                                writes to the gluster area to be
>                                roughly 8x faster than to the
>                                non-gluster.
>
>
>                            I think a better test is to try and
>                            write to a file using nfs without
>                            any
>                            gluster to a location that is not
>                            inside
>                            the brick but someother location
>                            that
>                            is
>                            on same disk(s). If you are mounting
>                            the
>                            partition as the brick, then we can
>                            write to a file inside .glusterfs
>                            directory, something like
>                            <brick-path>/.glusterfs/<file-to-be-removed-after-test>.
>
>
>
>                                I still think we have a speed
>                                issue,
>                                I can't tell if fuse vs nfs is
>                                part
>                                of the problem.
>
>
>                            I got interested in the post because
>                            I
>                            read that fuse speed is lesser than
>                            nfs
>                            speed which is counter-intuitive to
>                            my
>                            understanding. So wanted
>                            clarifications.
>                            Now that I got my clarifications
>                            where
>                            fuse outperformed nfs without sync,
>                            we
>                            can resume testing as described
>                            above
>                            and try to find what it is. Based on
>                            your email-id I am guessing you are
>                            from
>                            Boston and I am from Bangalore so if
>                            you
>                            are okay with doing this debugging
>                            for
>                            multiple days because of timezones,
>                            I
>                            will be happy to help. Please be a
>                            bit
>                            patient with me, I am under a
>                            release
>                            crunch but I am very curious with
>                            the
>                            problem you posted.
>
>                                Was there anything useful in the
>                                profiles?
>
>
>                            Unfortunately profiles didn't help
>                            me
>                            much, I think we are collecting the
>                            profiles from an active volume, so
>                            it
>                            has a lot of information that is not
>                            pertaining to dd so it is difficult
>                            to
>                            find the contributions of dd. So I
>                            went
>                            through your post again and found
>                            something I didn't pay much
>                            attention
>                            to
>                            earlier i.e. oflag=sync, so did my
>                            own
>                            tests on my setup with FUSE so sent
>                            that
>                            reply.
>
>
>                                Pat
>
>
>
>                                On 05/10/2017 12:15 PM, Pranith
>                                Kumar Karampuri wrote:
>
>                                Okay good. At least this
>                                validates
>                                my doubts. Handling O_SYNC in
>                                gluster NFS and fuse is a bit
>                                different.
>                                When application opens a file
>                                with
>                                O_SYNC on fuse mount then each
>                                write syscall has to be written
>                                to
>                                disk as part of the syscall
>                                where
>                                as in case of NFS, there is no
>                                concept of open. NFS performs
>                                write
>                                though a handle saying it needs
>                                to
>                                be a synchronous write, so
>                                write()
>                                syscall is performed first then
>                                it
>                                performs fsync(). so an write
>                                on
>                                an
>                                fd with O_SYNC becomes
>                                write+fsync.
>                                I am suspecting that when
>                                multiple
>                                threads do this write+fsync()
>                                operation on the same file,
>                                multiple writes are batched
>                                together to be written do disk
>                                so
>                                the throughput on the disk is
>                                increasing is my guess.
>
>                                Does it answer your doubts?
>
>                                On Wed, May 10, 2017 at 9:35
>                                PM,
>                                Pat Haley <phaley at mit.edu
>                                <mailto:phaley at mit.edu> <phaley at mit.edu>> wrote:
>
>
>                                    Without the oflag=sync and
>                                    only
>                                    a single test of each, the
>                                    FUSE
>                                    is going faster than NFS:
>
>                                    FUSE:
>                                    mseas-data2(dri_nascar)% dd
>                                    if=/dev/zero count=4096
>                                    bs=1048576 of=zeros.txt
>                                    conv=sync
>                                    4096+0 records in
>                                    4096+0 records out
>                                    4294967296 bytes (4.3 GB)
>                                    copied, 7.46961 s, 575 MB/s
>
>
>                                    NFS
>                                    mseas-data2(HYCOM)% dd
>                                    if=/dev/zero count=4096
>                                    bs=1048576 of=zeros.txt
>                                    conv=sync
>                                    4096+0 records in
>                                    4096+0 records out
>                                    4294967296 bytes (4.3 GB)
>                                    copied, 11.4264 s, 376 MB/s
>
>
>
>                                    On 05/10/2017 11:53 AM,
>                                    Pranith
>                                    Kumar Karampuri wrote:
>
>                                    Could you let me know the
>                                    speed without oflag=sync
>                                    on
>                                    both the mounts? No need
>                                    to
>                                    collect profiles.
>
>                                    On Wed, May 10, 2017 at
>                                    9:17
>                                    PM, Pat Haley
>                                    <phaley at mit.edu
>                                    <mailto:phaley at mit.edu> <phaley at mit.edu>>
>                                    wrote:
>
>
>                                        Here is what I see
>                                        now:
>
>                                        [root at mseas-data2 ~]#
>                                        gluster volume info
>
>                                        Volume Name:
>                                        data-volume
>                                        Type: Distribute
>                                        Volume ID:
>                                        c162161e-2a2d-4dac-b015-f31fd89ceb18
>                                        Status: Started
>                                        Number of Bricks: 2
>                                        Transport-type: tcp
>                                        Bricks:
>                                        Brick1:
>                                        mseas-data2:/mnt/brick1
>                                        Brick2:
>                                        mseas-data2:/mnt/brick2
>                                        Options Reconfigured:
>                                        diagnostics.count-fop-hits:
>                                        on
>                                        diagnostics.latency-measurement:
>                                        on
>                                        nfs.exports-auth-enable:
>                                        on
>                                        diagnostics.brick-sys-log-level:
>                                        WARNING
>                                        performance.readdir-ahead:
>                                        on
>                                        nfs.disable: on
>                                        nfs.export-volumes:
>                                        off
>
>
>
>                                        On 05/10/2017 11:44
>                                        AM,
>                                        Pranith Kumar
>                                        Karampuri
>                                        wrote:
>
>                                        Is this the volume
>                                        info
>                                        you have?
>
>                                        >/[root at
>                                        >mseas-data2
>                                        <http://www.gluster.org/mailman/listinfo/gluster-users> <http://www.gluster.org/mailman/listinfo/gluster-users>
>                                        ~]# gluster volume
>                                        info
>                                        />//>/Volume Name:
>                                        data-volume />/Type:
>                                        Distribute />/Volume
>                                        ID:
>                                        c162161e-2a2d-4dac-b015-f31fd89ceb18
>                                        />/Status: Started
>                                        />/Number
>                                        of Bricks: 2
>                                        />/Transport-type:
>                                        tcp
>                                        />/Bricks: />/Brick1:
>                                        mseas-data2:/mnt/brick1
>                                        />/Brick2:
>                                        mseas-data2:/mnt/brick2
>                                        />/Options
>                                        Reconfigured:
>                                        />/performance.readdir-ahead:
>                                        on />/nfs.disable: on
>                                        />/nfs.export-volumes:
>                                        off
>                                        /
>                                        ​I copied this from
>                                        old
>                                        thread from 2016.
>                                        This
>                                        is
>                                        distribute volume.
>                                        Did
>                                        you change any of the
>                                        options in between?
>
>                                        --
>
>                                        -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                                        Pat Haley
>                                        Email:phaley at mit.edu
>                                        <mailto:phaley at mit.edu> <phaley at mit.edu>
>                                        Center for Ocean
>                                        Engineering
>                                        Phone:  (617) 253-6824
>                                        Dept. of Mechanical
>                                        Engineering
>                                        Fax:    (617) 253-8125
>                                        MIT, Room
>                                        5-213http://web.mit.edu/phaley/www/
>                                        77 Massachusetts
>                                        Avenue
>                                        Cambridge, MA
>                                        02139-4301
>
>                                    --
>                                    Pranith
>
>                                    --
>
>                                    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                                    Pat Haley
>                                    Email:phaley at mit.edu
>                                    <mailto:phaley at mit.edu> <phaley at mit.edu>
>                                    Center for Ocean
>                                    Engineering
>                                    Phone:  (617) 253-6824
>                                    Dept. of Mechanical
>                                    Engineering
>                                    Fax:    (617) 253-8125
>                                    MIT, Room
>                                    5-213http://web.mit.edu/phaley/www/
>                                    77 Massachusetts Avenue
>                                    Cambridge, MA  02139-4301
>
>                                --
>                                Pranith
>
>                                --
>
>                                -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                                Pat Haley
>                                Email:phaley at mit.edu
>                                <mailto:phaley at mit.edu> <phaley at mit.edu>
>                                Center for Ocean Engineering
>                                Phone:
>                                (617) 253-6824
>                                Dept. of Mechanical Engineering
>                                Fax:
>                                (617) 253-8125
>                                MIT, Room
>                                5-213http://web.mit.edu/phaley/www/
>                                77 Massachusetts Avenue
>                                Cambridge, MA  02139-4301
>
>                            --
>                            Pranith
>
>                            --
>
>                            -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                            Pat Haley
>                            Email:phaley at mit.edu
>                            <mailto:phaley at mit.edu> <phaley at mit.edu>
>                            Center for Ocean Engineering
>                            Phone:
>                            (617) 253-6824
>                            Dept. of Mechanical Engineering
>                            Fax:
>                            (617) 253-8125
>                            MIT, Room
>                            5-213http://web.mit.edu/phaley/www/
>                            77 Massachusetts Avenue
>                            Cambridge, MA  02139-4301
>
>                        --
>                        Pranith
>
>                        --
>
>                        -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                        Pat Haley
>                        Email:phaley at mit.edu
>                        <mailto:phaley at mit.edu> <phaley at mit.edu>
>                        Center for Ocean Engineering       Phone:
>                        (617)
>                        253-6824
>                        Dept. of Mechanical Engineering    Fax:
>                        (617)
>                        253-8125
>                        MIT, Room
>                        5-213http://web.mit.edu/phaley/www/
>                        77 Massachusetts Avenue
>                        Cambridge, MA  02139-4301
>
>                    --
>                    Pranith
>
>                    --
>
>                    -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                    Pat Haley
>                    Email:phaley at mit.edu
>                    <mailto:phaley at mit.edu> <phaley at mit.edu>
>                    Center for Ocean Engineering       Phone:
>                    (617)
>                    253-6824
>                    Dept. of Mechanical Engineering    Fax:
>                    (617)
>                    253-8125
>                    MIT, Room 5-213http://web.mit.edu/phaley/www/
>                    77 Massachusetts Avenue
>                    Cambridge, MA  02139-4301
>
>                --
>                Pranith
>
>                --
>
>                -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>                Pat Haley
>                Email:phaley at mit.edu
>                <mailto:phaley at mit.edu> <phaley at mit.edu>
>                Center for Ocean Engineering       Phone:  (617)
>                253-6824
>                Dept. of Mechanical Engineering    Fax:    (617)
>                253-8125
>                MIT, Room 5-213http://web.mit.edu/phaley/www/
>                77 Massachusetts Avenue
>                Cambridge, MA  02139-4301
>
>
>
>
>            --
>            Pranith
>
>            --
>
>            -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>            Pat Haley                          Email:phaley at mit.edu
>            <mailto:phaley at mit.edu> <phaley at mit.edu>
>            Center for Ocean Engineering       Phone:  (617) 253-6824
>            Dept. of Mechanical Engineering    Fax:    (617) 253-8125
>            MIT, Room 5-213http://web.mit.edu/phaley/www/
>            77 Massachusetts Avenue
>            Cambridge, MA  02139-4301
>
>
>
>
>        --
>        Pranith
>
>        --
>
>        -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>        Pat Haley                          Email:phaley at mit.edu
>        <mailto:phaley at mit.edu> <phaley at mit.edu>
>        Center for Ocean Engineering       Phone:  (617) 253-6824
>        Dept. of Mechanical Engineering    Fax:    (617) 253-8125
>        MIT, Room 5-213http://web.mit.edu/phaley/www/
>        77 Massachusetts Avenue
>        Cambridge, MA  02139-4301
>
>
>
>
> --
> Pranith
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>
>
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>
>
> _______________________________________________
> Gluster-users mailing listGluster-users at gluster.orghttp://lists.gluster.org/mailman/listinfo/gluster-users
>
>
> --
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Pat Haley                          Email:  phaley at mit.edu
> Center for Ocean Engineering       Phone:  (617) 253-6824
> Dept. of Mechanical Engineering    Fax:    (617) 253-8125
> MIT, Room 5-213                    http://web.mit.edu/phaley/www/
> 77 Massachusetts Avenue
> Cambridge, MA  02139-4301
>
>


-- 
Pranith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170623/6d1055ce/attachment.html>


More information about the Gluster-users mailing list