<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
Hi Pranith,<br>
<br>
I ran the same 'dd' test both in the gluster test volume and in the
.glusterfs directory of each brick. The median results (12 dd
trials in each test) are similar to before<br>
<ul>
<li>gluster test volume: 586.5 MB/s</li>
<li>bricks (in .glusterfs): 1.4 GB/s<br>
</li>
</ul>
The profile for the gluster test-volume is in<br>
<br>
<a class="moz-txt-link-freetext" href="http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/profile_testvol_gluster.txt">http://mseas.mit.edu/download/phaley/GlusterUsers/TestVol/profile_testvol_gluster.txt</a><br>
<br>
Thanks<br>
<br>
Pat<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 05/30/2017 12:10 PM, Pranith Kumar
Karampuri wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAOgeEnY1nLK=XP-x1HWich5tsyM+j5azhwXGdq+xfwAEoERq9A@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">Let's start with the same 'dd' test we were testing
with to see, what the numbers are. Please provide profile
numbers for the same. From there on we will start tuning the
volume to see what we can do.<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, May 30, 2017 at 9:16 PM, Pat
Haley <span dir="ltr"><<a href="mailto:phaley@mit.edu"
target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> <br>
Hi Pranith,<br>
<br>
Thanks for the tip. We now have the gluster volume
mounted under /home. What tests do you recommend we run?<br>
<br>
Thanks<span class="HOEnZb"><font color="#888888"><br>
<br>
Pat</font></span>
<div>
<div class="h5"><br>
<br>
<br>
<div class="m_-3319843143517043073moz-cite-prefix">On
05/17/2017 05:01 AM, Pranith Kumar Karampuri wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, May 16, 2017 at
9:20 PM, Pat Haley <span dir="ltr"><<a
href="mailto:phaley@mit.edu"
target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Hi Pranith,<br>
<br>
Sorry for the delay. I never saw received
your reply (but I did receive Ben Turner's
follow-up to your reply). So we tried to
create a gluster volume under /home using
different variations of<br>
<br>
<tt><span>gluster volume create
test-volume
mseas-data2:/home/gbrick_test_<wbr>1
mseas-data2:/home/gbrick_test_<wbr>2
transport tcp</span></tt><tt><br>
</tt><br>
However we keep getting errors of the form<br>
<br>
<tt>Wrong brick type: transport, use
<HOSTNAME>:<export-dir-abs-pat<wbr>h></tt><tt><br>
</tt><br>
Any thoughts on what we're doing wrong?<br>
</div>
</blockquote>
<div><br>
</div>
<div>You should give transport tcp at the
beginning I think. Anyways, transport tcp is
the default, so no need to specify so remove
those two words from the CLI. <br>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Also do you have a list of the test we
should be running once we get this volume
created? Given the time-zone difference
it might help if we can run a small
battery of tests and post the results
rather than test-post-new test-post... .<br>
</div>
</blockquote>
<div><br>
</div>
<div>This is the first time I am doing
performance analysis on users as far as I
remember. In our team there are separate
engineers who do these tests. Ben who
replied earlier is one such engineer.<br>
<br>
</div>
<div>Ben,<br>
</div>
<div> Have any suggestions?<br>
</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> <br>
Thanks<span
class="m_-3319843143517043073HOEnZb"><font
color="#888888"><br>
<br>
Pat</font></span>
<div>
<div class="m_-3319843143517043073h5"><br>
<br>
<br>
<div
class="m_-3319843143517043073m_5903432224896103956moz-cite-prefix">On
05/11/2017 12:06 PM, Pranith Kumar
Karampuri wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu,
May 11, 2017 at 9:32 PM, Pat
Haley <span dir="ltr"><<a
href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> <br>
Hi Pranith,<br>
<br>
The /home partition is
mounted as ext4<br>
<tt>/home
ext4
defaults,usrquota,grpquota
1 2</tt><tt><br>
</tt><br>
The brick partitions are
mounted ax xfs<br>
<tt>/mnt/brick1 xfs
defaults 0 0</tt><tt><br>
</tt><tt>/mnt/brick2 xfs
defaults 0 0</tt><tt><br>
</tt><br>
Will this cause a problem
with creating a volume
under /home?<span
class="m_-3319843143517043073m_5903432224896103956gmail-HOEnZb"><font
color="#888888"><br>
</font></span></div>
</blockquote>
<div><br>
</div>
<div>I don't think the
bottleneck is disk. You can
do the same tests you did on
your new volume to confirm?<br>
</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span
class="m_-3319843143517043073m_5903432224896103956gmail-HOEnZb"><font
color="#888888"> <br>
Pat</font></span>
<div>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-h5"><br>
<br>
<br>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372moz-cite-prefix">On
05/11/2017 11:32 AM,
Pranith Kumar
Karampuri wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr"><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Thu, May 11,
2017 at 8:57
PM, Pat Haley
<span
dir="ltr"><<a
href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
Hi Pranith,<br>
<br>
Unfortunately,
we don't have
similar
hardware for a
small scale
test. All we
have is our
production
hardware.<span
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372HOEnZb"><font
color="#888888"><br>
</font></span></div>
</blockquote>
<div><br>
</div>
<div>You said
something
about /home
partition
which has
lesser disks,
we can create
plain
distribute
volume inside
one of those
directories.
After we are
done, we can
remove the
setup. What do
you say?<br>
<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF"><span
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372HOEnZb"><font
color="#888888"> <br>
Pat</font></span>
<div>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372h5"><br>
<br>
<br>
<br>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417moz-cite-prefix">On
05/11/2017
07:05 AM,
Pranith Kumar
Karampuri
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr"><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Thu, May 11,
2017 at 2:48
AM, Pat Haley
<span
dir="ltr"><<a
href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
Hi Pranith,<br>
<br>
Since we are
mounting the
partitions as
the bricks, I
tried the dd
test writing
to
<brick-path>/.glusterfs/<file-<wbr>to-be-removed-after-test>.
The results
without
oflag=sync
were 1.6 Gb/s
(faster than
gluster but
not as fast as
I was
expecting
given the 1.2
Gb/s to the
no-gluster
area w/ fewer
disks).<span
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417HOEnZb"><font
color="#888888"><br>
</font></span></div>
</blockquote>
<div><br>
</div>
<div>Okay,
then 1.6Gb/s
is what we
need to target
for,
considering
your volume is
just
distribute. Is
there any way
you can do
tests on
similar
hardware but
at a small
scale? Just so
we can run the
workload to
learn more
about the
bottlenecks in
the system? We
can probably
try to get the
speed to
1.2Gb/s on
your /home
partition you
were telling
me yesterday.
Let me know if
that is
something you
are okay to
do.<br>
</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF"><span
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417HOEnZb"><font
color="#888888"> <br>
Pat</font></span>
<div>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417h5"><br>
<br>
<br>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055moz-cite-prefix">On
05/10/2017
01:27 PM,
Pranith Kumar
Karampuri
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr"><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Wed, May 10,
2017 at 10:15
PM, Pat Haley
<span
dir="ltr"><<a
href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
Hi Pranith,<br>
<br>
Not entirely
sure (this
isn't my area
of
expertise).
I'll run your
answer by some
other people
who are more
familiar with
this.<br>
<br>
I am also
uncertain
about how to
interpret the
results when
we also add
the dd tests
writing to the
/home area (no
gluster, still
on the same
machine)<br>
<ul>
<li>dd test
without
oflag=sync
(rough average
of multiple
tests)<br>
</li>
<ul>
<li>gluster w/
fuse mount :
570 Mb/s</li>
<li>gluster w/
nfs mount:
390 Mb/s</li>
<li>nfs (no
gluster): 1.2
Gb/s</li>
</ul>
<li>dd test
with
oflag=sync
(rough average
of multiple
tests)</li>
<ul>
<li>gluster w/
fuse mount: 5
Mb/s</li>
<li>gluster w/
nfs mount:
200 Mb/s</li>
<li>nfs (no
gluster): 20
Mb/s<br>
</li>
</ul>
</ul>
Given that the
non-gluster
area is a
RAID-6 of 4
disks while
each brick of
the gluster
area is a
RAID-6 of 32
disks, I would
naively expect
the writes to
the gluster
area to be
roughly 8x
faster than to
the
non-gluster.<br>
</div>
</blockquote>
<div><br>
</div>
<div>I think a
better test is
to try and
write to a
file using nfs
without any
gluster to a
location that
is not inside
the brick but
someother
location that
is on same
disk(s). If
you are
mounting the
partition as
the brick,
then we can
write to a
file inside
.glusterfs
directory,
something like
<brick-path>/.glusterfs/<file-<wbr>to-be-removed-after-test>.
<br>
<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
I still think
we have a
speed issue, I
can't tell if
fuse vs nfs is
part of the
problem.</div>
</blockquote>
<div><br>
</div>
I got
interested in
the post
because I read
that fuse
speed is
lesser than
nfs speed
which is
counter-intuitive
to my
understanding.
So wanted
clarifications.
Now that I got
my
clarifications
where fuse
outperformed
nfs without
sync, we can
resume testing
as described
above and try
to find what
it is. Based
on your
email-id I am
guessing you
are from
Boston and I
am from
Bangalore so
if you are
okay with
doing this
debugging for
multiple days
because of
timezones, I
will be happy
to help.
Please be a
bit patient
with me, I am
under a
release crunch
but I am very
curious with
the problem
you posted.<br>
<br>
</div>
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
Was there
anything
useful in the
profiles?<span
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055HOEnZb"><font
color="#888888"><br>
</font></span></div>
</blockquote>
<div><br>
</div>
<div>Unfortunately
profiles
didn't help me
much, I think
we are
collecting the
profiles from
an active
volume, so it
has a lot of
information
that is not
pertaining to
dd so it is
difficult to
find the
contributions
of dd. So I
went through
your post
again and
found
something I
didn't pay
much attention
to earlier
i.e.
oflag=sync, so
did my own
tests on my
setup with
FUSE so sent
that reply.<br>
<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF"><span
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055HOEnZb"><font
color="#888888"> <br>
Pat</font></span>
<div>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055h5"><br>
<br>
<br>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930moz-cite-prefix">On
05/10/2017
12:15 PM,
Pranith Kumar
Karampuri
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">
<div>
<div>Okay
good. At least
this validates
my doubts.
Handling
O_SYNC in
gluster NFS
and fuse is a
bit different.<br>
</div>
When
application
opens a file
with O_SYNC on
fuse mount
then each
write syscall
has to be
written to
disk as part
of the syscall
where as in
case of NFS,
there is no
concept of
open. NFS
performs write
though a
handle saying
it needs to be
a synchronous
write, so
write()
syscall is
performed
first then it
performs
fsync(). so an
write on an fd
with O_SYNC
becomes
write+fsync. I
am suspecting
that when
multiple
threads do
this
write+fsync()
operation on
the same file,
multiple
writes are
batched
together to be
written do
disk so the
throughput on
the disk is
increasing is
my guess.<br>
<br>
</div>
Does it answer
your doubts?<br>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Wed, May 10,
2017 at 9:35
PM, Pat Haley
<span
dir="ltr"><<a
href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
Without the
oflag=sync and
only a single
test of each,
the FUSE is
going faster
than NFS:<br>
<br>
FUSE:<br>
<tt>mseas-data2(dri_nascar)%
dd
if=/dev/zero
count=4096
bs=1048576
of=zeros.txt
conv=sync</tt><tt><br>
</tt><tt>4096+0
records in</tt><tt><br>
</tt><tt>4096+0
records out</tt><tt><br>
</tt><tt>4294967296
bytes (4.3 GB)
copied,
7.46961 s, 575
MB/s</tt><tt><br>
</tt><tt><br>
<br>
</tt>NFS<br>
<tt>mseas-data2(HYCOM)%
dd
if=/dev/zero
count=4096
bs=1048576
of=zeros.txt
conv=sync</tt><tt><br>
</tt><tt>4096+0
records in</tt><tt><br>
</tt><tt>4096+0
records out</tt><tt><br>
</tt><tt>4294967296
bytes (4.3 GB)
copied,
11.4264 s, 376
MB/s</tt>
<div>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930h5"><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365moz-cite-prefix">On
05/10/2017
11:53 AM,
Pranith Kumar
Karampuri
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">Could
you let me
know the speed
without
oflag=sync on
both the
mounts? No
need to
collect
profiles.<br>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Wed, May 10,
2017 at 9:17
PM, Pat Haley
<span
dir="ltr"><<a
href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div
bgcolor="#FFFFFF">
<br>
Here is what I
see now:<br>
<br>
<tt>[root@mseas-data2
~]# gluster
volume info</tt><span><tt><br>
</tt><tt> </tt><tt><br>
</tt><tt>Volume
Name:
data-volume</tt><tt><br>
</tt><tt>Type:
Distribute</tt><tt><br>
</tt><tt>Volume
ID:
c162161e-2a2d-4dac-b015-f31fd8<wbr>9ceb18</tt><tt><br>
</tt><tt>Status:
Started</tt><tt><br>
</tt><tt>Number
of Bricks: 2</tt><tt><br>
</tt><tt>Transport-type:
tcp</tt><tt><br>
</tt><tt>Bricks:</tt><tt><br>
</tt><tt>Brick1:
mseas-data2:/mnt/brick1</tt><tt><br>
</tt><tt>Brick2:
mseas-data2:/mnt/brick2</tt><tt><br>
</tt><tt>Options
Reconfigured:</tt><tt><br>
</tt></span><tt>diagnostics.count-fop-hits:
on</tt><tt><br>
</tt><tt>diagnostics.latency-measuremen<wbr>t:
on</tt><tt><br>
</tt><tt>nfs.exports-auth-enable:
on</tt><tt><br>
</tt><tt>diagnostics.brick-sys-log-leve<wbr>l:
WARNING</tt><span><tt><br>
</tt><tt>performance.readdir-ahead:
on</tt><tt><br>
</tt><tt>nfs.disable:
on</tt><tt><br>
</tt><tt>nfs.export-volumes:
off</tt><tt><br>
</tt><br>
<br>
<br>
</span>
<div>
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365h5">
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365m_5079054141158038028moz-cite-prefix">On
05/10/2017
11:44 AM,
Pranith Kumar
Karampuri
wrote:<br>
</div>
<blockquote
type="cite">
<div dir="ltr">
<div>Is this
the volume
info you have?<br>
<br>
<pre>><i> [<a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank" moz-do-not-send="true">root at mseas-data2</a> ~]# gluster volume info
</i>><i>
</i>><i> Volume Name: data-volume
</i>><i> Type: Distribute
</i>><i> Volume ID: c162161e-2a2d-4dac-b015-f31fd8<wbr>9ceb18
</i>><i> Status: Started
</i>><i> Number of Bricks: 2
</i>><i> Transport-type: tcp
</i>><i> Bricks:
</i>><i> Brick1: mseas-data2:/mnt/brick1
</i>><i> Brick2: mseas-data2:/mnt/brick2
</i>><i> Options Reconfigured:
</i>><i> performance.readdir-ahead: on
</i>><i> nfs.disable: on
</i>><i> nfs.export-volumes: off
</i></pre>
</div>
I copied this
from old
thread from
2016. This is
distribute
volume. Did
you change any
of the options
in between? </div>
</blockquote>
</div>
</div>
<span>
<pre class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365m_5079054141158038028moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365m_5079054141158038028moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365m_5079054141158038028moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</span></div>
</blockquote>
</div>
--
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365gmail_signature">
<div dir="ltr">Pranith
</div>
</div>
</div>
</blockquote>
<pre class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930m_7039630128565981365moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
--
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930gmail_signature">
<div dir="ltr">Pranith
</div>
</div>
</div>
</blockquote>
<pre class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055m_-1330903101597946930moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
--
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055gmail_signature">
<div dir="ltr">Pranith
</div>
</div>
</div>
</div>
</blockquote>
<pre class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417m_-7935477196994992055moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
--
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417gmail_signature">
<div dir="ltr">Pranith
</div>
</div>
</div>
</div>
</blockquote>
<pre class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372m_6626346771705009417moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
--
<div
class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372gmail_signature">
<div dir="ltr">Pranith
</div>
</div>
</div>
</div>
</blockquote>
<pre class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956gmail-m_-1677399571678699372moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
--
<div
class="m_-3319843143517043073m_5903432224896103956gmail_signature">
<div dir="ltr">Pranith </div>
</div>
</div>
</div>
</blockquote>
<pre class="m_-3319843143517043073m_5903432224896103956moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073m_5903432224896103956moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073m_5903432224896103956moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div
class="m_-3319843143517043073gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">Pranith<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="m_-3319843143517043073moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=<wbr>-=-=-
Pat Haley Email: <a class="m_-3319843143517043073moz-txt-link-abbreviated" href="mailto:phaley@mit.edu" target="_blank" moz-do-not-send="true">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="m_-3319843143517043073moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/" target="_blank" moz-do-not-send="true">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">Pranith<br>
</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email: <a class="moz-txt-link-abbreviated" href="mailto:phaley@mit.edu">phaley@mit.edu</a>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 <a class="moz-txt-link-freetext" href="http://web.mit.edu/phaley/www/">http://web.mit.edu/phaley/www/</a>
77 Massachusetts Avenue
Cambridge, MA 02139-4301
</pre>
</body>
</html>