<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hello</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I am thinking of building a Gluster file system for <b>archival data</b>. Initially it will start as 6 brick dispersed volume then expand to distributed dispersed as we increase capacity.
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Since metadata in Gluster isn't centralized it will eventually not perform well at scale. So I am wondering if anyone can help identify that point? Ceph can scale to extremely high levels though the complexity required for management seems much greater than
Gluster. <br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The first six bricks would be a little over <b>2PB</b> of raw space. Each server will have 24 7200 RPM NL-SAS drives sans RAID. I estimate we would max out at about<b> 100 million files</b> within these first six servers, though that can be reduced by having
users tar their small files before writing to Gluster. I/O patterns would be sequential upon initial copy with<b><i> very infrequent reads
</i></b>thereafter. Given the demands of erasure coding, especially if we lose a brick, the CPUs will be high thread count AMD Rome. The back-end network would be
<b>EDR</b> Infiniband, so I will mount via RDMA, while all bricks will be leaf local.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Given these variables can anyone say whether Gluster would be able to operate at this level of metadata and continue to scale? If so where could it break, 4PB, 12PB, with that being defined as I/O, with all bricks still online, breaking down dramatically?<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thank you!</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Doug</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p style="margin-top: 0px; margin-bottom: 0px;margin-top:0; margin-bottom:0"><span id="ms-rterangepaste-start"></span><span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">-- </span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">Thanks,</span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">Douglas Duckworth, MSc, LFCS</span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">HPC System Administrator</span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px"><a href="https://scu.med.cornell.edu/" class="OWAAutoLink">Scientific Computing Unit</a></span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">Weill Cornell Medicine</span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">E: doug@med.cornell.edu</span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">O: 212-746-6305</span><br style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">
<span style="color:rgb(33,33,33); font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont; font-size:14.6667px">F: 212-746-8690</span><span id="ms-rterangepaste-end"></span><br>
</p>
</div>
</div>
</body>
</html>