<div dir="ltr">Hi,<div><br></div><div><br></div><div>I see a lot of the following messages in the logs:</div><div><div>[2018-02-04 03:22:01.544446] I [glusterfsd-mgmt.c:1821:mgmt_<wbr>getspec_cbk] 0-glusterfs: No change in volfile,continuing</div><div>[2018-02-04 07:41:16.189349] W [MSGID: 109011] [dht-layout.c:186:dht_layout_<wbr>search] 48-gv0-dht: no subvolume for hash (value) = 122440868</div><div>[2018-02-04 07:41:16.244261] W [fuse-bridge.c:2398:fuse_<wbr>writev_cbk] 0-glusterfs-fuse: 3615890: WRITE => -1 gfid=c73ca10f-e83e-42a9-9b0a-<wbr>1de4e12c6798 fd=0x7ffa3802a5f0 (Ошибка ввода/вывода)</div><div>[2018-02-04 07:41:16.254503] W [fuse-bridge.c:1377:fuse_err_<wbr>cbk] 0-glusterfs-fuse: 3615891: FLUSH() ERR => -1 (Ошибка ввода/вывода)</div><div>The message "W [MSGID: 109011] [dht-layout.c:186:dht_layout_<wbr>search] 48-gv0-dht: no subvolume for hash (value) = 122440868" repeated 81 times between [2018-02-04 07:41:16.189349] and [2018-02-04 07:41:16.254480]</div><div>[2018-02-04 10:50:27.624283] W [MSGID: 109011] [dht-layout.c:186:dht_layout_<wbr>search] 48-gv0-dht: no subvolume for hash (value) = 116958174</div><div>[2018-02-04 10:50:27.752107] W [fuse-bridge.c:2398:fuse_<wbr>writev_cbk] 0-glusterfs-fuse: 3997764: WRITE => -1 gfid=18e2adee-ff52-414f-aa37-<wbr>506cff1472ee fd=0x7ffa3801d7d0 (Ошибка ввода/вывода)</div><div>[2018-02-04 10:50:27.762331] W [fuse-bridge.c:1377:fuse_err_<wbr>cbk] 0-glusterfs-fuse: 3997765: FLUSH() ERR => -1 (Ошибка ввода/вывода)</div><div>The message "W [MSGID: 109011] [dht-layout.c:186:dht_layout_<wbr>search] 48-gv0-dht: no subvolume for hash (value) = 116958174" repeated 147 times between [2018-02-04 10:50:27.624283] and [2018-02-04 10:50:27.762292]</div><div>[2018-02-04 10:55:35.256018] W [MSGID: 109011] [dht-layout.c:186:dht_layout_<wbr>search] 48-gv0-dht: no subvolume for hash (value) = 28918667</div><div>[2018-02-04 10:55:35.387073] W [fuse-bridge.c:2398:fuse_<wbr>writev_cbk] 0-glusterfs-fuse: 4006263: WRITE => -1 gfid=54e6f8ea-27d7-4e92-ae64-<wbr>5e198bd3cb42 fd=0x7ffa38036bf0 (Ошибка ввода/вывода)</div><div>[2018-02-04 10:55:35.407554] W [fuse-bridge.c:1377:fuse_err_<wbr>cbk] 0-glusterfs-fuse: 4006264: FLUSH() ERR => -1 (Ошибка ввода/вывода)</div><div>[2018-02-04 10:55:59.677734] W [MSGID: 109011] [dht-layout.c:186:dht_layout_<wbr>search] 48-gv0-dht: no subvolume for hash (value) = 69319528</div><div>[2018-02-04 10:55:59.827012] W [fuse-bridge.c:2398:fuse_<wbr>writev_cbk] 0-glusterfs-fuse: 4014645: WRITE => -1 gfid=ce700d9b-ef55-4e55-a371-<wbr>9642e90555cb fd=0x7ffa38036bf0 (Ошибка ввода/вывода)</div></div><div><br></div><div><br></div><div><br></div><div>This is the reason for the I/O errors you are seeing. Gluster cannot find the subvolume for the file in question so it will fail the write with I/O error. It looks like some bricks may not have been up at the time the volume tried to get the layout.</div><div><br></div><div>This is a problem as this is a pure distributed volume. For some reason the layout is not set on some bricks/some bricks are unreachable. </div><div><br></div><div>There are a lot of graph changes in the logs - I would recommend against so many changes in such a short interval. There aren't logs for the interval before to find out why. Can you send me the rebalance logs from the nodes?</div><div><br></div><div><br></div><div>>I case we have too much capacity that's not needed at the moment we are going to remove-brick and fix-layout again in order to shrink >storage.</div><div><br></div><div><br></div><div>I do see the number of bricks reducing in the graphs.Are you sure a remove-brick has not been run? There is no need to run a fix-layout after using "remove-brick start" as that will automatically rebalance data.</div><div><br></div><div><br></div><div><br></div><div>Regards,</div><div>Nithya</div><div class="gmail_extra"><br><div class="gmail_quote">On 5 February 2018 at 14:06, Nikita Yeryomin <span dir="ltr"><<a href="mailto:nikyer@gmail.com" target="_blank">nikyer@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Attached the log. There are some errors in it like<div><br>[2018-02-04 18:50:41.112962] E [fuse-bridge.c:903:fuse_getatt<wbr>r_resume] 0-glusterfs-fuse: 9613852: GETATTR 140712792330896 (7d39d329-c0e0-4997-85e6-0e66e<wbr>0436315) resolution failed<br><br>But when it occurs it seems not affecting current file i/o operations.</div><div>I've already re-created the volume yesterday and I was not able to reproduce the error during file download after that, but still there are errors in logs like above and system seems a bit unstable.<br>Let me share some more details on how we are trying to use glusterfs.<br>So it's distributed NOT replicated volume with sharding enabled.</div><div>We have many small servers (20GB each) in a cloud and a need to work with rather large files (~300GB).</div><div>We start volume with one 15GB brick which is a separate XFS partition on each server and then add bricks one by one to reach needed capacity.</div><div>After each brick is added we do rebalance fix-layout. </div><div>I case we have too much capacity that's not needed at the moment we are going to remove-brick and fix-layout again in order to shrink storage. But we have not yet been able to test removing bricks as system behaves not stable after scaling out.</div><div><br></div><div>What I've found here <a href="https://bugzilla.redhat.com/show_bug.cgi?id=875076" target="_blank">https://bugzilla.redhat.c<wbr>om/show_bug.cgi?id=875076</a> - seems starting with one brick is not a good idea.. so we are going to try starting with 2 bricks.<br>Please let me know if there are anything else we should consider changing in our strategy.</div><div><br></div><div>Many thanks in advance!</div><span class="gmail-m_5673477752034760262HOEnZb"><font color="#888888"><div>Nikita Yeryomin</div></font></span></div><div class="gmail-m_5673477752034760262HOEnZb"><div class="gmail-m_5673477752034760262h5"><div class="gmail_extra"><br><div class="gmail_quote">2018-02-05 7:53 GMT+02:00 Nithya Balachandran <span dir="ltr"><<a href="mailto:nbalacha@redhat.com" target="_blank">nbalacha@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>Please provide the log for the mount process from the node on which you have mounted the volume. This should be in /var/log/glusterfs and the name of the file will the the hyphenated path of the mount point. For e.g., If the volume in mounted at /mnt/glustervol, the log file will be /var/log/glusterfs/mnt-gluster<wbr>fs.log</div><div><br></div><div>Regards,</div><div>Nithya</div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-m_5673477752034760262m_7028291216856547373h5">On 4 February 2018 at 21:09, Nikita Yeryomin <span dir="ltr"><<a href="mailto:nikyer@gmail.com" target="_blank">nikyer@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-m_5673477752034760262m_7028291216856547373h5"><div dir="ltr"><div class="gmail_quote"><div dir="ltr"><span>Please help troubleshooting glusterfs with the following setup:<br>Distributed volume without replication. Sharding enabled.<br><br>
<span></span>
</span><p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-s1" style="font-variant-ligatures:no-common-ligatures"># cat /etc/centos-release</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-s1" style="font-variant-ligatures:no-common-ligatures">CentOS release 6.9 (Final)</span></p>
<br>
<span></span>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-s1" style="font-variant-ligatures:no-common-ligatures"># glusterfs --version</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246gmail-s1" style="font-variant-ligatures:no-common-ligatures">glusterfs 3.12.3</span></p><div><div class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318h5">
<br>
<span></span>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">[root@master-5f81bad0054a11e8b<wbr>f7d0671029ed6b8 uploads]# gluster volume info</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p2" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255);min-height:13px"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-Apple-converted-space"> </span></span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Volume Name: gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Type: Distribute</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Volume ID: 1a7e05f6-4aa8-48d3-b8e3-300637<wbr>031925</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Status: Started</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Snapshot Count: 0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Number of Bricks: 27</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Transport-type: tcp</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Bricks:</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick1: gluster3.qencode.com:/var/stor<wbr>age/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick2: encoder-376cac0405f311e8847006<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick3: encoder-ee6761c0091c11e891ba06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick4: encoder-ee68b8ea091c11e89c2d06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick5: encoder-ee663700091c11e8b48f06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick6: encoder-efcf113e091c11e8995206<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick7: encoder-efcd5a24091c11e8963a06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick8: encoder-099f557e091d11e882f706<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick9: encoder-099bdda4091d11e8810906<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick10: encoder-099dca56091d11e8b34106<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick11: encoder-09a1ba4e091d11e8a3c206<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick12: encoder-099a826a091d11e8959406<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick13: encoder-0998aa8a091d11e8a81606<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick14: encoder-0b582724091d11e8b3b406<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick15: encoder-0dff527c091d11e896f206<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick16: encoder-0e0d5c14091d11e886cf06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick17: encoder-7f1bf3d4093b11e8a35806<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick18: encoder-7f70378c093b11e8852606<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick19: encoder-7f19528c093b11e88f1006<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick20: encoder-7f76c048093b11e8a74706<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick21: encoder-7f7fc90e093b11e8a74e06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick22: encoder-7f6bc382093b11e8b8a306<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick23: encoder-7f7b44d8093b11e8906f06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick24: encoder-7f72aa30093b11e89a8e06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick25: encoder-7f7d735c093b11e8b46506<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick26: encoder-7f1a5006093b11e89bcb06<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Brick27: encoder-95791076093b11e8af1706<wbr>71029ed6b8.qencode.com:/var/st<wbr>orage/brick/gv0</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Options Reconfigured:</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">cluster.min-free-disk: 10%</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">performance.cache-max-file-siz<wbr>e: 1048576</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">nfs.disable: on</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">transport.address-family: inet</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">features.shard: on</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">performance.client-io-threads: on</span></p>
<br>Each brick is 15Gb size.<div><br>After using volume for several hours with intensive read/write operations (~300GB written and then deleted) an attempt to write to volume results in an Input/Output error:<br><br>
<span></span>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<span></span>
</p><p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures"># wget <a href="https://speed.hetzner.de/1GB.bin" target="_blank">https://speed.hetzner.de/1GB.b<wbr>in</a></span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">--2018-02-04 12:02:34--<span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-Apple-converted-space"> </span><a href="https://speed.hetzner.de/1GB.bin" target="_blank">https://speed.hetzner.de/1GB.b<wbr>in</a></span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Resolving speed.hetzner.de... 88.198.248.254, 2a01:4f8:0:59ed::2</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Connecting to <a href="http://speed.hetzner.de" target="_blank">speed.hetzner.de</a>|88.198.248.25<wbr>4|:443... connected.</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">HTTP request sent, awaiting response... 200 OK</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Length: 1048576000 (1000M) [application/octet-stream]</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Saving to: `1GB.bin'</span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p2" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255);min-height:13px"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">38% [=============================<wbr>==============================<wbr>==> <span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-Apple-converted-space"> </span>] 403,619,518 27.8M/s <span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-Apple-converted-space"> </span>in 15s<span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-Apple-converted-space"> </span></span></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p2" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255);min-height:13px"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p2" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255);min-height:13px"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-p1" style="margin:0px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);background-color:rgb(255,255,255)"><span class="gmail-m_5673477752034760262m_7028291216856547373m_8144923930713690318m_-3596286394412968246m_7848951383961559755gmail-s1" style="font-variant-ligatures:no-common-ligatures">Cannot write to `1GB.bin' (Input/output error).</span></p>
<br><p></p></div><div>I don't see anything written to glusterd.log, or any other logs in /var/log/glusterfs/* when this error occurs.<br><br>Deleting partially downloaded file works without error.<br><br>Thanks,</div><div>Nikita Yeryomin</div></div></div></div>
</div><br></div>
<br></div></div>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>