<div dir="ltr"><div><div><div>Herb,<br></div>I'm trying to weed out issues here.<br><br></div>So, I can see quota turned <b>on</b> and would like you to check the quota settings and test to see system behavior <b>if quota is turned off</b>.<br></div><div><br></div><div>Although the file size that failed migration was 29K, I'm being a bit paranoid while weeding out issues.</div><div><br></div><div>Are you still facing tiering errors ?</div><div>I can see your response to Alex with the disk space consumption and found it a bit ambiguous w.r.t. state of affairs.<br></div><div><br></div><div>--<br></div>Milind<br><br><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 24, 2017 at 11:34 PM, Herb Burnswell <span dir="ltr"><<a href="mailto:herbert.burnswell@gmail.com" target="_blank">herbert.burnswell@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div style="font-size:12.8px"><div>Milind - Thank you for the response..</div><span class=""><div><br></div><div>>> What are the high and low watermarks for the tier set at ?<br><br></div><div style="font-size:small"># gluster volume get <vol> cluster.watermark-hi</div></span><div style="font-size:small">Option Value </div><div style="font-size:small">------ ----- </div><div style="font-size:small">cluster.watermark-hi 90 </div><span class=""><div style="font-size:small"><br></div><div style="font-size:small"># gluster volume get <vol> cluster.watermark-low</div></span><div style="font-size:small">Option Value </div><div style="font-size:small">------ ----- </div><div style="font-size:small">cluster.watermark-low 75 </div><div><br></div></div><span class=""><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">>> What is the size of the file that failed to migrate as per the following tierd log:</div></span><span class="m_5508430874479203959gmail-im" style="font-size:12.8px"><span class=""><div><div style="font-size:12.8px"><br></div><div>>> [2017-10-19 17:52:07.519614] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:edaf97e1-02e0-4838<wbr>-9d26-71ea3aab22fb)</div></div><div><br></div></span><div>The file was a word doc @ 29K in size.</div><div><br></div></span><span class=""><div style="font-size:12.8px">>>If possible, a <b>gluster volume info</b> would also help, instead of going to and fro with questions.<br></div><div style="font-size:12.8px"><br></div></span><div style="font-size:12.8px"><div style="font-size:small"># gluster vol info</div><div style="font-size:small"> </div><div style="font-size:small">Volume Name: ctdb</div><div style="font-size:small">Type: Replicate</div><div style="font-size:small">Volume ID: f679c476-e0dd-4f3a-9813-<wbr>1b26016b5384</div><div style="font-size:small">Status: Started</div><div style="font-size:small">Snapshot Count: 0</div><div style="font-size:small">Number of Bricks: 1 x 2 = 2</div><div style="font-size:small">Transport-type: tcp</div><div style="font-size:small">Bricks:</div><div style="font-size:small">Brick1: <node1>:/mnt/ctdb_local/brick</div><div style="font-size:small">Brick2: <node2>:/mnt/ctdb_local/brick</div><div style="font-size:small">Options Reconfigured:</div><div style="font-size:small">nfs.disable: on</div><div style="font-size:small">transport.address-family: inet</div><div style="font-size:small"> </div><div style="font-size:small">Volume Name: <vol></div><div style="font-size:small">Type: Tier</div><div style="font-size:small">Volume ID: 7710ed2f-775e-4dd9-92ad-<wbr>66407c72b0ad</div><div style="font-size:small">Status: Started</div><div style="font-size:small">Snapshot Count: 0</div><div style="font-size:small">Number of Bricks: 8</div><div style="font-size:small">Transport-type: tcp</div><div style="font-size:small">Hot Tier :</div><div style="font-size:small">Hot Tier Type : Distributed-Replicate</div><div style="font-size:small">Number of Bricks: 2 x 2 = 4</div><div style="font-size:small">Brick1: <node2>:/mnt/brick_nvme1/brick</div><div style="font-size:small">Brick2: <node1>:/mnt/brick_nvme2/brick</div><div style="font-size:small">Brick3: <node2>:/mnt/brick_nvme2/brick</div><div style="font-size:small">Brick4: <node1>:/mnt/brick_nvme1/brick</div><div style="font-size:small">Cold Tier:</div><div style="font-size:small">Cold Tier Type : Distributed-Replicate</div><div style="font-size:small">Number of Bricks: 2 x 2 = 4</div><div style="font-size:small">Brick5: <node1>:/mnt/brick1/brick</div><div style="font-size:small">Brick6: <node2>:/mnt/brick2/brick</div><div style="font-size:small">Brick7: <node1>:/mnt/brick2/brick</div><div style="font-size:small">Brick8: <node2>:/mnt/brick1/brick</div><div style="font-size:small">Options Reconfigured:</div><div style="font-size:small">cluster.lookup-optimize: on</div><div style="font-size:small">client.event-threads: 4</div><div style="font-size:small">server.event-threads: 4</div><div style="font-size:small">performance.write-behind-<wbr>window-size: 4MB</div><div style="font-size:small">performance.cache-size: 16GB</div><div style="font-size:small">features.quota-deem-statfs: on</div><div style="font-size:small">features.inode-quota: on</div><div style="font-size:small">features.quota: on</div><div style="font-size:small">nfs.disable: on</div><div style="font-size:small">transport.address-family: inet</div><div style="font-size:small">features.ctr-enabled: on</div><div style="font-size:small">cluster.tier-mode: cache</div><div style="font-size:small">performance.io-cache: off</div><div style="font-size:small">performance.quick-read: off</div><div style="font-size:small">cluster.tier-max-files: 1000000</div></div></div><span class="HOEnZb"><font color="#888888"><div><br></div><div><br></div><div>HB</div><div> <br></div><div><br></div><div><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Oct 22, 2017 at 8:41 AM, Milind Changire <span dir="ltr"><<a href="mailto:mchangir@redhat.com" target="_blank">mchangir@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Herb,</div><div>What are the high and low watermarks for the tier set at ?<br><br></div># gluster volume get <vol> cluster.watermark-hi<br><br></div># gluster volume get <vol> cluster.watermark-low</div><div><br></div><div>What is the size of the file that failed to migrate as per the following tierd log:</div><span><div><div style="font-size:12.8px"><br></div><div>[2017-10-19 17:52:07.519614] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:edaf97e1-02e0-4838<wbr>-9d26-71ea3aab22fb)</div></div><div><br></div></span><div>If possible, a <b>gluster volume info</b> would also help, instead of going to and fro with questions.<br></div><div><br></div><div>--</div><div>Milind</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_5508430874479203959h5">On Fri, Oct 20, 2017 at 12:42 AM, Herb Burnswell <span dir="ltr"><<a href="mailto:herbert.burnswell@gmail.com" target="_blank">herbert.burnswell@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_5508430874479203959h5"><div dir="ltr"><span style="font-size:12.8px">All,</span><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">I am new to gluster and have some questions/concerns about some tiering errors that I see in the log files.</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">OS: CentOs 7.3.1611</div><div style="font-size:12.8px">Gluster version: 3.10.5</div><div style="font-size:12.8px">Samba version: 4.6.2</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">I see the following (scrubbed):</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Node 1 /var/log/glusterfs/tier/<vol>/<wbr>tierd.log:</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>[2017-10-19 17:52:07.519614] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:edaf97e1-02e0-4838<wbr>-9d26-71ea3aab22fb)</div><div>[2017-10-19 17:52:07.525110] E [MSGID: 109011] [dht-common.c:7188:dht_create] 0-<vol>-hot-dht: no subvolume in layout for path=/path/to/<file></div><div>[2017-10-19 17:52:07.526088] E [MSGID: 109023] [dht-rebalance.c:757:__dht_reb<wbr>alance_create_dst_file] 0-<vol>-tier-dht: failed to create <file> on <vol>-hot-dht [Input/output error]</div><div>[2017-10-19 17:52:07.526111] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr>ate_file] 0-<vol>-tier-dht: Create dst failed on - <vol>-hot-dht for file - <file></div><div>[2017-10-19 17:52:07.527214] E [MSGID: 109037] [tier.c:969:tier_migrate_link] 0-<vol>-tier-dht: Failed to migrate <file> [No space left on device]</div><div>[2017-10-19 17:52:07.527244] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:fb4411c4-a387-4e5f<wbr>-a2b7-897633ef4aa8)</div><div>[2017-10-19 17:52:07.533510] E [MSGID: 109011] [dht-common.c:7188:dht_create] 0-<vol>-hot-dht: no subvolume in layout for path=/path/to/<file></div><div>[2017-10-19 17:52:07.534434] E [MSGID: 109023] [dht-rebalance.c:757:__dht_reb<wbr>alance_create_dst_file] 0-<vol>-tier-dht: failed to create <file> on <vol>-hot-dht [Input/output error]</div><div>[2017-10-19 17:52:07.534453] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr>ate_file] 0-<vol>-tier-dht: Create dst failed on - <vol>-hot-dht for file - <file></div><div>[2017-10-19 17:52:07.535570] E [MSGID: 109037] [tier.c:969:tier_migrate_link] 0-<vol>-tier-dht: Failed to migrate <file> [No space left on device]</div><div>[2017-10-19 17:52:07.535594] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:fba421e7-0500-47c4<wbr>-bf67-10a40690e13d)</div><div>[2017-10-19 17:52:07.541363] E [MSGID: 109011] [dht-common.c:7188:dht_create] 0-<vol>-hot-dht: no subvolume in layout for path=/path/to/<file></div><div>[2017-10-19 17:52:07.542296] E [MSGID: 109023] [dht-rebalance.c:757:__dht_reb<wbr>alance_create_dst_file] 0-<vol>-tier-dht: failed to create <file> on <vol>-hot-dht [Input/output error]</div><div>[2017-10-19 17:52:07.542357] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr>ate_file] 0-<vol>-tier-dht: Create dst failed on - <vol>-hot-dht for file - <file></div><div>[2017-10-19 17:52:07.543480] E [MSGID: 109037] [tier.c:969:tier_migrate_link] 0-<vol>-tier-dht: Failed to migrate <file> [No space left on device]</div><div>[2017-10-19 17:52:07.543521] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:fe6799e1-42e6-43e5<wbr>-a7eb-ac8facfcbc9f)</div><div>[2017-10-19 17:52:07.549959] E [MSGID: 109011] [dht-common.c:7188:dht_create] 0-<vol>-hot-dht: no subvolume in layout for path=/path/to/<file></div><div>[2017-10-19 17:52:07.550901] E [MSGID: 109023] [dht-rebalance.c:757:__dht_reb<wbr>alance_create_dst_file] 0-<vol>-tier-dht: failed to create <file> on <vol>-hot-dht [Input/output error]</div><div>[2017-10-19 17:52:07.550922] E [MSGID: 0] [dht-rebalance.c:1696:dht_migr<wbr>ate_file] 0-<vol>-tier-dht: Create dst failed on - <vol>-hot-dht for file - <file></div><div>[2017-10-19 17:52:07.551896] E [MSGID: 109037] [tier.c:969:tier_migrate_link] 0-<vol>-tier-dht: Failed to migrate <file> [No space left on device]</div><div>[2017-10-19 17:52:07.551917] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:ffe3a3f2-b170-43f0<wbr>-a9fb-97c78e3173eb)</div><div>[2017-10-19 17:52:07.551945] E [MSGID: 109037] [tier.c:2565:tier_run] 0-<vol>-tier-dht: Promotion failed</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Node 1 /var/log/samba/glusterfs-<vol><wbr>-pool.log:</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>[2017-10-18 17:13:41.481860] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-0: remote operation failed. Path: /pool/testing (7d89b9a8-3e5d-4f28-9e57-039fe<wbr>4416994) [Invalid argument]</div><div>[2017-10-18 17:13:41.481860] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-1: remote operation failed. Path: /pool/testing (7d89b9a8-3e5d-4f28-9e57-039fe<wbr>4416994) [Invalid argument]</div><div>[2017-10-18 17:13:41.485916] E [MSGID: 109089] [dht-helper.c:517:dht_check_an<wbr>d_open_fd_on_subvol_task] 0-<vol>-tier-dht: Failed to open the fd (0x7f02bf1ff570, flags=00) on file 7d89b9a8-3e5d-4f28-9e57-039fe4<wbr>416994 @ <vol>-cold-dht [Invalid argument]</div><div>[2017-10-18 17:13:41.488223] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-0: remote operation failed. Path: /pool/testing (7d89b9a8-3e5d-4f28-9e57-039fe<wbr>4416994) [Invalid argument]</div><div>[2017-10-18 17:13:41.488235] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-1: remote operation failed. Path: /pool/testing (7d89b9a8-3e5d-4f28-9e57-039fe<wbr>4416994) [Invalid argument]</div><div>[2017-10-18 17:13:41.489060] E [MSGID: 109089] [dht-helper.c:517:dht_check_an<wbr>d_open_fd_on_subvol_task] 0-<vol>-tier-dht: Failed to open the fd (0x7f02bf1feb50, flags=00) on file 7d89b9a8-3e5d-4f28-9e57-039fe4<wbr>416994 @ <vol>-cold-dht [Invalid argument]</div><div>[2017-10-18 17:13:42.339936] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-4: remote operation failed. Path: /pool (34d76e11-412f-4bc6-9a3e-b1f89<wbr>658f13b) [Invalid argument]</div><div>[2017-10-18 17:13:42.339988] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-5: remote operation failed. Path: /pool (34d76e11-412f-4bc6-9a3e-b1f89<wbr>658f13b) [Invalid argument]</div><div>[2017-10-18 17:13:42.343769] E [MSGID: 109089] [dht-helper.c:517:dht_check_an<wbr>d_open_fd_on_subvol_task] 0-<vol>-tier-dht: Failed to open the fd (0x7f02bf2012c0, flags=00) on file 34d76e11-412f-4bc6-9a3e-b1f896<wbr>58f13b @ <vol>-hot-dht [Invalid argument]</div><div>[2017-10-18 17:13:42.345374] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-4: remote operation failed. Path: /pool (34d76e11-412f-4bc6-9a3e-b1f89<wbr>658f13b) [Invalid argument]</div><div>[2017-10-18 17:13:42.345401] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-5: remote operation failed. Path: /pool (34d76e11-412f-4bc6-9a3e-b1f89<wbr>658f13b) [Invalid argument]</div><div>[2017-10-18 17:13:42.346259] E [MSGID: 109089] [dht-helper.c:517:dht_check_an<wbr>d_open_fd_on_subvol_task] 0-<vol>-tier-dht: Failed to open the fd (0x7f02bf201130, flags=00) on file 34d76e11-412f-4bc6-9a3e-b1f896<wbr>58f13b @ <vol>-hot-dht [Invalid argument]</div><div>[2017-10-18 17:13:59.541591] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.</div><div>[2017-10-18 17:13:59.541748] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up.</div><div>[2017-10-18 17:13:59.541887] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-2: All subvolumes are down. Going offline until atleast one of them comes back up.</div><div>[2017-10-18 17:13:59.541977] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-3: All subvolumes are down. Going offline until atleast one of them comes back up.</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Node 2 /var/log/gluster/tier/<vol>/ti<wbr>erd.log:<br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>[2017-10-16 15:54:08.662873] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:fffd714e-b2d2-42d3<wbr>-a31f-72673276e3d0)</div><div>[2017-10-16 16:00:07.201584] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:f10365e1-747b-4985<wbr>-97b9-8b5dc61ac464)</div><div>[2017-10-16 16:00:07.372559] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:f95f17bf-b696-44cd<wbr>-aae0-d8ac38149aa5)</div><div>[2017-10-16 16:06:06.880522] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:ec451f6c-8971-4f9b<wbr>-a04f-00f96db9b46a)</div><div>[2017-10-16 16:06:08.062080] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:e658cd70-3f6d-4b25<wbr>-8d9f-0d4c24d3ec5d)</div><div>[2017-10-16 16:06:08.288298] I [MSGID: 109038] [tier.c:1169:tier_migrate_usin<wbr>g_query_file] 0-<vol>-tier-dht: Promotion failed for <file>(gfid:f22df67a-88e5-4fae<wbr>-aab0-b00e04f9a6e1)</div><div>[2017-10-18 15:55:06.446416] I [MSGID: 109028] [dht-rebalance.c:4792:gf_defra<wbr>g_status_get] 0-glusterfs: Rebalance is in progress. Time taken is 1376671.00 secs</div><div>[2017-10-18 15:55:06.446433] I [MSGID: 109028] [dht-rebalance.c:4796:gf_defra<wbr>g_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 47887089, failures: 3594, skipped: 0</div><div>[2017-10-19 00:00:00.501576] I [MSGID: 109038] [tier.c:2391:tier_prepare_comp<wbr>act] 0-<vol>-tier-dht: Start compaction on cold tier</div><div>[2017-10-19 00:00:00.502016] I [MSGID: 109038] [tier.c:2403:tier_prepare_comp<wbr>act] 0-<vol>-tier-dht: End compaction on cold tier</div><div>[2017-10-19 00:00:00.501608] I [MSGID: 109038] [tier.c:2391:tier_prepare_comp<wbr>act] 0-<vol>-tier-dht: Start compaction on cold tier</div><div>[2017-10-19 00:00:00.502076] I [MSGID: 109038] [tier.c:2403:tier_prepare_comp<wbr>act] 0-<vol>-tier-dht: End compaction on cold tier</div><div>[2017-10-19 16:03:49.522991] I [MSGID: 109028] [dht-rebalance.c:4792:gf_defra<wbr>g_status_get] 0-glusterfs: Rebalance is in progress. Time taken is 1463594.00 secs</div><div>[2017-10-19 16:03:49.523017] I [MSGID: 109028] [dht-rebalance.c:4796:gf_defra<wbr>g_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 52790654, failures: 3594, skipped: 0</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Node 2 /var/log/samba/glusterfs-<vol><wbr>-pool.log:</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>[2017-10-18 16:49:09.218062] E [MSGID: 114031] [client-rpc-fops.c:443:client3<wbr>_3_open_cbk] 0-<vol>-client-4: remote operation failed. Path: /pool (34d76e11-412f-4bc6-9a3e-b1f89<wbr>658f13b) [Invalid argument]</div><div>[2017-10-18 16:49:09.218254] E [MSGID: 109089] [dht-helper.c:517:dht_check_an<wbr>d_open_fd_on_subvol_task] 0-<vol>-tier-dht: Failed to open the fd (0x7f009b36bac0, flags=00) on file 34d76e11-412f-4bc6-9a3e-b1f896<wbr>58f13b @ <vol>-hot-dht [Invalid argument]</div><div>[2017-10-18 16:49:09.222783] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.</div><div>[2017-10-18 16:49:09.222912] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up.</div><div>[2017-10-18 16:49:09.223079] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-2: All subvolumes are down. Going offline until atleast one of them comes back up.</div><div>[2017-10-18 16:49:09.223200] E [MSGID: 108006] [afr-common.c:4808:afr_notify] 0-<vol>-replicate-3: All subvolumes are down. Going offline until atleast one of them comes back up.</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Status:<br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div># gluster vol tier <vol> status</div><div><br></div><div>Node Promoted files Demoted files Status run time in h:m:s </div><div>--------- --------- --------- --------- --------- </div><div>Node1 190861 0 in progress 408:34:13</div><div>Node2 0 0 in progress 408:34:14</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Hot tier bricks:</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"># df -h</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>/dev/mapper/vg_bricks-brick_nv<wbr>me1 1.4T 551G 883G 39% /mnt/brick_nvme1</div><div>/dev/mapper/vg_bricks-brick_nv<wbr>me2 1.4T 512G 922G 36% /mnt/brick_nvme2</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Can anyone point me in the right direction as to what may be going on? Any guidance is greatly appreciated.</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Thanks in advance,</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">HB</div></div>
<br></div></div>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><span class="m_5508430874479203959HOEnZb"><font color="#888888"><br></font></span></blockquote></div><span class="m_5508430874479203959HOEnZb"><font color="#888888"><br><br clear="all"><br>-- <br><div class="m_5508430874479203959m_-819107161572781637gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Milind<br><br></div></div></div></div>
</font></span></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Milind<br><br></div></div></div></div>
</div>