[Bugs] [Bug 1221099] New: gNFSd does not work correctly/consistently with FSCache/CacheFilesd

[bugzilla at redhat.com]

https://bugzilla.redhat.com/show_bug.cgi?id=1221099

            Bug ID: 1221099
           Summary: gNFSd does not work correctly/consistently with
                    FSCache/CacheFilesd
           Product: GlusterFS
           Version: 3.4.5
         Component: nfs
          Severity: medium
          Assignee: bugs at gluster.org
          Reporter: rwareing at fb.com
                CC: bugs at gluster.org, gluster-bugs at redhat.com



Description of problem:
Possibly a GFS bug, possibly a FSCache/CacheFilesd bug.

When mounting a GFS volume via FSCache:

mount -t nfs -overs=3,fsc,noacl,noatime <gNFSd host>:/<path> <mount dir>

/etc/cachefilesd.conf:
dir  /data/fscache
tag  default_cache
brun  10%
bcull  7%
bstop  3%
frun  10%
fcull  7%
fstop  3%

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Compile kernel w/ FSCache + Cachefilesd
CONFIG_FSCACHE=m
CONFIG_FSCACHE_STATS=y
CONFIG_CACHEFILES=m
CONFIG_NFS_FSCACHE=y

2. Configure cachefilesd.conf file like so:
/etc/cachefilesd.conf:
dir  /data/fscache  <---- In our setup this is an SSD
tag  default_cache
brun  10%
bcull  7%
bstop  3%
frun  10%
fcull  7%
fstop  3%


3. Mount a gluster volume like so:
mount -t nfs -overs=3,fsc,noacl,noatime <gNFSd host>:/<path> /mnt/mycachedmount

4. Write some data to the mount:
for i in {1..20}; do dd if=/dev/urandom of=/mnt/mycachedmount/testfile$i bs=1M
count=100; done

5. For good measure, umount/remount the mount to dump any caches.

6. Read the data back from the mount:
for f in /mnt/mycachedmount/*; do echo $f; dd if=$f of=/dev/null bs=1M; done

Repeat the reading a few times to observe results.

Actual results:
Inconsistent read speeds are observed, and TCPDump shows lots of (NFS) read
requests over the wire.

Expected results:
After the first read loop, no read requests should be observed and the read
throughput should be approximate to the performance of the local cache device
(/data/fscache in this example).

Additional info:
I can only speculate that this behavior is due to gNFSd sending back
inconsistent atime (maybe mtime?) responses which are consumed by FSCache
causing false "misses" on the cache.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.