[Gluster-users] file size coherence issue

[Vijay Bellur]

On 01/19/2014 06:35 AM, Jason Ferrara wrote:
> I'm having an issue where glusterfs is reporting the incorrect size for
> recently changed files.
>
> On client A, a process acquires a write lock on a file (using fcntl) and
> begins to modify the file. On client B, a process attempts to a aquire a
> write lock on the same file and blocks.  The process on client A
> finishes its modifications, calls fsync, and releases the lock. The
> process on client B then successfully acquires the lock and reads the
> file. At this point while the contents of the file match what was
> written on client A, the size of the file is still reported as the old
> size.
>
> The program below can be used to demonstrate the problem. It repeatedly
> locks a file, reads in the whole file, checks that a string at the
> beginning of the file that indicates the expected file size matches the
> actual size read, and then truncates the file and writes out new data of
> a random size. Running it on the same file from two different clients I
> get things like...
>
> Client A:
> locked
> Old size: 531
> New size: 459
> unlocked
>
> Client B:
> locked
> Old size: 799
> New size: 531
> unlocked
> locked
> Short read. Expected 531 but got 459
> File size now reported as 531
> Size indicated in file is 459
> Press enter to exit
>
> After client A creates a 459 byte file and releases the lock, when
> client B accesses the file, seeking to the end of the file reports the
> end as 531 (the previous size created by client B), while a read of the
> file only returns 459 bytes, and the string at the beginning of the file
> is the 459 written by client A.
>
> If I add a sleep(1) after acquiring the lock then everything works as
> expected.
>
> I'm using gluster 3.4.1, with a single server running CentOS 6.5, with
> multiple bricks (no replication) formatted with xfs. The clients are
> running Fedora 19. Filesystems mounted with
> attribute-timeout=0,entry-timeout=0,acl,selinux. Is there some other
> option I need to maintain client file metadata coherence?


Can you check if the results are different after disabling md-cache? 
md-cache can be disabled by running:

volume set <volname> stat-prefetch off

Regards,
Vijay