[Gluster-users] Slow file open

[Mohit Anchlia]

I am running 3.1.3. This application is a java application that simply
spawns threads and writes files. I've broken down the latency into 2
commands:

1) FileOpen
2) File Write including close.

Below is the code:

    public boolean writeTextFile(String dir, String fname, String data){
        BufferedWriter bufferedOutput = null;
        String filename = commandArgs.path + "/" + String.valueOf(dir)
+ "/" + fname;
        boolean suc=false;
        long lat = System.currentTimeMillis();
        long lat_w = 0l;
        try {
            //Construct the BufferedOutputStream object
            log.info("Writing to text file " + filename);
            bufferedOutput = new BufferedWriter(new FileWriter(filename));

            log.info("FileOpen: write " + (System.currentTimeMillis() - lat));
            //Start writing to the output stream
            lat_w = System.currentTimeMillis();
            bufferedOutput.write(data);
            suc=true;
        } catch (FileNotFoundException ex) {
        	log.error("File not found " + filename);
        } catch (IOException ex) {
            ex.printStackTrace();
        } finally {
            //Close the BufferedOutputStream
            try {
                if (bufferedOutput != null) {
                   // bufferedOutput.flush();
                    bufferedOutput.close();
                }
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }
        log.info("File: write " + (System.currentTimeMillis() - lat_w));
        return suc;
   	
    }

On Tue, May 3, 2011 at 11:00 PM, Pavan <tcp at gluster.com> wrote:
> On Tuesday 03 May 2011 09:52 PM, Mohit Anchlia wrote:
>>
>> Yes I can reproduce everytime I run my tests.
>> This happens always. Not sure if you call 50 threads writing files to
>> 4 X 2 servers high load.
>
> What version of GlusterFS are you using?
>
> What application is seeing the slow open? How are you measuring the open
> latency?
>
>> Application see slower response time and low throughput. For eg: you
>> would expect at least 80MB/s on 1GigE with 4 X 2 when each cluster
>> with "dd" can do 130 MB/s. But because of this I see only 20-30MB/s.
>
> I am not very sure I would dive right away to that conclusion. Lets first
> measure what is taking long to complete.
>
> Pavan
>
>>
>> On Mon, May 2, 2011 at 9:36 PM, Pavan<tcp at gluster.com>  wrote:
>>>
>>> On Sunday 01 May 2011 03:59 AM, Mohit Anchlia wrote:
>>>>
>>>> Can someone from dev help with this? Is it overhead in fuse or
>>>> something else? I am seeing file open operation taking anywhere
>>>> between 30ms - 2 secs and this is killing the performance.
>>>
>>> * Can you recreate this problem on your setup at will?
>>> * Does the delay happen when you are opening any file or is it under
>>> certain
>>> scenarios - like excessive load? What is application that is facing this
>>> hang?
>>>
>>> To start with, check if the open on the backend (on a single brick) is
>>> taking long, with the same application.
>>>
>>> Pavan
>>>
>>>>
>>>> On Fri, Apr 29, 2011 at 4:27 PM, Mohit Anchlia<mohitanchlia at gmail.com>
>>>>  wrote:
>>>>>
>>>>> I am not seeing optimal performance out of my 4 X 2 cluster node. When
>>>>> I look at thread state that's writing files I see most of them are
>>>>> waiting on file open.
>>>>>
>>>>> Does anyone know how to further troubleshoot this issue?
>>>>>
>>>> _______________________________________________
>>>> Gluster-users mailing list
>>>> Gluster-users at gluster.org
>>>> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>>>
>>>
>
>