[Gluster-devel] Proposal for change regarding latency calculation
joe at julianfamily.org
Sun Jul 27 19:03:33 UTC 2014
I would want tests of all the posix operations. Need a difference not
just in throughput, but in max iops for the various ops.
On 07/27/2014 08:27 AM, Vipul Nayyar wrote:
> As guided by you, I performed the experiment regarding measurement of
> the effect of always enabled profiling. I performed two write tests,
> one with a 20 MB file and the other with a 730 MB file. Each file was
> written 20 times to the mounted volume after clearing the buffers on
> every iteration and the time taken measured with the time command. I
> ran the following bash script for this purpose.
> while [[ $i -lt 21 ]]; do
> sync && echo 3 > /proc/sys/vm/drop_caches
> out=$( time cp /home/vipul/test.avi $path)
> Since the values at different times for writing the same file are
> quite varied, I plotted a graph using the obtained values(Y-axis
> represents seconds) which can be found attached. As you might see in
> these images, there is no clear pattern found in the variation of
> values obtained while writing.
> So according to me, values in both the conditions are quite near to
> each other and equally capable of going quite high or low than the
> mean value and hence, there is no negative effect seen due to the
> change proposed. I hope someone else can shed more light on whether
> setting the option(always enabled profiling) really decreased the
> performance or not.
> Vipul Nayyar
> On Wednesday, 16 July 2014 12:50 PM, Krishnan Parthasarathi
> <kparthas at redhat.com> wrote:
> Following is a proposal for modifying the io profiling capability
> of the io-stats xlator. I recently sent in a
> patch(review.gluster.org/#/c/8244/) regarding that, which uses the
> already written latency related functions in io-stats to dump info
> through meta and added some more data containers which would track
> some more fops related info each time a request goes through
> io-stats. Currently, before the io-stats' custom latency functions
> can run, the measure_latency and count_fop_hits option should be
> enabled. I propose to remove these two options entirely from io-stats.
> In order to track io performance, these options should be enabled
> all the time, or removed entirely, so that a record of io requests
> can be kept since mount time, else enabling these options only
> when it is required will not give you the average statistics over
> the whole period since the start. This is based on the methodology
> of Linux kernel itself, since it internally maintains the io
> statistics data structures all the time and presents it via /proc
> filesystem whenever required. Enabling of any option is not
> required, and the data available represents statistics since the
> boot time.
> I would like to know the views over this, if having io-stats
> profiling info available all the time would be a good thing?
> Could you run the following experiment to measure the effect of
> profiling being enabled always?
> - Fix the I/O workload to be run.
> - Setup 1 (control group) : Run the fixed workload on a volume with
> both the profiling options NOT set.
> - Setup 2 : Run the (same) fixed workload on the same volume with the
> profiling options set.
> - In both setup, measure the latencies observed by the said workload.
> You could use time(1) command
> for a crude measurement.
> This should allow us to make an informed decision on whether there is
> any performance effect
> when profiling is enabled on a volume by default.
> Apart from this, I was going over latency.c in libglusterfs, which
> does a fine job of maintaining latency info for every xlator and
> encountered an anomaly which I thought should be dealt with. The
> function gf_proc_dump_latency_info which dumps the latency array
> for the specified xlator consists of a last line which in the end
> flushes this array through memset after every dump. That means,
> you get different latency info every time you read the profile
> file in meta. I think, flushing the data structure after every
> dump is wrong since, you don't get overall stats since one enabled
> the option at the top of meta, and more importantly, multiple
> applications reading this file can get wrong info, since it gets
> cleared after one read only.
> Clearing of the statistics on every request sounds incorrect to me.
> Could you please send a patch to fix this?
> If my reasons seem apt for you, I'll send a patch over for evaluation.
> Vipul Nayyar
> Gluster-devel mailing list
> Gluster-devel at gluster.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Gluster-devel