[Gluster-devel] Open source SPC-1 Workload IO Pattern

[Michael O'Sullivan]

The git repo for the fio/SPC-1 tool is here: http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/spc1-test

We talked about using this tool here: Lee, D., O'Sullivan, M. J., & Walker, C. G. (2010). Practical Measurement of Typical Disk Performance and Power Consumption using Open Source SPC-1. In 2010 Annual Conference on GreenIT (pp. 8 pages). Singapore. doi:10.5176/978-981-08-7240-3_G-15

As you will see from the git repo, this hasn't been worked on since 2010, but it may be useful and it would be cool to see this work extended. It was quite a while ago, but I think we may have had to modify the original NetApp open source SPC-1 code a little to fix some bugs...

Thanks, Mike

-----Original Message-----
From: gluster-devel-bounces at gluster.org [mailto:gluster-devel-bounces at gluster.org] On Behalf Of Michael O'Sullivan
Sent: Wednesday, 19 November 2014 5:57 p.m.
To: Justin Clift
Cc: gluster-devel at gluster.org
Subject: Re: [Gluster-devel] Open source SPC-1 Workload IO Pattern

Hi Justin & Luis,

We did a branch of fio that implemented this SPC-1 trace a few years ago. I can dig up the code and paper we wrote if it is useful?

Cheers, Mike

> On 19/11/2014, at 4:21 pm, "Justin Clift" <justin at gluster.org> wrote:
> 
> Nifty. :)
> 
> (Yeah, catching up on old unread email, as the wifi in this hotel is 
> so bad I can barely do anything else.  8-10 second ping times to 
> www.gluster.org. :/)
> 
> As a thought, would there be useful analysis/visualisation 
> capabilities if you stored the data into a time series database (eg 
> InfluxDB) then used Grafana (http://grafana.org) on it?
> 
> + Justin
> 
> 
> On Fri, 07 Nov 2014 12:01:56 +0100
> Luis Pabón <lpabon at redhat.com> wrote:
> 
>> Hi guys,
>> I created a simple test program to visualize the I/O pattern of 
>> NetApp's open source spc-1 workload generator. SPC-1 is an enterprise 
>> OLTP type workload created by the Storage Performance Council 
>> (http://www.storageperformance.org/results).  Some of the results are 
>> published and available here:
>> http://www.storageperformance.org/results/benchmark_results_spc1_active .
>> 
>> NetApp created an open source version of this workload and described 
>> it in their publication "A portable, open-source implementation of 
>> the SPC-1 workload" ( 
>> http://www3.lrgl.uqam.ca/csdl/proceedings/iiswc/2005/9461/00/01526014
>> .pdf
>> )
>> 
>> The code is available onGithub: https://github.com/lpabon/spc1 .  All 
>> it does at the moment is capture the pattern, no real IO is 
>> generated. I will be working on a command line program to enable 
>> usage on real block storage systems.  I may either extend fio or 
>> create a tool specifically tailored to the requirements needed to run 
>> this workload.
>> 
>> On github, I have an example IO pattern for a simulation running 50 
>> mil IOs using HRRW_V2. The simulation ran with an ASU1 (Data Store) 
>> size of 45GB, ASU2 (User Store) size of 45GB, and ASU3 (Log) size of 
>> 10GB.
>> 
>> - Luis
>> 
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> 
> 
> 
> --
> GlusterFS - http://www.gluster.org
> 
> An open source, distributed file system scaling to several petabytes, 
> and handling thousands of clients.
> 
> My personal twitter: twitter.com/realjustinclift 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel at gluster.org
http://supercolony.gluster.org/mailman/listinfo/gluster-devel