Rocode >> Contracting >> StockPrice
Using a Ubuntu Linux Virtual Machine on NeCTAR.
Linux Horse Race
A NeCTAR virtual machine running Ubuntu Linux 12.04 LTS was used for a horse race.
The virtual machine had an operating system disc, a scratch disc and an ephemeral disc. Finally it had 920GB of ram.
A bash script was written to drive MySQL through a test of loading data, cleaning the data and then running a sequence of queries on the scratch disk, the ephemeral disc and in memory.
Notice the --local_infile=1 as needed with MySQL on Ubuntu!
Details of this command option may be found on stackoverflow:
A logger was written, using a bash script, to capture each second the memory usage of mysqld, the disc usage on the scratch disc and the disc usage on the ephemeral disc.
It was adapted from the logger detailed on stackoverflow:
The logger output its results to a file in the /tmp folder. These results were downloaded and put in to Excel.
Using the following timestamplog table, the progress of the sql scripts are logged and maybe reviewed afterwards.
It was adapted from the time stamp log table detailed on stackoverflow:
This is a screen shot of what the sql looks like that is used in the testing of the scratch and ephemeral discs.
This is a screen shot of the records generated by the above sql.
The final step was to run the race between the scratch disc, the ephemeral disc and memory.
This is the disc usage on the scratch disc after running the horse race.
The logger output was cut using the timestamplog table details and loaded in to Excel. 3 worksheets were created being scratch, persistent and ram. A duration key column was added that contained the offset from the test start for each entry.
A summary of results was generated combining all three logger outputs. The spreadsheets formulas used are shown here. These resulting columns were graphed as a chart.
The horserace results chart shows the scratch disc usage, the persistent disc usage and ram usage on a second by second basis.
The lines are synchronised at the starting second of each individual test script. The chart shows that scratch and persistent usage were almost identical.
As the logger was synchronised to the nearest second, there is a slight phase shift with all three lines.