iccsa-16-factory-extended

git clone https://git.igankevich.com/iccsa-16-factory-extended.git
Log | Files | Refs

commit be4844fabcaa8e427d230e2f23bf6d3630738ae9
parent f34aebc87afa3544022b39ec92a66175588045fb
Author: Ivan Gankevich <igankevich@ya.ru>
Date:   Thu,  4 May 2017 12:00:27 +0300

Revise the text after reviewers suggestions.

Diffstat:
Makefile | 5++++-
data/mpp-time-color-diff.gnu | 28++++++++++++++++++++++++++++
data/mpp-time-color-diff.svg | 189+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
data/mpp-time-color.gnu | 29+++++++++++++++++++++++++++++
data/mpp-time-color.svg | 253+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
data/style-perf-plot.gnu | 10++++++++++
data/time-backup-node-fault.tsv | 11+++++++++++
data/time-master-fault.tsv | 11+++++++++++
data/time-no-faults.tsv | 12++++++++++++
data/time-slave-fault.tsv | 11+++++++++++
data/time.csv | 352+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
refs.bib | 13+++++++++++++
src/intro.tex | 47+++++++++++++++++++++++++++++++++--------------
src/sections.tex | 61++++++++++++++++++++++++++++++++++++++++++++-----------------
14 files changed, 1000 insertions(+), 32 deletions(-)

diff --git a/Makefile b/Makefile @@ -1,10 +1,13 @@ -build/iccsa-16-factory.pdf: build *.tex src/* build/ppl.pdf +build/iccsa-16-factory.pdf: build *.tex src/* build/ppl.pdf build/ratio.eps latexmk -interaction=nonstopmode -output-directory=build \ -pdf -bibtex -shell-escape -quiet -f main.tex build/ppl.pdf: dot/ppl.dot dot -Tpdf -o $@ $< +build/ratio.eps: data/mpp-time-color-diff.svg + inkscape -z --export-eps=$@ $< + build: mkdir -p build diff --git a/data/mpp-time-color-diff.gnu b/data/mpp-time-color-diff.gnu @@ -0,0 +1,28 @@ +#!/usr/bin/gnuplot -persist + +load 'style-perf-plot.gnu' + +set style line 1 lw 2 lc rgb '#9F2D20' +set style line 2 lw 2 lc rgb '#404040' dashtype (8,6) +set style line 3 lw 2 lc rgb '#404040' pointtype 6 +set style line 4 lw 2 lc rgb '#404040' dashtype (3,3) +set style line 5 lw 2 lc rgb '#404040' dashtype (2,5) + +set terminal svg size 16*100/3.5,9*100/4 \ + font "Times, 12" \ + lw 1 rounded dashed enhanced + +set key top center Left reverse width -20 +set xlabel 'Initial no. of nodes' +set ylabel 'Performance ratio' offset 2.25,0 + +set xtics 1,1 +set ytics 0,1,3 +set yrange [0:3] + +set output 'mpp-time-color-diff.svg' +plot \ +'< paste time-slave-fault.tsv time-no-faults.tsv' using 1:($2/$10) with lines ls 2 title 'Time with a slave node failure',\ +'< paste time-master-fault.tsv time-no-faults.tsv' using 1:($2/$10) with points ls 3 title 'Time with a master node failure',\ +'< paste time-backup-node-fault.tsv time-no-faults.tsv' using 1:($2/$10) with lines ls 4 title 'Time with a backup node failure' + diff --git a/data/mpp-time-color-diff.svg b/data/mpp-time-color-diff.svg @@ -0,0 +1,189 @@ +<?xml version="1.0" encoding="utf-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" + "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg + width="457" height="225" + viewBox="0 0 457 225" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" +> + +<title>Gnuplot</title> +<desc>Produced by GNUPLOT 5.0 patchlevel 5 </desc> + +<g id="gnuplot_canvas"> + +<rect x="0" y="0" width="457" height="225" fill="none"/> +<defs> + + <circle id='gpDot' r='0.5' stroke-width='0.5'/> + <path id='gpPt0' stroke-width='0.222' stroke='currentColor' d='M-1,0 h2 M0,-1 v2'/> + <path id='gpPt1' stroke-width='0.222' stroke='currentColor' d='M-1,-1 L1,1 M1,-1 L-1,1'/> + <path id='gpPt2' stroke-width='0.222' stroke='currentColor' d='M-1,0 L1,0 M0,-1 L0,1 M-1,-1 L1,1 M-1,1 L1,-1'/> + <rect id='gpPt3' stroke-width='0.222' stroke='currentColor' x='-1' y='-1' width='2' height='2'/> + <rect id='gpPt4' stroke-width='0.222' stroke='currentColor' fill='currentColor' x='-1' y='-1' width='2' height='2'/> + <circle id='gpPt5' stroke-width='0.222' stroke='currentColor' cx='0' cy='0' r='1'/> + <use xlink:href='#gpPt5' id='gpPt6' fill='currentColor' stroke='none'/> + <path id='gpPt7' stroke-width='0.222' stroke='currentColor' d='M0,-1.33 L-1.33,0.67 L1.33,0.67 z'/> + <use xlink:href='#gpPt7' id='gpPt8' fill='currentColor' stroke='none'/> + <use xlink:href='#gpPt7' id='gpPt9' stroke='currentColor' transform='rotate(180)'/> + <use xlink:href='#gpPt9' id='gpPt10' fill='currentColor' stroke='none'/> + <use xlink:href='#gpPt3' id='gpPt11' stroke='currentColor' transform='rotate(45)'/> + <use xlink:href='#gpPt11' id='gpPt12' fill='currentColor' stroke='none'/> + <path id='gpPt13' stroke-width='0.222' stroke='currentColor' d='M0,1.330 L1.265,0.411 L0.782,-1.067 L-0.782,-1.076 L-1.265,0.411 z'/> + <use xlink:href='#gpPt13' id='gpPt14' fill='currentColor' stroke='none'/> + <filter id='textbox' filterUnits='objectBoundingBox' x='0' y='0' height='1' width='1'> + <feFlood flood-color='white' flood-opacity='1' result='bgnd'/> + <feComposite in='SourceGraphic' in2='bgnd' operator='atop'/> + </filter> + <filter id='greybox' filterUnits='objectBoundingBox' x='0' y='0' height='1' width='1'> + <feFlood flood-color='lightgrey' flood-opacity='1' result='grey'/> + <feComposite in='SourceGraphic' in2='grey' operator='atop'/> + </filter> +</defs> +<g fill="none" color="white" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,176.4 L37.3,176.4 '/> <g transform="translate(33.1,180.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="end"> + <text><tspan font-family="Times" > 0</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,123.2 L37.3,123.2 '/> <g transform="translate(33.1,127.1)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="end"> + <text><tspan font-family="Times" > 1</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,69.9 L37.3,69.9 '/> <g transform="translate(33.1,73.8)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="end"> + <text><tspan font-family="Times" > 2</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,16.7 L37.3,16.7 '/> <g transform="translate(33.1,20.6)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="end"> + <text><tspan font-family="Times" > 3</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,176.4 L46.3,185.4 '/> <g transform="translate(46.3,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 2</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M84.9,176.4 L84.9,185.4 '/> <g transform="translate(84.9,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 3</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M123.5,176.4 L123.5,185.4 '/> <g transform="translate(123.5,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 4</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M162.0,176.4 L162.0,185.4 '/> <g transform="translate(162.0,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 5</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M200.6,176.4 L200.6,185.4 '/> <g transform="translate(200.6,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 6</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M239.2,176.4 L239.2,185.4 '/> <g transform="translate(239.2,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 7</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M277.8,176.4 L277.8,185.4 '/> <g transform="translate(277.8,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 8</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M316.4,176.4 L316.4,185.4 '/> <g transform="translate(316.4,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 9</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M354.9,176.4 L354.9,185.4 '/> <g transform="translate(354.9,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 10</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M393.5,176.4 L393.5,185.4 '/> <g transform="translate(393.5,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 11</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M432.1,176.4 L432.1,185.4 '/> <g transform="translate(432.1,198.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" > 12</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,16.7 L46.3,176.4 L432.1,176.4 M432.1,16.7 M46.3,16.7 '/></g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(17.6,96.6) rotate(270)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" >Performance ratio</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(239.2,216.3)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="middle"> + <text><tspan font-family="Times" >Initial no. of nodes</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> + <g id="gnuplot_plot_1" ><title>Time with a slave node failure</title> +<g fill="none" color="white" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(222.9,38.6)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="start"> + <text><tspan font-family="Times" >Time with a slave node failure</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='rgb( 64, 64, 64)' stroke-dasharray='8.0,6.0' d='M172.4,34.7 L214.6,34.7 M46.3,122.7 L84.9,122.4 L162.0,123.4 L200.6,128.1 L239.2,107.9 L277.8,119.2 + L316.4,113.4 L354.9,94.1 L393.5,121.7 L432.1,122.2 '/></g> + </g> + <g id="gnuplot_plot_2" ><title>Time with a master node failure</title> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(222.9,56.6)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="start"> + <text><tspan font-family="Times" >Time with a master node failure</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <use xlink:href='#gpPt5' transform='translate(46.3,123.4) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(84.9,123.2) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(162.0,122.4) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(200.6,125.3) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(239.2,118.1) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(277.8,126.7) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(316.4,99.0) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(354.9,111.0) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(393.5,117.0) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(432.1,112.9) scale(4.50)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(193.5,52.7) scale(4.50)' color='rgb( 64, 64, 64)'/> +</g> + </g> + <g id="gnuplot_plot_3" ><title>Time with a backup node failure</title> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(222.9,74.6)" stroke="none" fill="black" font-family="Times" font-size="12.00" text-anchor="start"> + <text><tspan font-family="Times" >Time with a backup node failure</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='rgb( 64, 64, 64)' stroke-dasharray='3.0,3.0' d='M172.4,70.7 L214.6,70.7 M46.3,112.8 L84.9,113.1 L162.0,124.0 L200.6,124.4 L239.2,125.3 L277.8,124.1 + L316.4,120.9 L354.9,116.6 L393.5,117.2 L432.1,120.2 '/></g> + </g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="black" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M46.3,16.7 L46.3,176.4 L432.1,176.4 M432.1,16.7 M46.3,16.7 '/></g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +</g> +</svg> + diff --git a/data/mpp-time-color.gnu b/data/mpp-time-color.gnu @@ -0,0 +1,29 @@ +#!/usr/bin/gnuplot -persist + +load 'style-perf-plot.gnu' + +set style line 1 lw 2 lc rgb '#9F2D20' +set style line 2 lw 2 lc rgb '#404040' dashtype (8,6) +set style line 3 lw 2 lc rgb '#404040' pointtype 6 +set style line 4 lw 2 lc rgb '#404040' dashtype (3,3) +set style line 5 lw 2 lc rgb '#404040' dashtype (2,5) + +set terminal svg size 16*100/3,9*100/3.5 \ + font "Open Sans Regular, 14" \ + lw 1 rounded dashed enhanced + +set key top center Left reverse width -20 +set xlabel 'Initial no. of nodes' +set ylabel 'Time, s' offset 2.25,0 + +set xtics 1,1 +set ytics 0,30 +set yrange [0:*] + +set output 'mpp-time-color.svg' +plot \ +'time-no-faults.tsv' using 1:2 with lines smooth csplines ls 1 title 'Time without failures',\ +'time-slave-fault.tsv' using 1:2 with lines smooth csplines ls 2 title 'Time with a slave node failure',\ +'time-master-fault.tsv' using 1:2 with points ls 3 title 'Time with a master node failure',\ +'time-backup-node-fault.tsv' using 1:2 with lines smooth csplines ls 4 title 'Time with a backup node failure' + diff --git a/data/mpp-time-color.svg b/data/mpp-time-color.svg @@ -0,0 +1,253 @@ +<?xml version="1.0" encoding="utf-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" + "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<svg + width="533" height="257" + viewBox="0 0 533 257" + xmlns="http://www.w3.org/2000/svg" + xmlns:xlink="http://www.w3.org/1999/xlink" +> + +<title>Gnuplot</title> +<desc>Produced by GNUPLOT 5.0 patchlevel 5 </desc> + +<g id="gnuplot_canvas"> + +<rect x="0" y="0" width="533" height="257" fill="none"/> +<defs> + + <circle id='gpDot' r='0.5' stroke-width='0.5'/> + <path id='gpPt0' stroke-width='0.190' stroke='currentColor' d='M-1,0 h2 M0,-1 v2'/> + <path id='gpPt1' stroke-width='0.190' stroke='currentColor' d='M-1,-1 L1,1 M1,-1 L-1,1'/> + <path id='gpPt2' stroke-width='0.190' stroke='currentColor' d='M-1,0 L1,0 M0,-1 L0,1 M-1,-1 L1,1 M-1,1 L1,-1'/> + <rect id='gpPt3' stroke-width='0.190' stroke='currentColor' x='-1' y='-1' width='2' height='2'/> + <rect id='gpPt4' stroke-width='0.190' stroke='currentColor' fill='currentColor' x='-1' y='-1' width='2' height='2'/> + <circle id='gpPt5' stroke-width='0.190' stroke='currentColor' cx='0' cy='0' r='1'/> + <use xlink:href='#gpPt5' id='gpPt6' fill='currentColor' stroke='none'/> + <path id='gpPt7' stroke-width='0.190' stroke='currentColor' d='M0,-1.33 L-1.33,0.67 L1.33,0.67 z'/> + <use xlink:href='#gpPt7' id='gpPt8' fill='currentColor' stroke='none'/> + <use xlink:href='#gpPt7' id='gpPt9' stroke='currentColor' transform='rotate(180)'/> + <use xlink:href='#gpPt9' id='gpPt10' fill='currentColor' stroke='none'/> + <use xlink:href='#gpPt3' id='gpPt11' stroke='currentColor' transform='rotate(45)'/> + <use xlink:href='#gpPt11' id='gpPt12' fill='currentColor' stroke='none'/> + <path id='gpPt13' stroke-width='0.190' stroke='currentColor' d='M0,1.330 L1.265,0.411 L0.782,-1.067 L-0.782,-1.076 L-1.265,0.411 z'/> + <use xlink:href='#gpPt13' id='gpPt14' fill='currentColor' stroke='none'/> + <filter id='textbox' filterUnits='objectBoundingBox' x='0' y='0' height='1' width='1'> + <feFlood flood-color='white' flood-opacity='1' result='bgnd'/> + <feComposite in='SourceGraphic' in2='bgnd' operator='atop'/> + </filter> + <filter id='greybox' filterUnits='objectBoundingBox' x='0' y='0' height='1' width='1'> + <feFlood flood-color='lightgrey' flood-opacity='1' result='grey'/> + <feComposite in='SourceGraphic' in2='grey' operator='atop'/> + </filter> +</defs> +<g fill="none" color="white" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,200.4 L63.0,200.4 '/> <g transform="translate(58.1,205.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 0</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,170.2 L63.0,170.2 '/> <g transform="translate(58.1,174.8)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 30</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,140.1 L63.0,140.1 '/> <g transform="translate(58.1,144.7)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 60</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,109.9 L63.0,109.9 '/> <g transform="translate(58.1,114.5)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 90</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,79.8 L63.0,79.8 '/> <g transform="translate(58.1,84.4)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 120</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,49.6 L63.0,49.6 '/> <g transform="translate(58.1,54.2)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 150</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,19.5 L63.0,19.5 '/> <g transform="translate(58.1,24.1)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="end"> + <text><tspan font-family="Open Sans Regular" > 180</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,200.4 L73.5,210.9 '/> <g transform="translate(73.5,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 1</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M112.6,200.4 L112.6,210.9 '/> <g transform="translate(112.6,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 2</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M151.7,200.4 L151.7,210.9 '/> <g transform="translate(151.7,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 3</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M190.9,200.4 L190.9,210.9 '/> <g transform="translate(190.9,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 4</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M230.0,200.4 L230.0,210.9 '/> <g transform="translate(230.0,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 5</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M269.1,200.4 L269.1,210.9 '/> <g transform="translate(269.1,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 6</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M308.2,200.4 L308.2,210.9 '/> <g transform="translate(308.2,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 7</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M347.3,200.4 L347.3,210.9 '/> <g transform="translate(347.3,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 8</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M386.4,200.4 L386.4,210.9 '/> <g transform="translate(386.4,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 9</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M425.6,200.4 L425.6,210.9 '/> <g transform="translate(425.6,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 10</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M464.7,200.4 L464.7,210.9 '/> <g transform="translate(464.7,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 11</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M503.8,200.4 L503.8,210.9 '/> <g transform="translate(503.8,226.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" > 12</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,19.5 L73.5,200.4 L503.8,200.4 M503.8,19.5 M73.5,19.5 '/></g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(20.6,110.0) rotate(270)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" >Time, s</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(288.6,247.0)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="middle"> + <text><tspan font-family="Open Sans Regular" >Initial no. of nodes</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> + <g id="gnuplot_plot_1" ><title>Time without failures</title> +<g fill="none" color="white" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(269.6,45.1)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="start"> + <text><tspan font-family="Open Sans Regular" >Time without failures</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='rgb(159, 45, 32)' d='M210.6,40.5 L259.9,40.5 M73.5,64.8 L77.8,72.9 L82.2,80.9 L86.5,88.7 L90.9,96.4 L95.2,103.8 + L99.6,110.9 L103.9,117.6 L108.3,123.7 L112.6,129.4 L117.0,134.4 L121.3,138.9 L125.7,142.9 L130.0,146.3 + L134.4,149.3 L138.7,151.9 L143.0,154.1 L147.4,155.9 L151.7,157.4 L156.1,158.7 L160.4,159.7 L164.8,160.5 + L169.1,161.1 L173.5,161.7 L177.8,162.1 L182.2,162.5 L186.5,162.8 L190.9,163.2 L195.2,163.6 L199.5,164.1 + L203.9,164.6 L208.2,165.1 L212.6,165.7 L216.9,166.3 L221.3,166.9 L225.6,167.4 L230.0,168.0 L234.3,168.5 + L238.7,169.0 L243.0,169.5 L247.4,169.8 L251.7,170.2 L256.1,170.4 L260.4,170.5 L264.7,170.6 L269.1,170.5 + L273.4,170.3 L277.8,170.0 L282.1,169.7 L286.5,169.3 L290.8,169.0 L295.2,168.7 L299.5,168.5 L303.9,168.5 + L308.2,168.6 L312.6,168.9 L316.9,169.5 L321.2,170.1 L325.6,170.9 L329.9,171.8 L334.3,172.7 L338.6,173.7 + L343.0,174.7 L347.3,175.7 L351.7,176.7 L356.0,177.6 L360.4,178.5 L364.7,179.3 L369.1,180.0 L373.4,180.7 + L377.8,181.3 L382.1,181.8 L386.4,182.2 L390.8,182.6 L395.1,182.8 L399.5,182.9 L403.8,183.0 L408.2,183.0 + L412.5,183.0 L416.9,182.9 L421.2,182.8 L425.6,182.6 L429.9,182.4 L434.3,182.3 L438.6,182.1 L442.9,181.9 + L447.3,181.7 L451.6,181.6 L456.0,181.5 L460.3,181.5 L464.7,181.5 L469.0,181.6 L473.4,181.7 L477.7,181.9 + L482.1,182.1 L486.4,182.4 L490.8,182.7 L495.1,183.0 L499.5,183.3 L503.8,183.6 '/></g> + </g> + <g id="gnuplot_plot_2" ><title>Time with a slave node failure</title> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(269.6,66.1)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="start"> + <text><tspan font-family="Open Sans Regular" >Time with a slave node failure</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='rgb( 64, 64, 64)' stroke-dasharray='8.0,6.0' d='M210.6,61.5 L259.9,61.5 M112.6,63.5 L116.6,71.2 L120.5,78.8 L124.5,86.3 L128.4,93.6 L132.4,100.6 + L136.3,107.2 L140.3,113.4 L144.2,119.2 L148.2,124.3 L152.1,128.8 L156.1,132.7 L160.0,135.9 L164.0,138.6 + L167.9,140.9 L171.9,142.7 L175.8,144.3 L179.8,145.7 L183.7,146.9 L187.7,148.1 L191.6,149.3 L195.6,150.6 + L199.5,152.0 L203.5,153.4 L207.5,154.8 L211.4,156.3 L215.4,157.8 L219.3,159.3 L223.3,160.8 L227.2,162.3 + L231.2,163.8 L235.1,165.2 L239.1,166.5 L243.0,167.7 L247.0,168.8 L250.9,169.7 L254.9,170.4 L258.8,170.9 + L262.8,171.1 L266.7,171.1 L270.7,170.8 L274.6,170.2 L278.6,169.4 L282.5,168.4 L286.5,167.2 L290.4,166.1 + L294.4,164.9 L298.3,163.9 L302.3,162.9 L306.2,162.2 L310.2,161.7 L314.1,161.5 L318.1,161.6 L322.0,161.9 + L326.0,162.3 L329.9,162.9 L333.9,163.6 L337.8,164.3 L341.8,165.1 L345.7,165.9 L349.7,166.7 L353.6,167.5 + L357.6,168.2 L361.6,168.8 L365.5,169.4 L369.5,169.9 L373.4,170.4 L377.4,170.7 L381.3,171.0 L385.3,171.2 + L389.2,171.2 L393.2,171.2 L397.1,171.2 L401.1,171.1 L405.0,171.1 L409.0,171.1 L412.9,171.1 L416.9,171.3 + L420.8,171.7 L424.8,172.2 L428.7,172.9 L432.7,173.8 L436.6,174.8 L440.6,175.9 L444.5,177.1 L448.5,178.2 + L452.4,179.3 L456.4,180.4 L460.3,181.3 L464.3,182.0 L468.2,182.6 L472.2,182.9 L476.1,183.1 L480.1,183.1 + L484.0,183.0 L488.0,182.7 L491.9,182.4 L495.9,182.0 L499.8,181.6 L503.8,181.1 '/></g> + </g> + <g id="gnuplot_plot_3" ><title>Time with a master node failure</title> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(269.6,87.1)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="start"> + <text><tspan font-family="Open Sans Regular" >Time with a master node failure</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <use xlink:href='#gpPt5' transform='translate(112.6,65.4) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(151.7,129.5) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(190.9,129.5) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(230.0,162.6) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(269.1,169.3) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(308.2,167.6) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(347.3,170.7) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(386.4,164.5) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(425.6,178.1) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(464.7,180.6) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(503.8,177.8) scale(5.25)' color='rgb( 64, 64, 64)'/> + <use xlink:href='#gpPt5' transform='translate(235.3,82.5) scale(5.25)' color='rgb( 64, 64, 64)'/> +</g> + </g> + <g id="gnuplot_plot_4" ><title>Time with a backup node failure</title> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <g transform="translate(269.6,108.1)" stroke="none" fill="black" font-family="Open Sans Regular" font-size="14.00" text-anchor="start"> + <text><tspan font-family="Open Sans Regular" >Time with a backup node failure</tspan></text> + </g> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='rgb( 64, 64, 64)' stroke-dasharray='3.0,3.0' d='M210.6,103.5 L259.9,103.5 M112.6,38.4 L116.6,46.9 L120.5,55.3 L124.5,63.5 L128.4,71.6 L132.4,79.4 + L136.3,86.9 L140.3,94.0 L144.2,100.6 L148.2,106.6 L152.1,112.1 L156.1,116.9 L160.0,121.2 L164.0,124.9 + L167.9,128.2 L171.9,131.2 L175.8,133.9 L179.8,136.4 L183.7,138.7 L187.7,141.0 L191.6,143.4 L195.6,145.8 + L199.5,148.2 L203.5,150.6 L207.5,153.0 L211.4,155.3 L215.4,157.4 L219.3,159.4 L223.3,161.2 L227.2,162.7 + L231.2,163.8 L235.1,164.7 L239.1,165.3 L243.0,165.6 L247.0,165.8 L250.9,165.9 L254.9,165.9 L258.8,165.9 + L262.8,166.0 L266.7,166.1 L270.7,166.3 L274.6,166.7 L278.6,167.2 L282.5,167.8 L286.5,168.3 L290.4,168.9 + L294.4,169.4 L298.3,169.8 L302.3,170.0 L306.2,170.0 L310.2,169.8 L314.1,169.4 L318.1,168.8 L322.0,168.1 + L326.0,167.3 L329.9,166.6 L333.9,165.9 L337.8,165.4 L341.8,165.0 L345.7,164.9 L349.7,165.0 L353.6,165.5 + L357.6,166.2 L361.6,167.2 L365.5,168.2 L369.5,169.4 L373.4,170.7 L377.4,172.0 L381.3,173.2 L385.3,174.3 + L389.2,175.4 L393.2,176.3 L397.1,177.1 L401.1,177.7 L405.0,178.3 L409.0,178.8 L412.9,179.2 L416.9,179.5 + L420.8,179.7 L424.8,180.0 L428.7,180.1 L432.7,180.3 L436.6,180.4 L440.6,180.5 L444.5,180.5 L448.5,180.6 + L452.4,180.6 L456.4,180.6 L460.3,180.6 L464.3,180.6 L468.2,180.6 L472.2,180.6 L476.1,180.6 L480.1,180.6 + L484.0,180.6 L488.0,180.5 L491.9,180.5 L495.9,180.5 L499.8,180.5 L503.8,180.4 '/></g> + </g> +<g fill="none" color="black" stroke="currentColor" stroke-width="2.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="black" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> + <path stroke='black' d='M73.5,19.5 L73.5,200.4 L503.8,200.4 M503.8,19.5 M73.5,19.5 '/></g> +<g fill="none" color="black" stroke="currentColor" stroke-width="1.00" stroke-linecap="round" stroke-linejoin="round"> +</g> +</g> +</svg> + diff --git a/data/style-perf-plot.gnu b/data/style-perf-plot.gnu @@ -0,0 +1,10 @@ +#!/usr/bin/gnuplot +set style line 1 lt 1 lc rgb "#9F2D20" lw 2 +set style line 2 lt 1 lc rgb "#404040" lw 2 +unset border +set border 3 +set xtics nomirror out offset 0,0.5 +set ytics nomirror out offset 0.5,0 +set xlabel offset 0,1 +set ylabel offset 1.75,0 +unset grid diff --git a/data/time-backup-node-fault.tsv b/data/time-backup-node-fault.tsv @@ -0,0 +1,11 @@ +2 161.2 330 445422 master 10.0.0.2 10.0.0.1 30 +3 84.0827 330 445422 master 10.0.0.2 10.0.0.1 30 +#4 56.5185 330 445422 master 10.0.0.2 10.0.0.1 30 +5 36.4645 330 445422 master 10.0.0.2 10.0.0.1 30 +6 31.5238 330 445422 master 10.0.0.2 10.0.0.1 30 +7 28.5404 330 445422 master 10.0.0.2 10.0.0.1 30 +8 31.054 330 445422 master 10.0.0.2 10.0.0.1 30 +9 25.6013 330 445422 master 10.0.0.2 10.0.0.1 30 +10 20.3034 330 445422 master 10.0.0.2 10.0.0.1 30 +11 19.6733 330 445422 master 10.0.0.2 10.0.0.1 30 +12 19.8681 330 445422 master 10.0.0.2 10.0.0.1 30 diff --git a/data/time-master-fault.tsv b/data/time-master-fault.tsv @@ -0,0 +1,11 @@ +2 134.326 330 445422 master 10.0.0.2 10.0.0.2 30 +3 70.5781 330 445422 master 10.0.0.2 10.0.0.2 30 +#4 70.5781 330 445422 master 10.0.0.2 10.0.0.2 30 +5 37.5648 330 445422 master 10.0.0.2 10.0.0.2 30 +6 30.9335 330 445422 master 10.0.0.2 10.0.0.2 30 +7 32.6034 330 445422 master 10.0.0.2 10.0.0.2 30 +8 29.5219 330 445422 master 10.0.0.2 10.0.0.2 30 +9 35.6903 330 445422 master 10.0.0.2 10.0.0.2 30 +10 22.2137 330 445422 master 10.0.0.2 10.0.0.2 30 +11 19.7449 330 445422 master 10.0.0.2 10.0.0.2 30 +12 22.4393 330 445422 master 10.0.0.2 10.0.0.2 30 diff --git a/data/time-no-faults.tsv b/data/time-no-faults.tsv @@ -0,0 +1,12 @@ +1 134.947 330 445422 slave 10.0.0.1 +2 70.6591 330 445422 slave 10.0.0.1 +#3 77.8426 330 445422 slave 10.0.0.1 +4 37.025 330 445422 slave 10.0.0.1 +5 32.2458 330 445422 slave 10.0.0.1 +6 29.7542 330 445422 slave 10.0.0.1 +7 31.616 330 445422 slave 10.0.0.1 +8 24.5504 330 445422 slave 10.0.0.1 +9 18.0728 330 445422 slave 10.0.0.1 +10 17.7044 330 445422 slave 10.0.0.1 +11 18.8038 330 445422 slave 10.0.0.1 +12 16.7043 330 445422 slave 10.0.0.1 diff --git a/data/time-slave-fault.tsv b/data/time-slave-fault.tsv @@ -0,0 +1,11 @@ +2 136.196 330 445422 slave 10.0.0.1 10.0.0.2 30 +3 71.6354 330 445422 slave 10.0.0.1 10.0.0.2 30 +#4 51.0735 330 445422 slave 10.0.0.1 10.0.0.2 30 +5 36.8751 330 445422 slave 10.0.0.1 10.0.0.2 30 +6 29.2801 330 445422 slave 10.0.0.1 10.0.0.2 30 +7 38.277 330 445422 slave 10.0.0.1 10.0.0.2 30 +8 33.9606 330 445422 slave 10.0.0.1 10.0.0.2 30 +9 29.0621 330 445422 slave 10.0.0.1 10.0.0.2 30 +10 27.9565 330 445422 slave 10.0.0.1 10.0.0.2 30 +11 18.2023 330 445422 slave 10.0.0.1 10.0.0.2 30 +12 19.1562 330 445422 slave 10.0.0.1 10.0.0.2 30 diff --git a/data/time.csv b/data/time.csv @@ -0,0 +1,352 @@ +1,32.7408 +2,32.7408 +3,32.7408 +4,32.7408 +5,32.7408 +6,32.7408 +7,32.7408 +8,32.7408 +9,32.7408 +10,32.7408 +11,32.7408 +12,32.7408 +1,32.336 +2,32.2617 +3,32.3155 +4,32.3257 +5,32.2289 +6,32.2859 +7,32.2822 +8,31.9421 +9,5.99232 +10,5.31981 +11,4.87818 +12,4.19531 +1,31.6809 +2,31.1618 +3,31.3773 +4,31.4316 +5,31.3515 +6,31.0103 +7,31.2074 +8,31.1922 +9,5.35432 +10,5.13254 +11,3.83541 +12,4.04164 +2,31.4599 +1,31.6128 +2,31.4113 +2,31.5293 +2,31.1986 +2,31.2436 +2,31.9261 +2,31.3314 +2,31.2293 +2,31.2293 +2,31.2293 +2,31.2293 +2,31.4396 +2,31.4089 +2,31.2739 +2,31.563 +2,31.3894 +2,32.0597 +2,32.4268 +2,32.1717 +2,32.1717 +2,16.7196 +2,16.6984 +1,32.5028 +2,16.6351 +3,11.9063 +4,8.69439 +5,7.23569 +6,6.86142 +7,6.83105 +8,6.82642 +9,5.74178 +10,5.41891 +11,4.1223 +12,3.86847 +1,34.6434,3000,10.0.0.2,10 +2,31.9566,3000,10.0.0.2,10 +3,17.2847,3000,10.0.0.2,10 +4,12.0142,3000,10.0.0.2,10 +5,8.78353,3000,10.0.0.2,10 +6,6.85797,3000,10.0.0.2,10 +7,7.82573,3000,10.0.0.2,10 +8,5.81164,3000,10.0.0.2,10 +9,6.67989,3000,10.0.0.2,10 +10,6.59441,3000,10.0.0.2,10 +11,4.05578,3000,10.0.0.2,10 +12,4.0365,3000,10.0.0.2,10 +2,33.8468,3000,10.0.0.2,10 +3,33.8468,3000,10.0.0.2,10 +4,11.8255,3000,10.0.0.2,10 +5,8.77837,3000,10.0.0.2,10 +6,7.64993,3000,10.0.0.2,10 +7,7.64613,3000,10.0.0.2,10 +8,6.84666,3000,10.0.0.2,10 +9,6.8451,3000,10.0.0.2,10 +10,5.20588,3000,10.0.0.2,10 +11,4.07123,3000,10.0.0.2,10 +12,31.8467,3000,10.0.0.2,10 +3,11.8748,3000,, +3,16.8473,3000,10.0.0.2,10 +12,3.90546,3000,10.0.0.2,10 +1,3.90546,3000,master,10.0.0.2,10.0.0.1,10 +2,48.2225,3000,master,10.0.0.2,10.0.0.1,10 +3,42.5311,3000,master,10.0.0.2,10.0.0.1,10 +4,19.8945,3000,master,10.0.0.2,10.0.0.1,10 +5,38.258,3000,master,10.0.0.2,10.0.0.1,10 +6,22.2827,3000,master,10.0.0.2,10.0.0.1,10 +7,12.6758,3000,master,10.0.0.2,10.0.0.1,10 +8,5.51378,3000,master,10.0.0.2,10.0.0.1,10 +9,5.46293,3000,master,10.0.0.2,10.0.0.1,10 +10,5.22776,3000,master,10.0.0.2,10.0.0.1,10 +11,3.95897,3000,master,10.0.0.2,10.0.0.1,10 +12,3.89017,3000,master,10.0.0.2,10.0.0.1,10 +4,19.5167,3000,master,10.0.0.2,10.0.0.1,10 +5,38.1816,3000,master,10.0.0.2,10.0.0.1,10 +5,38.1069,3000,master,10.0.0.2,10.0.0.1,10 +5,38.1716,3000,master,10.0.0.2,10.0.0.1,10 +5,38.12,3000,master,10.0.0.2,10.0.0.1,10 +5,38.0409,3000,master,10.0.0.2,10.0.0.1,10 +5,38.0902,3000,master,10.0.0.2,10.0.0.1,10 +5,16.659,3000,master,10.0.0.2,10.0.0.1,10 +6,14.5384,3000,master,10.0.0.2,10.0.0.1,10 +7,12.6119,3000,master,10.0.0.2,10.0.0.1,10 +8,13.4705,3000,master,10.0.0.2,10.0.0.1,10 +9,5.53305,3000,master,10.0.0.2,10.0.0.1,10 +10,11.9554,3000,master,10.0.0.2,10.0.0.1,10 +11,3.91913,3000,master,10.0.0.2,10.0.0.1,10 +12,4.06456,3000,master,10.0.0.2,10.0.0.1,10 + +1,32.3116,6000,slave,10.0.0.1,, +2,32.4138,6000,slave,10.0.0.1,, +3,21.6441,6000,slave,10.0.0.1,, +4,22.2427,6000,slave,10.0.0.1,, +5,14.1305,6000,slave,10.0.0.1,, +6,12.0451,6000,slave,10.0.0.1,, +7,11.8793,6000,slave,10.0.0.1,, +8,10.1871,6000,slave,10.0.0.1,, +9,9.01233,6000,slave,10.0.0.1,, +10,8.53571,6000,slave,10.0.0.1,, +11,7.18004,6000,slave,10.0.0.1,, +12,7.06562,6000,slave,10.0.0.1,, +1,7.06562,6000,slave,10.0.0.1,10.0.0.2,10 +2,7.06562,6000,slave,10.0.0.1,10.0.0.2,10 +3,32.6523,6000,slave,10.0.0.1,10.0.0.2,10 +4,22.413,6000,slave,10.0.0.1,10.0.0.2,10 +5,17.3325,6000,slave,10.0.0.1,10.0.0.2,10 +6,13.7935,6000,slave,10.0.0.1,10.0.0.2,10 +7,13.8161,6000,slave,10.0.0.1,10.0.0.2,10 +8,11.78,6000,slave,10.0.0.1,10.0.0.2,10 +9,10.3437,6000,slave,10.0.0.1,10.0.0.2,10 +10,9.53005,6000,slave,10.0.0.1,10.0.0.2,10 +11,8.72914,6000,slave,10.0.0.1,10.0.0.2,10 +12,7.75036,6000,slave,10.0.0.1,10.0.0.2,10 +1,7.75036,6000,master,10.0.0.2,10.0.0.2,10 +2,7.75036,6000,master,10.0.0.2,10.0.0.2,10 +3,32.2108,6000,master,10.0.0.2,10.0.0.2,10 +4,22.6625,6000,master,10.0.0.2,10.0.0.2,10 +5,17.1134,6000,master,10.0.0.2,10.0.0.2,10 +6,14.0891,6000,master,10.0.0.2,10.0.0.2,10 +7,13.9485,6000,master,10.0.0.2,10.0.0.2,10 +8,11.795,6000,master,10.0.0.2,10.0.0.2,10 +9,10.6376,6000,master,10.0.0.2,10.0.0.2,10 +10,9.67405,6000,master,10.0.0.2,10.0.0.2,10 +11,8.70466,6000,master,10.0.0.2,10.0.0.2,10 +12,7.613,6000,master,10.0.0.2,10.0.0.2,10 +1,7.613,6000,master,10.0.0.2,10.0.0.1,10 +2,7.613,6000,master,10.0.0.2,10.0.0.1,10 +3,52.7312,6000,master,10.0.0.2,10.0.0.1,10 +4,38.6789,6000,master,10.0.0.2,10.0.0.1,10 +5,29.5583,6000,master,10.0.0.2,10.0.0.1,10 +6,24.8817,6000,master,10.0.0.2,10.0.0.1,10 +7,24.9546,6000,master,10.0.0.2,10.0.0.1,10 +8,21.8629,6000,master,10.0.0.2,10.0.0.1,10 +9,18.7421,6000,master,10.0.0.2,10.0.0.1,10 +10,16.4871,6000,master,10.0.0.2,10.0.0.1,10 +11,14.6139,6000,master,10.0.0.2,10.0.0.1,10 +12,27.5998,6000,master,10.0.0.2,10.0.0.1,10 +1,12.1157,330,slave,10.0.0.1,, +2,12.1157,330,slave,10.0.0.1,, +3,12.1157,330,slave,10.0.0.1,, +4,12.1157,330,slave,10.0.0.1,, +5,12.1157,330,slave,10.0.0.1,, +6,12.1157,330,slave,10.0.0.1,, +7,12.1157,330,slave,10.0.0.1,, +8,12.1157,330,slave,10.0.0.1,, +9,12.1157,330,slave,10.0.0.1,, +10,12.1157,330,slave,10.0.0.1,, +11,12.1157,330,slave,10.0.0.1,, +12,12.1157,330,slave,10.0.0.1,, +1,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +2,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +3,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +4,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +5,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +6,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +7,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +8,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +9,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +10,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +11,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +12,12.1157,330,slave,10.0.0.1,10.0.0.2,10 +1,12.1157,330,master,10.0.0.2,10.0.0.2,10 +2,12.1157,330,master,10.0.0.2,10.0.0.2,10 +3,12.1157,330,master,10.0.0.2,10.0.0.2,10 +4,12.1157,330,master,10.0.0.2,10.0.0.2,10 +5,12.1157,330,master,10.0.0.2,10.0.0.2,10 +6,12.1157,330,master,10.0.0.2,10.0.0.2,10 +7,12.1157,330,master,10.0.0.2,10.0.0.2,10 +8,12.1157,330,master,10.0.0.2,10.0.0.2,10 +9,12.1157,330,master,10.0.0.2,10.0.0.2,10 +10,12.1157,330,master,10.0.0.2,10.0.0.2,10 +11,12.1157,330,master,10.0.0.2,10.0.0.2,10 +12,12.1157,330,master,10.0.0.2,10.0.0.2,10 +1,12.1157,330,master,10.0.0.2,10.0.0.1,10 +2,12.1157,330,master,10.0.0.2,10.0.0.1,10 +3,12.1157,330,master,10.0.0.2,10.0.0.1,10 +4,12.1157,330,master,10.0.0.2,10.0.0.1,10 +5,12.1157,330,master,10.0.0.2,10.0.0.1,10 +6,12.1157,330,master,10.0.0.2,10.0.0.1,10 +7,12.1157,330,master,10.0.0.2,10.0.0.1,10 +8,12.1157,330,master,10.0.0.2,10.0.0.1,10 +9,44.8852,330,master,10.0.0.2,10.0.0.1,10 +10,44.8852,330,master,10.0.0.2,10.0.0.1,10 +11,44.8852,330,master,10.0.0.2,10.0.0.1,10 +12,44.8852,330,master,10.0.0.2,10.0.0.1,10 + +1,135.393,330,slave,10.0.0.1,, +2,72.3508,330,slave,10.0.0.1,, +1,16.3999,60,slave,10.0.0.1,, +1,16.7549,60,slave,10.0.0.1,, +1,16.5084,60,slave,10.0.0.1,, +2,9.98067,60,slave,10.0.0.1,, +1,16.3771,60,slave,10.0.0.1,, +2,9.88496,60,slave,10.0.0.1,, + +1,134.759,330,slave,10.0.0.1,, +2,71.6349,330,slave,10.0.0.1,, +3,71.6349,330,slave,10.0.0.1,, +4,77.8311,330,slave,10.0.0.1,, +5,77.8311,330,slave,10.0.0.1,, +6,62.1225,330,slave,10.0.0.1,, +7,62.1225,330,slave,10.0.0.1,, +8,61.1542,330,slave,10.0.0.1,, +9,64.6314,330,slave,10.0.0.1,, +10,57.4141,330,slave,10.0.0.1,, +11,229.517,330,slave,10.0.0.1,, +12,229.835,330,slave,10.0.0.1,, +1,133.674,330,slave,10.0.0.1,10.0.0.2,10 +2,133.711,330,slave,10.0.0.1,10.0.0.2,10 +3,71.9921,330,slave,10.0.0.1,10.0.0.2,10 +4,87.8873,330,slave,10.0.0.1,10.0.0.2,10 +5,69.5444,330,slave,10.0.0.1,10.0.0.2,10 +6,69.5444,330,slave,10.0.0.1,10.0.0.2,10 +7,66.812,330,slave,10.0.0.1,10.0.0.2,10 +8,60.8345,330,slave,10.0.0.1,10.0.0.2,10 +9,62.8811,330,slave,10.0.0.1,10.0.0.2,10 +10,62.8811,330,slave,10.0.0.1,10.0.0.2,10 +11,57.3347,330,slave,10.0.0.1,10.0.0.2,10 +12,229.909,330,slave,10.0.0.1,10.0.0.2,10 +1,229.909,330,master,10.0.0.2,10.0.0.2,10 +2,229.909,330,master,10.0.0.2,10.0.0.2,10 +3,229.909,330,master,10.0.0.2,10.0.0.2,10 +4,86.87,330,master,10.0.0.2,10.0.0.2,10 +5,78.0517,330,master,10.0.0.2,10.0.0.2,10 +6,78.0517,330,master,10.0.0.2,10.0.0.2,10 +7,78.0517,330,master,10.0.0.2,10.0.0.2,10 +8,78.0517,330,master,10.0.0.2,10.0.0.2,10 +9,78.0517,330,master,10.0.0.2,10.0.0.2,10 +10,78.0517,330,master,10.0.0.2,10.0.0.2,10 +11,61.7,330,master,10.0.0.2,10.0.0.2,10 +12,61.7,330,master,10.0.0.2,10.0.0.2,10 +1,61.7,330,master,10.0.0.2,10.0.0.1,10 +2,190.963,330,master,10.0.0.2,10.0.0.1,10 +3,106.669,330,master,10.0.0.2,10.0.0.1,10 +4,53.4635,330,master,10.0.0.2,10.0.0.1,10 +5,69.9564,330,master,10.0.0.2,10.0.0.1,10 +6,62.4806,330,master,10.0.0.2,10.0.0.1,10 +7,55.5295,330,master,10.0.0.2,10.0.0.1,10 +8,55.5295,330,master,10.0.0.2,10.0.0.1,10 +9,55.5295,330,master,10.0.0.2,10.0.0.1,10 +10,55.5295,330,master,10.0.0.2,10.0.0.1,10 +11,55.5295,330,master,10.0.0.2,10.0.0.1,10 +12,55.5295,330,master,10.0.0.2,10.0.0.1,10 + +1,16.364,60,slave,10.0.0.1,, +2,10.2836,60,slave,10.0.0.1,, +3,12.0819,60,slave,10.0.0.1,, +4,9.09292,60,slave,10.0.0.1,, +5,8.06462,60,slave,10.0.0.1,, +6,9.91398,60,slave,10.0.0.1,, +7,7.83301,60,slave,10.0.0.1,, +8,9.76658,60,slave,10.0.0.1,, +9,9.67471,60,slave,10.0.0.1,, +10,20.0133,60,slave,10.0.0.1,, +11,8.61517,60,slave,10.0.0.1,, +12,19.9283,60,slave,10.0.0.1,, + +3,84.7268,330,slave,10.0.0.1,, +4,78.2283,330,slave,10.0.0.1,, +2,72.071,330,slave,10.0.0.1,, +3,75.5299,330,slave,10.0.0.1,, +3,86.9932,330,slave,10.0.0.1,, +3,86.2351,330,slave,10.0.0.1,, +3,89.809,330,slave,10.0.0.1,, +2,73.0699,330,slave,10.0.0.1,, +3,73.7194,330,slave,10.0.0.1,, +3,73.7194,330,slave,10.0.0.1,, +3,86.9686,330,slave,10.0.0.1,, +3,86.9686,330,slave,10.0.0.1,, +3,88.6341,330,slave,10.0.0.1,, +3,50.5009,330,slave,10.0.0.1,, +3,50.5149,330,slave,10.0.0.1,, + +1,134.947,330,445422,slave,10.0.0.1,, +2,70.6591,330,445422,slave,10.0.0.1,, +3,77.8426,330,445422,slave,10.0.0.1,, +4,37.025,330,445422,slave,10.0.0.1,, +5,32.2458,330,445422,slave,10.0.0.1,, +6,29.7542,330,445422,slave,10.0.0.1,, +7,31.616,330,445422,slave,10.0.0.1,, +8,24.5504,330,445422,slave,10.0.0.1,, +9,18.0728,330,445422,slave,10.0.0.1,, +10,17.7044,330,445422,slave,10.0.0.1,, +11,18.8038,330,445422,slave,10.0.0.1,, +12,16.7043,330,445422,slave,10.0.0.1,, + +2 136.196 330 445422 slave 10.0.0.1 10.0.0.2 30 +3 71.6354 330 445422 slave 10.0.0.1 10.0.0.2 30 +4 51.0735 330 445422 slave 10.0.0.1 10.0.0.2 30 +5 36.8751 330 445422 slave 10.0.0.1 10.0.0.2 30 +6 29.2801 330 445422 slave 10.0.0.1 10.0.0.2 30 +7 38.277 330 445422 slave 10.0.0.1 10.0.0.2 30 +8 33.9606 330 445422 slave 10.0.0.1 10.0.0.2 30 +9 29.0621 330 445422 slave 10.0.0.1 10.0.0.2 30 +10 27.9565 330 445422 slave 10.0.0.1 10.0.0.2 30 +11 18.2023 330 445422 slave 10.0.0.1 10.0.0.2 30 +12 19.1562 330 445422 slave 10.0.0.1 10.0.0.2 30 + +2 134.326 330 445422 master 10.0.0.2 10.0.0.2 30 +3 70.5781 330 445422 master 10.0.0.2 10.0.0.2 30 +4 70.5781 330 445422 master 10.0.0.2 10.0.0.2 30 +5 37.5648 330 445422 master 10.0.0.2 10.0.0.2 30 +6 30.9335 330 445422 master 10.0.0.2 10.0.0.2 30 +7 32.6034 330 445422 master 10.0.0.2 10.0.0.2 30 +8 29.5219 330 445422 master 10.0.0.2 10.0.0.2 30 +9 35.6903 330 445422 master 10.0.0.2 10.0.0.2 30 +10 22.2137 330 445422 master 10.0.0.2 10.0.0.2 30 +11 19.7449 330 445422 master 10.0.0.2 10.0.0.2 30 +12 22.4393 330 445422 master 10.0.0.2 10.0.0.2 30 + +2 22.4393 330 445422 master 10.0.0.2 10.0.0.1 30 +3 92.6854 330 445422 master 10.0.0.2 10.0.0.1 30 +4 57.9155 330 445422 master 10.0.0.2 10.0.0.1 30 +5 36.4645 330 445422 master 10.0.0.2 10.0.0.1 30 +6 36.5257 330 445422 master 10.0.0.2 10.0.0.1 30 +7 28.5404 330 445422 master 10.0.0.2 10.0.0.1 30 diff --git a/refs.bib b/refs.bib @@ -264,3 +264,16 @@ doi={10.1109/CSITechnol.2013.6710358}, art_number={6710358}, } + +@inproceedings{zaharia2012resilient, + title={Resilient distributed datasets: A fault-tolerant abstraction for + in-memory cluster computing}, + author={Zaharia, Matei and Chowdhury, Mosharaf and Das, Tathagata and Dave, + Ankur and Ma, Justin and McCauley, Murphy and Franklin, Michael J and + Shenker, Scott and Stoica, Ion}, + booktitle={Proceedings of the 9th USENIX conference on Networked Systems + Design and Implementation}, + pages={14}, + year={2012}, + organization={USENIX Association} +} diff --git a/src/intro.tex b/src/intro.tex @@ -22,32 +22,51 @@ either master or slave, rather than to think of a cluster as a whole with master and slave roles being \emph{dynamically} assigned to a particular physical machine. -This evolution in thinking allows implementing middleware that manages master +This evolution in thinking allows to implement middleware that manages master and slave roles automatically and handles node failures in a generic way. This software provides an API to distribute parallel tasks on the pool of available -nodes and among them. Using this API one can write an application that runs on a -cluster without knowing the exact number of online nodes. The middleware is +nodes and among them. Using this API one can write an application that runs on +a cluster without knowing the exact number of online nodes. The middleware is implemented as a daemon running on each cluster node which acts as an intermediate point of communication for distributed applications and transparently routes application messages between operating system processes running on different cluster nodes. +The rest of the paper is organised as follows. In Section 2 we discuss how +dynamic node role assignment is implemented in state-of-the-art middleware +systems and how hierarchical dependencies between kernels help establish rules +for restarting failed parts of a parallel application. In Section 3 we describe +programming model based on kernels~--- small independent units of work~--- +and an algorithm of handling master node failure. In Section 4 and 5 we +present and discuss results of experiments that show validity of the +proposed approach and conclude the paper in Section 6. + +The research for new fault-tolerance methods is motivated by the fact that the +size of large-scale computing systems (clusters and supercomputers) approaches +critical point, where the number of nodes is so large that probability of all +nodes simultaneously working without a faulure is tends to nought. In other +words, in future large-scale systems it is highly probable that a parallel +application experience node failure throughout its execution, and tolerating +this failure in a transparent way and without checkpointts will increase +performance of future parallel applications. + \section{Related work} Dynamic role assignment is an emerging trend in design of distributed systems~\citep{ostrovsky2015couchbase,divya2013elasticsearch,boyer2012glusterfs,anderson2010couchdb,lakshman2010cassandra}, however, it is still not used in big data job schedulers. For example, in -popular YARN job scheduler~\citep{vavilapalli2013yarn}, which is used by Hadoop -and Spark big data analysis frameworks, master and slave roles are static. -Failure of a slave node is tolerated by restarting a part of a job on a healthy -node, and failure of a master node is tolerated by setting up standby reserved -server~\citep{murthy2011architecture}. Both master servers are coordinated by -Zookeeper service which itself uses dynamic role assignment to ensure its -fault-tolerance~\citep{okorafor2012zookeeper}. So, the whole setup is complicated -due to Hadoop scheduler lacking dynamic roles: if dynamic roles were available, -Zookeeper would be redundant in this setup. Moreover, this setup does not -guarantee continuous operation of the master node because the standby server needs time -to recover current state after a failure. +popular YARN~\citep{vavilapalli2013yarn} and Spark~\citep{zaharia2012resilient} +job schedulers, which is used by Hadoop and Spark big data analysis frameworks, +master and slave roles are static. Failure of a slave node is tolerated by +restarting a part of a job on a healthy node, and failure of a master node is +tolerated by setting up standby reserved server~\citep{murthy2011architecture}. +Both master servers are coordinated by Zookeeper service which itself uses +dynamic role assignment to ensure its +fault-tolerance~\citep{okorafor2012zookeeper}. So, the whole setup is +complicated due to Hadoop scheduler lacking dynamic roles: if dynamic roles +were available, Zookeeper would be redundant in this setup. Moreover, this +setup does not guarantee continuous operation of the master node because the +standby server needs time to recover current state after a failure. The same problem occurs in high-performance computing where the master node of a job scheduler is the single point of failure. diff --git a/src/sections.tex b/src/sections.tex @@ -45,14 +45,7 @@ set of resources allocated for a batch job. The pipeline is likely to deliver greater efficiency for busy HPC clusters compared to executing a separate job for each input file, because the time that each subsequent job spends in the queue is eliminated. A diagram of a pipeline is presented in -fig.~\ref{fig:pipeline}. - -\begin{figure} - \centering - \includegraphics[scale=0.6]{ppl} - \caption{Data processing pipeline of a programme.} - \label{fig:pipeline} -\end{figure} +fig.~\ref{fig:pipeline} and discussed in sec~\ref{sec:results}. Computational model with a pipeline can be seen as \emph{bulk-asynchronous model}, because of the parallel nature of otherwise sequential execution steps. @@ -128,14 +121,15 @@ that constitute distributed system's ability to \emph{fail over}. The key feature that is missing in the current parallel programming and big data processing technologies is a possibility to specify hierarchical -dependencies between parallel tasks. When one has such dependency, it is -trivial to determine which task should be responsible for re-executing a failed -task on a healthy node. To re-execute the root of the hierarchy, a backup root -task is created and executed on a different node. There exists a number of -engines that are capable of executing directed acyclic graphs of tasks in -parallel~\citep{acun2014charmpp,islam2012oozie}, but graphs are not good to -infer master-slave relationship between tasks, because a node in the graph may -have multiple parent nodes. +(parent-child) dependencies between parallel tasks (kernels). When one has such +dependency, it is trivial to determine which kernel should be responsible for +re-executing a failed kernel on a healthy node. To re-execute the kernel on the +top of the hierarchy, its copy is created and executed on a different node +(detailed algorithm is discussed in sec.~\ref{sec:master-node-failure}). There +exists a number of engines that are capable of executing directed acyclic +graphs of kernels in parallel~\citep{acun2014charmpp,islam2012oozie}, but +graphs are not suitable to infer parent-child relationship between kernels, +because a node in the graph may have multiple parent nodes. \subsection{Programming model} @@ -220,6 +214,7 @@ kernels with passive one. Pipelines, in turn, allow implementing passive wait, and call correct kernel methods by analysing their internal state. \subsection{Handling master node failures} +\label{sec:master-node-failure} A possible way of handling a failure of a node where the first kernel is located (a master node) is to replicate this kernel to a backup node, and make @@ -261,6 +256,7 @@ checkpoints. The advantage is that they \end{itemize} \section{Results} +\label{sec:results} Master node fail over technique is evaluated on the example of wave energy spectra processing application. This programme uses NDBC @@ -305,7 +301,16 @@ cores of a node. Processing begins with joining corresponding measurements for each spectrum variables into a tuple, then for each tuple frequency-directional spectrum is reconstructed and its variance is computed. Results are gradually copied back to the machine where the application was executed and when the -processing is complete the programme terminates. +processing is complete the programme terminates. A data processing pipeling +corresponding to the algorithm is presented in fig.~\ref{fig:pipeline}. + +\begin{figure} + \centering + \includegraphics[scale=0.6]{ppl} + \caption{Data processing pipeline of a programme.} + \label{fig:pipeline} +\end{figure} + \begin{table} \centering @@ -379,6 +384,28 @@ arbitrarily chosen healthy subordinate node. \label{fig:benchmark-bigdata} \end{figure} +To measure how much time is lost due to a failure we divide the total execution +time with a failure by the total execution time without the failure but with +the number of nodes minus one. The results for this calculation are obtained +from the same benchmark and are presented in +fig.~\ref{fig:benchmark-bigdata-2}. The difference in performance lies within +20\% margin for the number of nodes less or equal to six. As is expected, the +slowdown is noticable for the small number of nodes, and performance penalty +descreases for large number of nodes, as smaller portion of execution state is +lost. The picture is different for the number of nodes larger than six: +performance ratio fluctuates within 50\% margin. The reason for such behaviour +is that spectrum processing application does not scale beyond six nodes, and +the benchmark does not give realiable results beyond this point. + +\begin{figure} + \centering + \includegraphics{build/ratio.eps} + \caption{Slowdown of the spectrum processing application in the presence of + different types of node failures compared to execution without failures but + with the number of nodes minus one.} + \label{fig:benchmark-bigdata-2} +\end{figure} + \section{Discussion} Described algorithm guarantees to handle one failure per computational step,