From 620f361d0778a48c25727c205109156bbc8a8fe0 Mon Sep 17 00:00:00 2001 From: Siddharth Ravikumar Date: Sat, 23 Jan 2016 10:19:47 -0500 Subject: 5-ucases-benchmarks.tex -> 5-testing.tex This file is under report/chapters. --- report/chapters/5-testing.tex | 96 ++++++++++++++++++++++++++++++++ report/chapters/5-ucases-benchmarks.tex | 96 -------------------------------- report/combox-report.pdf | Bin 119399 -> 119399 bytes report/combox.tex | 2 +- 4 files changed, 97 insertions(+), 97 deletions(-) create mode 100644 report/chapters/5-testing.tex delete mode 100644 report/chapters/5-ucases-benchmarks.tex diff --git a/report/chapters/5-testing.tex b/report/chapters/5-testing.tex new file mode 100644 index 0000000..08421a3 --- /dev/null +++ b/report/chapters/5-testing.tex @@ -0,0 +1,96 @@ +\chapter{Testing} + +\section{Unit testing} + +The \verb+nose+\cite{pylib:nose} testing framework was used to +write unit tests for the functions and classes part of the +\verb+combox.config+, \verb+combox.crypto+, \verb+combox.events+, +\verb+combox.file+, \verb+combox.silo+ \verb+combox._version+ +modules. Unit tests were not written for \verb+combox.cbox+, +\verb+combox.gui+, \verb+combox.combox.log+ modules. + +Unit tests for combox become reality by pure serendipity. During the +time, when I started working on combox, I was learning to use the +\verb+nose+ library to unit test python code. Since, \verb+combox+ was +being written in python, I started making it a norm to write unit +tests for functions and classes in combox modules. + +As mentioned before, unit tests were not written for some modules +either because it would make no sense to write one (for the +\verb+combox.cbox+ module, for instance, which basically uses +functions and classes defined in other modules to run combox) or it +was not clear how to write unit tests it (the \verb+combox.gui+ +contains just the \verb+ComboxConfigDialog+ a graphical front-end +which uses the configuration function defined in the +\verb+combox.config+ module to complete the combox configuration based +on the user input). + +It must be noted here that pure Test Driven Development (TDD) was not +observed -- most of the time the function/class was written before the +its corresponding test was written. + +\subsection{Benefits} + +While writing unit tests definitely increased the time to write a +particular feature, it enabled me to immediately check if a feature +worked as it should for the given use case or given set of inputs. + +With the benefit of hindsight, unit tests greatly helped in testing +the compatibility of combox on OSX. Before the \verb+v0.1.0+ release, +combox's node directory monitor always assumed that a file's first +shard (\verb+shard0+) is always available; while this assumption did +not create any problems on GNU/Linux, on OS X, this assumption made +the node directory monitor to behave erraticly -- this issue (bug +\#4\cite{combox-issue-tracker} was immediately found when the unit +tests were run for the first time on OS X. Another instance where unit +tests helped was just before the \verb+v0.2.0+ release; major changes, +including the introduction of file locks in the +\verb+ComboxDirMonitor+, were made to the \verb+combox.events+. When +the unit tests were run OS X, two tests failed, revealing a difference +in behaviour of watchdog\cite{pylib:watchdog} on GNU/Linux and OS X on +file creation\cite{combox-wd-fix}; without unit tests, there is a high +probability that this bug would never have been found by now. + +\subsection{Caveats} + +Unit tests are helpful in testing the correctness of a feature for +\verb+N+ number of use cases but it does not necessarily mean the +written feature correctly behaves for use cases that the author of the +feature did not consider or did not think about while writing the +respective feature. As Dijkstra correctly observed\cite{dijkstra69}: + +\begin{quote} +Testing shows the presence, not the absence of bugs +\end{quote} + +Unit tests failed reveal bugs \#4, \#5 \#6 \#7 \#5 \#10 +\#11\cite{combox-issue-tracker}; these bugs were found when manually +testing combox. + +\section{Manual testing} + +\section{Stress testing} + +\begin{figure}[h] +\centering +\input{graphs/tot-time.tex} +\caption{time to process all files} +\end{figure} + +\begin{figure}[h] +\centering +\input{graphs/avg-time-sae.tex} +\caption{avg. time to split and encrypt} +\end{figure} + +\begin{figure}[h] +\centering +\input{graphs/tot-time-diff.tex} +\caption{time to process all files - difference between 2015 and 2016} +\end{figure} + +\begin{figure}[h] +\centering +\input{graphs/avg-time-sae-diff.tex} +\caption{avg. time to split and encrypt - difference between 2015 and 2016} +\end{figure} \ No newline at end of file diff --git a/report/chapters/5-ucases-benchmarks.tex b/report/chapters/5-ucases-benchmarks.tex deleted file mode 100644 index 08421a3..0000000 --- a/report/chapters/5-ucases-benchmarks.tex +++ /dev/null @@ -1,96 +0,0 @@ -\chapter{Testing} - -\section{Unit testing} - -The \verb+nose+\cite{pylib:nose} testing framework was used to -write unit tests for the functions and classes part of the -\verb+combox.config+, \verb+combox.crypto+, \verb+combox.events+, -\verb+combox.file+, \verb+combox.silo+ \verb+combox._version+ -modules. Unit tests were not written for \verb+combox.cbox+, -\verb+combox.gui+, \verb+combox.combox.log+ modules. - -Unit tests for combox become reality by pure serendipity. During the -time, when I started working on combox, I was learning to use the -\verb+nose+ library to unit test python code. Since, \verb+combox+ was -being written in python, I started making it a norm to write unit -tests for functions and classes in combox modules. - -As mentioned before, unit tests were not written for some modules -either because it would make no sense to write one (for the -\verb+combox.cbox+ module, for instance, which basically uses -functions and classes defined in other modules to run combox) or it -was not clear how to write unit tests it (the \verb+combox.gui+ -contains just the \verb+ComboxConfigDialog+ a graphical front-end -which uses the configuration function defined in the -\verb+combox.config+ module to complete the combox configuration based -on the user input). - -It must be noted here that pure Test Driven Development (TDD) was not -observed -- most of the time the function/class was written before the -its corresponding test was written. - -\subsection{Benefits} - -While writing unit tests definitely increased the time to write a -particular feature, it enabled me to immediately check if a feature -worked as it should for the given use case or given set of inputs. - -With the benefit of hindsight, unit tests greatly helped in testing -the compatibility of combox on OSX. Before the \verb+v0.1.0+ release, -combox's node directory monitor always assumed that a file's first -shard (\verb+shard0+) is always available; while this assumption did -not create any problems on GNU/Linux, on OS X, this assumption made -the node directory monitor to behave erraticly -- this issue (bug -\#4\cite{combox-issue-tracker} was immediately found when the unit -tests were run for the first time on OS X. Another instance where unit -tests helped was just before the \verb+v0.2.0+ release; major changes, -including the introduction of file locks in the -\verb+ComboxDirMonitor+, were made to the \verb+combox.events+. When -the unit tests were run OS X, two tests failed, revealing a difference -in behaviour of watchdog\cite{pylib:watchdog} on GNU/Linux and OS X on -file creation\cite{combox-wd-fix}; without unit tests, there is a high -probability that this bug would never have been found by now. - -\subsection{Caveats} - -Unit tests are helpful in testing the correctness of a feature for -\verb+N+ number of use cases but it does not necessarily mean the -written feature correctly behaves for use cases that the author of the -feature did not consider or did not think about while writing the -respective feature. As Dijkstra correctly observed\cite{dijkstra69}: - -\begin{quote} -Testing shows the presence, not the absence of bugs -\end{quote} - -Unit tests failed reveal bugs \#4, \#5 \#6 \#7 \#5 \#10 -\#11\cite{combox-issue-tracker}; these bugs were found when manually -testing combox. - -\section{Manual testing} - -\section{Stress testing} - -\begin{figure}[h] -\centering -\input{graphs/tot-time.tex} -\caption{time to process all files} -\end{figure} - -\begin{figure}[h] -\centering -\input{graphs/avg-time-sae.tex} -\caption{avg. time to split and encrypt} -\end{figure} - -\begin{figure}[h] -\centering -\input{graphs/tot-time-diff.tex} -\caption{time to process all files - difference between 2015 and 2016} -\end{figure} - -\begin{figure}[h] -\centering -\input{graphs/avg-time-sae-diff.tex} -\caption{avg. time to split and encrypt - difference between 2015 and 2016} -\end{figure} \ No newline at end of file diff --git a/report/combox-report.pdf b/report/combox-report.pdf index 23cd75d..a3a2b68 100644 Binary files a/report/combox-report.pdf and b/report/combox-report.pdf differ diff --git a/report/combox.tex b/report/combox.tex index a787815..316c65c 100644 --- a/report/combox.tex +++ b/report/combox.tex @@ -353,7 +353,7 @@ \chapter{Architecture and Design} %% 5 -\include{chapters/5-ucases-benchmarks} +\include{chapters/5-testing} %% 6 \chapter{Conclusion and Future Work} -- cgit v1.2.3