diff options
Diffstat (limited to 'report/chapters/4-testing.tex')
| -rw-r--r-- | report/chapters/4-testing.tex | 62 |
1 files changed, 30 insertions, 32 deletions
diff --git a/report/chapters/4-testing.tex b/report/chapters/4-testing.tex index fea4cbc..bf57e73 100644 --- a/report/chapters/4-testing.tex +++ b/report/chapters/4-testing.tex @@ -5,8 +5,8 @@ \section{Unit testing}\label{sec:4-unit-testing} -The \verb+nose+\cite{pylib:nose} testing framework was used to write -unit tests for the functions and classes part of the +The \verb+nose+ \cite{pylib:nose} testing framework was used to write +unit tests for the functions and classes that are part of the \verb+combox.config+, \verb+combox.crypto+, \verb+combox.events+, \verb+combox.file+, \verb+combox.silo+ and \verb+combox._version+ modules. Unit tests were not written for \verb+combox.cbox+, @@ -30,17 +30,17 @@ of inputs. Unit tests greatly helped in testing the compatibility of combox on OS X. Before the \verb+v0.1.0+ release, combox's node directory monitor always assumed that a file's first shard (\verb+shard0+) is always -available; while this assumption did not create any problems on -GNU/Linux, on OS X, this assumption made the node directory monitor to -behave erratically -- this issue (bug \#4) was immediately found when +available. While this assumption did not create any problems on +GNU/Linux, on OS X this assumption made the node directory monitor to +behave erratically. This issue (bug \#4) was immediately found when the unit tests were run for the first time on OS X. Another instance -where unit tests helped was just before the \verb+v0.2.0+ release; -major changes, including the introduction of file locks in the +where unit tests helped was just before the \verb+v0.2.0+ release. +Major changes, including the introduction of file locks in the \verb+ComboxDirMonitor+, were made to the \verb+combox.events+. When the unit tests were run OS X, two tests failed, revealing a difference in behavior of watchdog\cite{pylib:watchdog} on GNU/Linux and OS X on file -creation\footnote{https://git.ricketyspace.net/combox/commit/?id=8c86e7c28738c66c0e04ae7886b44dbcdfc6369exo}; +creation \footnote{https://git.ricketyspace.net/combox/commit/?id=8c86e7c28738c66c0e04ae7886b44dbcdfc6369exo}; without unit tests, there is a high probability that this bug would never have been found by now. @@ -59,8 +59,8 @@ these bugs were found when manually testing combox. \section{Manual testing}\label{sec:4-manual-testing} The unit tests for the \verb+combox.events+ module tested the -correctness of the \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+ -independently; in order to comprehensively test the correctness of +correctness of the \\ \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+ +independently. In order to comprehensively test the correctness of both \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+, it was required to manually test combox running on more than one computer. Several bugs were found and fixed while doing manual @@ -69,9 +69,9 @@ testing. Three different types of setups were used to manually test combox. The first kind of setup has two GNU/Linux machines each using combox to sync files between each other with Dropbox and Google Drive being the -nodes; the second kind of setup has a GNU/Linux machine and a OS X +nodes. The second kind of setup has a GNU/Linux machine and a OS X machine each using combox to sync files between each other with -Dropbox and Google Drive being the nodes; the third kind of setup has +Dropbox and Google Drive being the nodes. The third kind of setup has a GNU/Linux machine and OS X machine each using combox to sync files between each other with Dropbox, Google Drive and a USB stick as nodes. @@ -100,8 +100,8 @@ combox was run on two GNU/Linux machines and a file was alternatively created/modified/renamed/deleted on one of the GNU/Linux machine and it was verified if the respective file was also created/modified/renamed/deleted on the other GNU/Linux machine. One -of the GNU/Linux machine (\verb+lyra)+ was a virtual machine running -Debian GNU/Linux stable (version 8.x); the other GNU/Linux machine +of the GNU/Linux machines, (\verb+lyra)+, was a virtual machine running +Debian GNU/Linux stable (version 8.x). The other GNU/Linux machine (\verb+grus+) was a physical machine running Debian GNU/Linux testing. The node directories to scatter the files' shards were the Dropbox directory and Google Drive directory. The official Dropbox @@ -144,7 +144,7 @@ data store. \verb+.dropbox.cache+ directory on this computer. \end{itemize} - All of the above behavior of the Dropbox client royally broke + All of the above behavior of the Dropbox client broke combox. Commits between \verb+3d714c5+ to \verb+6e1133f+\footnote{https://git.ricketyspace.net/combox/log/?qt=range\&q=3d714c5..6e1133f} fixed combox by making it aware of Dropbox's client behavior. @@ -152,10 +152,8 @@ data store. \subsubsection{Demo} -Demo of combox being used on two GNU/Linux machines can be viewed at -\url{https://ricketyspace.net/combox/combox-2-gnus.webm}. - -\verb+lyra+ (virtual machine) and \verb+grus+ (bare-metal) are the two +A demo of combox being used on two GNU/Linux machines can be viewed at +\url{https://ricketyspace.net/combox/combox-2-gnus.webm}. \verb+lyra+ (virtual machine) and \verb+grus+ (bare-metal) are the two GNU/Linux machines being used for the demo. Description of what happens in the demo follows: @@ -277,7 +275,7 @@ Google Drive directory to Google Drive's data store on GNU/Linux. \subsubsection{Demo} -Demo of combox being used on a GNU/Linux machine and OS X machine can +A demo of combox being used on a GNU/Linux machine and OS X machine can be viewed at \url{https://ricketyspace.net/combox/combox-gnu-osx.webm} \verb+lyra+ is the GNU/Linux (virtual) machine and @@ -359,7 +357,7 @@ files stored in combox directory. \subsubsection{Demo} -Demo of combox being used with a USB stick as the third node can be +A demo of combox being used with a USB stick as the third node can be viewed at \url{https://ricketyspace.net/combox/combox-usb-node-demo.webm} \verb+grus+ is the GNU/Linux machine and \verb+dhcp-129-1-66-1+ is the @@ -442,29 +440,29 @@ Description of what happens in the demo follows: \section{Stress testing} -Large number of files of different sizes were dumped to the combox +A large number of files of different sizes were dumped to the combox directory between an one second interval to see how combox responds to -high load. The file dump size was varied from \verb+424.798190MiB+ (27 -files) to \verb+10800.000000MiB+ (180 files); the average time taken +high load. The file dump size was varied from \verb+424.80MiB+ (27 +files) to \verb+10,800.00MiB+ (180 files). The average time taken to split a file and the total time to process all files were calculated for each dump. Stress testing was first done on \verb+2015-11-08+. In mid November -2015, the \verb+ComboxDirMonitor+ was drastically modified to make it +2015, the \\ \verb+ComboxDirMonitor+ was drastically modified to make it use the file Lock shared by the instances of -\verb+NodeDirMonitor+\footnote{https://git.ricketyspace.net/combox/commit/?id=5aa1ba0c1dcad62931ba27bb66bf115233086d6c}; -the hunch was that this change in \verb+ComboxDirMonitor+ directly +\verb+NodeDirMonitor+\footnote{https://git.ricketyspace.net/combox/commit/?id=5aa1ba0c1dcad62931ba27bb66bf115233086d6c}. +The hypothesis was that this change in \verb+ComboxDirMonitor+ directly affected the performance of combox and therefore the results that were got from stress testing on \verb+2015-11-08+ would no longer be -valid. Stress testing was again done on \verb+2016-01-16+; the results +valid. Stress testing was again done on \verb+2016-01-16+. The results of this stress test are in sections \ref{4-st-424} to -\ref{4-st-10800}, section \ref{4-st-tu} gives information about the +\ref{4-st-10800}. Section \ref{4-st-tu} gives information about the tools used for stress testing, section \ref{4-st-o} contains the observations and comparisons between this stress test and the one done -on \verb+2015-11-08+, lastly section \ref{4-st-if} reveals the issues +on \verb+2015-11-08+, and, lastly section \ref{4-st-if} reveals the issues that were found with combox by virtue of doing the stress tests. -\subsection{flac dump (27 files - 424.798190MiB)}\label{4-st-424} +\subsection{flac dump (27 files - 424.80MiB)}\label{4-st-424} \begin{center} \begin{table}[h] @@ -579,7 +577,7 @@ avg. time to split and encrypt a file & 3423.087539ms\\ \subsection{Tools used}\label{4-st-tu} The \verb+dump+ script\footnote{https://git.ricketyspace.net/combox-paper/plain/dumper/dump} was used to dump files to -the combox directory between one second intervals; a night of Emacs +the combox directory between one second intervals. A night of Emacs Lisp indulgence made it possible to quickly slurp the required data from the combox output and calculate the average time to split and encrypt a file and the total amount of time taken to process the files |