diff options
Diffstat (limited to 'report/chapters/4-testing.tex')
| -rw-r--r-- | report/chapters/4-testing.tex | 668 |
1 files changed, 668 insertions, 0 deletions
diff --git a/report/chapters/4-testing.tex b/report/chapters/4-testing.tex new file mode 100644 index 0000000..e6376f1 --- /dev/null +++ b/report/chapters/4-testing.tex @@ -0,0 +1,668 @@ +\chapter{Testing}\label{ch:5} + +\epigraph{Testing shows the presence, not the absence of + bugs.}{\textit{Dijkstra}\cite{dijkstra69}} + +\section{Unit testing}\label{sec:5-unit-testing} + +The \verb+nose+\cite{pylib:nose} testing framework was used to +write unit tests for the functions and classes part of the +\verb+combox.config+, \verb+combox.crypto+, \verb+combox.events+, +\verb+combox.file+, \verb+combox.silo+ \verb+combox._version+ +modules. Unit tests were not written for \verb+combox.cbox+, +\verb+combox.gui+, \verb+combox.combox.log+ modules. + +Unit tests for combox become reality by pure serendipity. During the +time, when I started working on combox, I was learning to use the +\verb+nose+ library to unit test python code. Since, \verb+combox+ was +being written in python, I started making it a norm to write unit +tests for functions and classes in combox modules. + +As mentioned before, unit tests were not written for some modules +either because it would make no sense to write one (for the +\verb+combox.cbox+ module, for instance, which basically uses +functions and classes defined in other modules to run combox) or it +was not clear how to write unit tests it (the \verb+combox.gui+ +contains just the \verb+ComboxConfigDialog+ a graphical front-end +which uses the configuration function defined in the +\verb+combox.config+ module to complete the combox configuration based +on the user input). + +It must be noted here that pure Test Driven Development (TDD) was not +observed -- most of the time the function/class was written before the +its corresponding test was written. + +\subsection{Benefits} + +While writing unit tests definitely increased the time to write a +particular feature, it enabled me to immediately check if a feature +worked as it should for the given use case or given set of inputs. + +With the benefit of hindsight, unit tests greatly helped in testing +the compatibility of combox on OSX. Before the \verb+v0.1.0+ release, +combox's node directory monitor always assumed that a file's first +shard (\verb+shard0+) is always available; while this assumption did +not create any problems on GNU/Linux, on OS X, this assumption made +the node directory monitor to behave erratically -- this issue (bug +\#4\cite{combox-issue-tracker} was immediately found when the unit +tests were run for the first time on OS X. Another instance where unit +tests helped was just before the \verb+v0.2.0+ release; major changes, +including the introduction of file locks in the +\verb+ComboxDirMonitor+, were made to the \verb+combox.events+. When +the unit tests were run OS X, two tests failed, revealing a difference +in behavior of watchdog\cite{pylib:watchdog} on GNU/Linux and OS X on +file creation\cite{combox-wd-fix}; without unit tests, there is a high +probability that this bug would never have been found by now. + +\subsection{Caveats} + +Unit tests are helpful in testing the correctness of a feature for +\verb+N+ number of use cases but it does not necessarily mean the +written feature correctly behaves for use cases that the author of the +feature did not consider or did not think about while writing the +respective feature. As Dijkstra correctly observed: + +Unit tests failed to reveal bugs \#4, \#5 \#6 \#7 \#5 \#10 +\#11\cite{combox-issue-tracker}; these bugs were found when manually +testing combox. + +\section{Manual testing}\label{sec:5-manual-testing} + +The unit tests for the \verb+combox.events+ module test the +correctness of the \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+ +independently; in order to comprehensively test the correctness of +both \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+, it was +required to manually test combox running on more than one computer. As +you'll see in the following subsections, several bugs were found and +fixed while doing manual testing. + +Three different types of setups were used to test combox. The first +kind of setup has two GNU/Linux machines each using combox to sync +files between each other with Dropbox and Google Drive being the +nodes; the second kind of setup has a GNU/Linux machine and a OS X +machine each using combox to sync files between each other with +Dropbox and Google Drive being the nodes; the third kind of setup has +a GNU/Linux machine and OS X machine each using combox to sync files +between each other with Dropbox, Google Drive and a USB stick as +nodes. + +\subsection{General setup and notes} + +\begin{itemize} +\item On the GNU/Linux machines, the official Dropbox client was used + to sync the Dropbox node directory to Dropbox' + servers. \verb+rclone+\cite{program:rclone} was used to sync the + Google Drive node directory to Google Drive' servers;At the time of + testing, Google Drive did not have client for GNU/Linux. +\item On OS X, the official Dropbox client was used to sync the + Dropbox node directory to Dropbox's servers; the official Google + Drive client was used to sync the Google Drive node directory to + Google Driver' servers. +\item Since combox is extremely event-driven, combox must be started + before the Dropbox and Google Drive clients start syncing their + respective directories (nodes). +\end{itemize} + +\subsection{Testing on two GNU/Linux machines} + +combox was run to two GNU/Linux machines and a file was alternatively +created/modified/renamed/deleted on an of the GNU/Linux machine and it +was verified if the respective file was also +created/modified/renamed/deleted on the other GNU/Linux machine. One +of the GNU/Linux machine (\verb+lyra)+ was a virtual machine running +Debian GNU/Linux stable (version 8.x); the other GNU/Linux machine +(\verb+grus+) was a physical machine running Debian GNU/Linux +testing. The node directories to scatter the files' shards were the +Dropbox directory and Google Drive directory. The official Dropbox +client was used to automatically sync files from the Dropbox directory +to the Dropbox' server; \verb+rclone+\cite{program:rclone} was used to +sync files from Google Drive directory to Google Drive' server. + +\subsubsection{Issues found}\label{ch-5-2gnus-issues} + +\begin{itemize} +\item Some editors, especially on POSIX complaint systems, create + backup version of the file being edited. combox was detecting this + backup file as a ``new file'' and it split it into shards, encrypted + the shards and scattered the shards across the node directories. The + right thing for combox to do was to ignore these backup files and do + nothing about them. This issue was fixed on + \verb+2015-09-29+\cite{combox-issue-tracker}. Now the + \verb+ComboxDirMonitor+, on a ``file created'' or ``file modified'' + event, returns from the \verb+on_created+ or \verb+on_modified+ + callback when it finds that the file is a backup/temporary file. +\item Dropbox client maintains the \verb+.dropbox.cache+ directory + under the root of the Dropbox directory. + + \begin{itemize} + \item When a file (shard) was created on another computer, the + Dropbox client pulls the new file (shard) to this computer into + \verb+.dropbox.cache+ as a temporary file and then moves the new + file (shard) to its respective location with the appropriate name. + \item When a file (shard) was modified on another computer, the + Dropbox client pulls the modified file (shard) to this computer + into the \verb+.dropbox.cache+ as a temporary file; moves the old + version of the file (shard) under the Dropbox directory into the + \verb+.dropbox.cache+; finally moves the updated copy of the file, + stored as a temporary file, into the Dropbox directory to its + respective location with the appropriate name. + \item When a file (shard) was deleted on another computer, the + Dropbox client moves the delete file into the + \verb+.dropbox.cache+ directory on this computer. + \end{itemize} + + All of the above behavior of the Dropbox client epically broke + combox. Commits \verb+3d714c5+ to + \verb+6e1133f+\cite{git:dropbox-fix} fixed combox by making it aware + of Dropbox's client behavior. +\end{itemize} + +\subsubsection{Demo} + +Demo of combox being used on two GNU/Linux machines can be viewed at +\url{https://ricketyspace.net/combox/combox-2-gnus.webm}. + +\verb+lyra+ (virtual machine) and \verb+grus+ (bare-metal) are the two +GNU/Linux machines being used for the demo. + +Description of what happens in the demo follows: + + - (lyra) install combox. + + - (lyra) run combox (test mode). + + - (lyra) create file \verb+walden.pond+ with content ``It must be + beautiful there''. + + - (lyra) sync Google Drive using \verb+rclone+. + + - (grus) sync Google Drive using \verb+rclone+. + + - (grus) git pull latest copy of combox. + + - (grus) install combox + + - (grus) run combox (testing mode). + + - (grus) verify that \verb+walden.pond+ was create on this machine. + + - (grus) append 'Peaceful too.' to \verb+walden.pond+. + + - (grus) sync Google Drive using \verb+rclone+. + + - (lyra) sync Google Drive using \verb+rclone+. + + - (lyra) verify that the latest copy of \verb+walden.pond+ is there + in the combox directory; it should contain 'Peaceful too.' in the + last line. + + - (lyra) append ``I've a dream'' to \verb+walden.pond+. + + - (lyra) sync Google Drive using \verb+rclone+. + + - (grus) sync Google Drive using \verb+rclone+. + + - (grus) verify that the latest copy of \verb+walden.pond+ is there + in the combox directory; it should contain ``I've a dream'' in the + last line. + + - (grus) remove \verb+walden.pond+ from combox directory. + + - (grus) sync Google Drive using \verb+rclone+. + + - (lyra) sync Google Drive using \verb+rclone+. + + - (lyra) verify that \verb+walden.pond+ is removed from the combox + directory. + + - (grus) open dropbox and Google drive accounts from the web browser. + + - (lyra) create file \verb+manufacturing.consent.+ with content ``Chomsky stuff?''. + + - (lyra) sync Google Drive using \verb+rclone+. + + - (grus) sync Google Drive using \verb+rclone+. + + - (grus) verify that \verb+manufacturing.consent+ was created in the + combox directory. + + - (grus) verify that the shards of \verb+manufacturing.consent+ were + created on Dropbox and Google Drive through the web browser. + +\subsection{Testing on a GNU/Linux and an OS X machine} + +combox was run on a GNU/Linux machine and an OS X machine and a file +was alternatively created/modified/renamed/deleted on one of the +machine and it was verified if the respective file was also +created/modified/renamed/deleted on the other machine. The GNU/Linux +machine was a virtual machine (\verb+lyra+) running Debian GNU/Linux +stable; the OS X machine was on Mavericks (10.9) during the initial +stage of testing, later it was upgraded to Yosemite (10.10). The node +directories to scatter files' shards were the Dropbox directory and +the Google Drive directory. The official Dropbox client was used to +automatically sync files from the Dropbox directory to the Dropbox' +server on both the GNU/Linux machine and the OS X machine; the +official Google Drive client was used to automatically sync files from +the Google Drive directory to Google Drive' server on OS X and +\verb+rclone+\cite{program:rclone} was used to sync files from the +Google Drive directory to Google Drive's server on GNU/Linux. + +\subsubsection{Issues found} + +\begin{itemize} +\item When a file was modified on another computer, on this computer + combox assumed that first shard (shard0) will be updated first and + also counted on the existence of the first shard (shard0). It was + observed that the order in which the shards were updated were + unpredictable on this computer and if the first shard (shard0) was + stored in the Dropbox directory, it will momentarily disappear + before the most updated shard becomes available in the Dropbox + directory; this broke combox. This issue was fixed on + 2015-08-25\cite{git:bug-four-fix}. This issue is not got to do with + the nature of the setup but it is related to the Dropbox's behavior + elaborated in section \ref{ch-5-2gnus-issues}. +\item The official Google Drive client when it pulls an updated + version of the file from Google Drive' server, instead directly + updating the respective file on the computer, it deletes the older + version of the file and creates the latest version of the file at + the respective location in the Google Drive directory; this behavior + of the Google Drive confused and broke combox. This issue was fixed + 2015-09-06 by making combox under the official Google Client's + behavior\cite{git:bug-googledc-fix}. +\item When a non-empty directory was move/renamed on another computer, + the old directory was not getting properly deleted on this computer; + this was happening because the files under the directory being + renamed were not deleted when it was time for \verb+NodeDirMonitor+ + to \verb+rmdir+ the old directory. This issue again is not specific + to the nature of the setup but was found while testing combox on + this setup. This issue was fixed on + 2015-09-12\cite{git:bug-six-fix}. +\item It was found that \verb+combox.file.rm_path+ function failed + when it was given a non-existent path to remove; this issue was + fixed on 2015-09-12\cite{git:bug-seven-fix}. +\end{itemize} + +\subsubsection{Demo} + +Demo of combox being used on a GNU/Linux machine and OS X machine can +be viewed at \url{https://ricketyspace.net/combox/combox-gnu-osx.webm} + +\verb+lyra+ is the GNU/Linux (virtual) machine and +\verb+dhcp-129-1-66-1+ is the OS X machine that is being used for the +demo. The OS X machine is accessed through VNC\cite{article:vnc}. + +Description of what happens in the demo follows: + + - (\verb+lyra+) create file \verb+cat.stevens+ with content ``peace train''. + + - (\verb+lyra+) sync Google Drive using \verb+rclone+. + + - (\verb+dhcp-129-1-66-1+) verify that file \verb+cat.stevens+ is + created with content ``peace train''. + + - (\verb+dhcp-129-1-66-1+) append string ``moonshadow'' to file + \verb+cat.stevens+. + + - (\verb+lyra+) sync Google Drive using \verb+rclone+. + + - (\verb+lyra+) verify that the file \verb+cat.stevens+ was updated + (modified); last line must have the string ``moonshadow''. + + - (\verb+lyra+) append string ``father and son'' to the file + \verb+cat.stevens+. + + - (\verb+lyra+) sync Google Drive using \verb+rclone+. + + - (\verb+dhcp-129-1-66-1+) verify that the file \verb+cat.stevens+ + was updated (modified); last line must have the string ``father and + son''. + + - (\verb+dhcp-129-1-66-1+) rename file \verb+cat.stevens+ to + \verb+yusuf.islam+ + + - (\verb+lyra+) sync Google Drive using \verb+rclone+. + + - (\verb+lyra+) verify that the file \verb+cat.stevens+ was renamed + to \verb+yusuf.islam+. + +\subsection{Testing with a USB stick as a node} + +combox was run on a GNU/Linux machine and an OS X machine and a file +was alternatively created/modified/deleted on one of the machine and +it was verified if the repsective file was also +create/modified/deleted on the other machine. The GNU/Linux machine +was a physical machine (\verb+grus+) running Debian GNU/Linux stable; +The OS X machine was on Mavericks (10.9). The node directories to +scatter files' shards were the Dropbox directory, Google Drive +directory and the USB stick (\verb+ZAPHOD+, FAT filesystem). The +official Dropbox client was used to automatically sync files from +Dropbox directory to Dropbox' server on both the GNU/Linux machine and +OS X machine; the official Google Drive client was used to +automatically sync files from the Google Drive directory to Google +Drive' server on OS X and \verb+rclone+\cite{program:rclone} was used +to sync files from the Google Drive directory to Google Drive's server +on GNU/Linux; the same USB stick (\verb+ZAPHOD+) was used on bothe +GNU/Linux and Dropbox to store the third shard (shard2) of a file. + +\subsubsection{Caveats} + +\begin{itemize} +\item When a removable USB disk is used as a node, combox must be + turned off before ejecting/unmounting the USB disk; combox does not + expect a node directory to disappear when it is running, if the USB + disk is removed when combox is running, then combox goes to a + undefined state. + +\item When a file modified on machine A is synced to machine B, combox + must be turned on first before turning on Dropbox and Google Drive + clients and the shard in the USB disk needs to be ``touched'' for + combox to detect that the file was modified on the remote computer + and update the file locally on this machine. + +\item File rename/move does not work. To make it work, core + functionality of combox must be re-written. +\end{itemize} + +\subsubsection{Demo} + +Demo of combox being used with a USB stick as the third node can be +view at \url{https://ricketyspace.net/combox/combox-usb-node-demo.webm} + +\verb+grus+ is the GNU/Linux machine and \verb+dhcp-129-1-66-1+ is the +OS X machine that is being used for the demo. \verb+ZAPHOD+ is the +FAT32 USB stick used as the third node. + +Description of what happens in the demo follows: + + - (\verb+grus+) start combox. + + - (\verb+grus+) create a file called \verb+simon.and.garfunkel+ with + content ``the boxer''. + + - (\verb+grus+) sync Google Drive using \verb+rclone+. + + - (\verb+grus+) stop combox. + + - (\verb+grus+) unmount USB stick (\verb+ZAPHOD+) from \verb+grus+. + + - (\verb+dhcp-129-1-66-1+) mount USB stick (\verb+ZAPHOD+) to + (\verb+dhcp-129-1-66-1+). + + - (\verb+dhcp-129-1-66-1+) start Dropbox client. + + - (\verb+dhcp-129-1-66-1+) start Google Drive client. + + - (\verb+dhcp-129-1-66-1+) start combox. + + - (\verb+dhcp-129-1-66-1+) verify that the file + \verb+simon.and.garfunkel+ with content ``the boxer'' was created. + + - (\verb+dhcp-129-1-66-1+) append string ``mrs. robinson'' to file + \verb+simon.and.garfunkel+. + + - (\verb+dhcp-129-1-66-1+) stop combox. + + - (\verb+dhcp-129-1-66-1+) stop Google Drive client. + + - (\verb+dhcp-129-1-66-1+) stop Dropbox client. + + - (\verb+dhcp-129-1-66-1+) unmount the USB stick (\verb+ZAPHOD+) + from (\verb+dhcp-129-1-66-1+). + + - (\verb+grus+) mount the USB stick (\verb+ZAPHOD+) to + (\verb+grus+). + + - (\verb+grus+) start combox. + + - (\verb+grus+) start Dropbox client. + + - (\verb+grus+) sync Google Drive using \verb+rclone+. + + - (\verb+grus+) touch \verb+simon.and.garfunkel.shard2+ in the USB + stick (\verb+ZAPHOD+). + + - (\verb+grus+) verify that the file \verb+simon.and.garfunkel+ is + updated; the last line must contain the string ``mrs. robinson''. + + - (\verb+grus+) remove the file \verb+simon.and.garfunkel+. + + - (\verb+grus+) sync Google Drive using \verb+rclone+. + + - (\verb+grus+) unmount the USB stick (\verb+ZAPHOD+) from + (\verb+grus+). + + - (\verb+grus+) stop Dropbox client. + + - (\verb+dhcp-129-1-66-1+) mount the USB stick (\verb+ZAPHOD+) to + (\verb+dhcp-129-1-66-1+). + + - (\verb+dhcp-129-1-66-1+) start Google Drive client. + + - (\verb+dhcp-129-1-66-1+) start Dropbox client. + + - (\verb+dhcp-129-1-66-1+) start combox. + + - (\verb+dhcp-129-1-66-1+) verify that the file + \verb+simon.and.garfunkel+ was deleted. + + +\section{Stress testing} + +Large number of files of different sizes were dumped to the combox +directory between an one second interval to see how combox responds to +high load. The file dump size was varied from \verb+424.798190MiB+ (27 +files) to \verb+10800.000000MiB+ (180 files); the average time taken +to split a file and the total time to process all files were +calculated for each dump. + +Stress testing was first done on \verb+2015-11-08+. In mid November +the \verb+ComboxDirMonitor+ was drastically modified to make it use +the file Lock shared the instances of +\verb+NodeDirMonitor+\cite{git:bug-eleven-fix}; my hunch was that this +change in \verb+ComboxDirMonitor+ directly affected the performance of +combox and therefore the results that were got from stress testing on +\verb+2015-11-08+ would no longer be valid. Stress testing was again +done on \verb+2016-01-16+; the results of this stress test are in +sections \ref{5-st-424} to \ref{5-st-10800}, section \ref{5-st-tu} +gives information about the tools used for stress testing, section +\ref{5-st-o} contains the observations and comparisons between this +stress test and the one done on \verb+2015-11-08+, lastly section +\ref{5-st-if} reveals the issues that were found with combox by virtue +of doing the stress tests. + +\subsection{flac dump (27 files - 424.798190MiB)}\label{5-st-424} + +\begin{center} +\begin{tabular}{ll} +field & value\\ +\hline +delay between a file dump & 1s\\ +start time of processing & 11:00:54\\ +end time of processing & 11:01:38\\ +total time taken to process all files & 00:00:44\\ +no. of files & 27\\ +total size of all files & 445433187.000000 bytes (424.798190MiB)\\ +avg. file size & 16497525.000000 bytes (15.733266MiB)\\ +avg. time to split and encrypt a file & 352.583370 ms\\ +\end{tabular} +\end{center} + +\subsubsection{Differences from previous stress test (2015-11-08)} + +\begin{itemize} +\item Total time to process all files was faster by 1min3secs. +\item Average time to split and encrypt a file reduced by + 28.337963000000002ms. +\end{itemize} + +\subsection{20MiB - 90MiB dump (27 files - 1620.000000MiB)}\label{5-st-1620} + +\begin{center} +\begin{tabular}{ll} +field & value\\ +\hline +delay between a file dump & 1s\\ +start time of processing & 12:26:45\\ +end time of processing & 12:29:07\\ +total time taken to process all files & 00:02:22\\ +no. of files & 27\\ +total size of all files & 1698693120.000000 bytes (1620.000000MiB)\\ +avg. file size & 62914560.000000 bytes (60.000000MiB)\\ +avg. time to split and encrypt a file & 2670.596556ms\\ +\end{tabular} +\end{center} + +\subsubsection{Differences from previous stress test (2015-11-08)} + +\begin{itemize} +\item Total time to process all files was slower by 4secs. +\item Average time to split and encrypt a file reduced by + 25.52536999999984ms. +\end{itemize} + +\subsection{20MiB - 90MiB dump (99 files - 5940.000000MiB)}\label{5-st-5940} + +\begin{center} +\begin{tabular}{ll} +field & value\\ +\hline +delay between a file dump & 1s\\ +start time of processing & 13:10:16\\ +end time of processing & 13:19:26\\ +total time taken to process all files & 00:09:10\\ +no. of files & 99\\ +total size of all files & 6228541440.000000 bytes (5940.000000MiB)\\ +avg. file size & 62914560.000000 bytes (60.000000MiB)\\ +avg. time to split and encrypt a file & 2979.647586ms\\ +\end{tabular} +\end{center} + +\subsubsection{Differences from previous stress test (2015-11-08)} + +\begin{itemize} +\item Total time to process all files was faster by 59secs. +\item Average time to split and encrypt a file increased by + 206.20906100000002ms. +\end{itemize} + +\subsection{20MiB - 90MiB dump (180 files - 10800.000000MiB)}\label{5-st-10800} + +\begin{center} +\begin{tabular}{ll} +field & value\\ +\hline +delay between a file dump & 1s\\ +start time of processing & 13:42:06\\ +end time of processing & 14:00:10\\ +total time taken to process all files & 00:18:04\\ +no. of files & 180\\ +total size of all files & 11324620800.000000 bytes (10800.000000MiB)\\ +avg. file size & 62914560.000000 bytes (60.000000MiB)\\ +avg. time to split and encrypt a file & 3423.087539ms\\ +\end{tabular} +\end{center} + +\subsubsection{Differences from previous stress test (2015-11-08)} + +\begin{itemize} +\item Total time to process all files was slower by 1min2secs +\item Average time to split and encrypt a file increased by + 399.87623299999996ms. +\end{itemize} + +\subsection{Tools used}\label{5-st-tu} + +The \verb+dump+ script\cite{program:dump} was used to dump files to +the combox directory between one second intervals; a night of Emacs +Lisp indulgence made it possible to quickly slurp the required data +from the combox output and calculate the average time to split and +encrypt a file and the total amount of time taken to process the files +for a given dump\cite{program:dumps.el}; lastly \verb+org-mode+ was +used to document all data gathered during stress +testing\cite{doc:benchmarks.org}. + +\subsection{Observations}\label{5-st-o} + +\begin{figure}[h] +\centering +\input{graphs/tot-time.tex} +\caption{time to process all files} +\label{fig:5-st-tt} +\end{figure} + +\begin{figure}[h] +\centering +\input{graphs/avg-time-sae.tex} +\caption{avg. time to split and encrypt} +\label{fig:5-st-atsae} +\end{figure} + + +\begin{itemize} +\item Figure \ref{fig:5-st-tt} shows the time it takes combox to + process files for a given file dump\footnote{A ``file dump'' here + means a bunch of files copied to the combox directory between 1 + sec intervals.}. As can be observed from the graph, the total time + taken to process all the files tends almost linearly increase with + the increase in the size of the file dump\footnote{The ``size of the + file dump'' is the total size of all files in a given file dump.}. +\item Figure \ref{fig:5-st-atsae} show the average time it takes + combox to split and encrypt a file for a given file dump. There is a + steep increase in the average time from the \verb+424.798190MiB+ + dump and the \verb+1620.000000MiB+ dump, after which the average + time to split and encrypt a file seems to almost linearly increase; + The main reason for this is that the average file size for dumps + from \verb+1620.000000MiB+ to \verb+10800.000000MiB+ are the same. +\end{itemize} + +\begin{figure}[h] +\centering +\input{graphs/tot-time-diff.tex} +\caption{time to process all files - difference between 2015 and 2016} +\label{fig:5-st-tt-diff} +\end{figure} + +\begin{figure}[h] +\centering +\input{graphs/avg-time-sae-diff.tex} +\caption{avg. time to split and encrypt - difference between 2015 and 2016} +\label{fig:5-st-atsae-diff} +\end{figure} + +\begin{itemize} +\item Figure \ref{fig:5-st-tt-diff} shows the graphs for the total + amount of time taken to process all files for a given file dump in + the \verb+2016-01-16+ and \verb+2015-11-8+ stress test. The amount + of time needed to process all fills seems to be reduced for the + \verb+5940.000000MiB+ file dump when compared to the \verb+2015+ + stress test results and it seems to be slightly higher for the + \verb+10800.000000MiB+ file dump when compared to the \verb+2015+ + stress test. +\item Similarly, figure \ref{fig:5-st-atsae-diff} shows the graphs for + the average time to split and encrypt for a given file dump in the + \verb+2016-01-16+ and the \verb+2015-11-8+ stress test. The average + time taken seems to able almost the same for the + \verb+424.798190MiB+ and the \verb+1620.000000+ dump, but for the + \verb+5940.000000MiB+ and the \verb+10800.000000MiB+ dump the + average time taken seems to higher for the \verb+2016+ stress test + when compared to the \verb+2015+ stress test. +\end{itemize} + +\subsection{Issues found}\label{5-st-if} + +\begin{itemize} +\item Initially when combox was stress tested with huge files, combox + would get overwhelmed leading to the computer running out of memory + and the load average sometimes peaking at \verb+8+. At first, it was + assumed that there was a bug in combox which caused this to happen, + but later it was found that \verb+watchdog+\cite{pylib:watchdog} was + generating a large number ``file modified'' events when a huge file + (\verb+~500MiB+ was modified). To prevent \verb+watchdog+ from + generating a large number ``file modified'' events for a single + modification of a huge file, a delay proportional to the size of the + file was created in the \verb+on_modified+ callback methods in both + \verb+ComboxDirMonitor+ and + \verb+NodeDirMonitor+\cite{git:bug-ten-fix}, this fixed the + issue. Also, this it might be useful to note here that this was + ``the'' hardest issue I dealt with in working on combox. +\end{itemize}
\ No newline at end of file |