diff options
Diffstat (limited to 'report/chapters/5-testing.tex')
| -rw-r--r-- | report/chapters/5-testing.tex | 668 |
1 files changed, 0 insertions, 668 deletions
diff --git a/report/chapters/5-testing.tex b/report/chapters/5-testing.tex deleted file mode 100644 index e6376f1..0000000 --- a/report/chapters/5-testing.tex +++ /dev/null @@ -1,668 +0,0 @@ -\chapter{Testing}\label{ch:5} - -\epigraph{Testing shows the presence, not the absence of - bugs.}{\textit{Dijkstra}\cite{dijkstra69}} - -\section{Unit testing}\label{sec:5-unit-testing} - -The \verb+nose+\cite{pylib:nose} testing framework was used to -write unit tests for the functions and classes part of the -\verb+combox.config+, \verb+combox.crypto+, \verb+combox.events+, -\verb+combox.file+, \verb+combox.silo+ \verb+combox._version+ -modules. Unit tests were not written for \verb+combox.cbox+, -\verb+combox.gui+, \verb+combox.combox.log+ modules. - -Unit tests for combox become reality by pure serendipity. During the -time, when I started working on combox, I was learning to use the -\verb+nose+ library to unit test python code. Since, \verb+combox+ was -being written in python, I started making it a norm to write unit -tests for functions and classes in combox modules. - -As mentioned before, unit tests were not written for some modules -either because it would make no sense to write one (for the -\verb+combox.cbox+ module, for instance, which basically uses -functions and classes defined in other modules to run combox) or it -was not clear how to write unit tests it (the \verb+combox.gui+ -contains just the \verb+ComboxConfigDialog+ a graphical front-end -which uses the configuration function defined in the -\verb+combox.config+ module to complete the combox configuration based -on the user input). - -It must be noted here that pure Test Driven Development (TDD) was not -observed -- most of the time the function/class was written before the -its corresponding test was written. - -\subsection{Benefits} - -While writing unit tests definitely increased the time to write a -particular feature, it enabled me to immediately check if a feature -worked as it should for the given use case or given set of inputs. - -With the benefit of hindsight, unit tests greatly helped in testing -the compatibility of combox on OSX. Before the \verb+v0.1.0+ release, -combox's node directory monitor always assumed that a file's first -shard (\verb+shard0+) is always available; while this assumption did -not create any problems on GNU/Linux, on OS X, this assumption made -the node directory monitor to behave erratically -- this issue (bug -\#4\cite{combox-issue-tracker} was immediately found when the unit -tests were run for the first time on OS X. Another instance where unit -tests helped was just before the \verb+v0.2.0+ release; major changes, -including the introduction of file locks in the -\verb+ComboxDirMonitor+, were made to the \verb+combox.events+. When -the unit tests were run OS X, two tests failed, revealing a difference -in behavior of watchdog\cite{pylib:watchdog} on GNU/Linux and OS X on -file creation\cite{combox-wd-fix}; without unit tests, there is a high -probability that this bug would never have been found by now. - -\subsection{Caveats} - -Unit tests are helpful in testing the correctness of a feature for -\verb+N+ number of use cases but it does not necessarily mean the -written feature correctly behaves for use cases that the author of the -feature did not consider or did not think about while writing the -respective feature. As Dijkstra correctly observed: - -Unit tests failed to reveal bugs \#4, \#5 \#6 \#7 \#5 \#10 -\#11\cite{combox-issue-tracker}; these bugs were found when manually -testing combox. - -\section{Manual testing}\label{sec:5-manual-testing} - -The unit tests for the \verb+combox.events+ module test the -correctness of the \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+ -independently; in order to comprehensively test the correctness of -both \verb+ComboxDirMonitor+ and \verb+NodeDirMonitor+, it was -required to manually test combox running on more than one computer. As -you'll see in the following subsections, several bugs were found and -fixed while doing manual testing. - -Three different types of setups were used to test combox. The first -kind of setup has two GNU/Linux machines each using combox to sync -files between each other with Dropbox and Google Drive being the -nodes; the second kind of setup has a GNU/Linux machine and a OS X -machine each using combox to sync files between each other with -Dropbox and Google Drive being the nodes; the third kind of setup has -a GNU/Linux machine and OS X machine each using combox to sync files -between each other with Dropbox, Google Drive and a USB stick as -nodes. - -\subsection{General setup and notes} - -\begin{itemize} -\item On the GNU/Linux machines, the official Dropbox client was used - to sync the Dropbox node directory to Dropbox' - servers. \verb+rclone+\cite{program:rclone} was used to sync the - Google Drive node directory to Google Drive' servers;At the time of - testing, Google Drive did not have client for GNU/Linux. -\item On OS X, the official Dropbox client was used to sync the - Dropbox node directory to Dropbox's servers; the official Google - Drive client was used to sync the Google Drive node directory to - Google Driver' servers. -\item Since combox is extremely event-driven, combox must be started - before the Dropbox and Google Drive clients start syncing their - respective directories (nodes). -\end{itemize} - -\subsection{Testing on two GNU/Linux machines} - -combox was run to two GNU/Linux machines and a file was alternatively -created/modified/renamed/deleted on an of the GNU/Linux machine and it -was verified if the respective file was also -created/modified/renamed/deleted on the other GNU/Linux machine. One -of the GNU/Linux machine (\verb+lyra)+ was a virtual machine running -Debian GNU/Linux stable (version 8.x); the other GNU/Linux machine -(\verb+grus+) was a physical machine running Debian GNU/Linux -testing. The node directories to scatter the files' shards were the -Dropbox directory and Google Drive directory. The official Dropbox -client was used to automatically sync files from the Dropbox directory -to the Dropbox' server; \verb+rclone+\cite{program:rclone} was used to -sync files from Google Drive directory to Google Drive' server. - -\subsubsection{Issues found}\label{ch-5-2gnus-issues} - -\begin{itemize} -\item Some editors, especially on POSIX complaint systems, create - backup version of the file being edited. combox was detecting this - backup file as a ``new file'' and it split it into shards, encrypted - the shards and scattered the shards across the node directories. The - right thing for combox to do was to ignore these backup files and do - nothing about them. This issue was fixed on - \verb+2015-09-29+\cite{combox-issue-tracker}. Now the - \verb+ComboxDirMonitor+, on a ``file created'' or ``file modified'' - event, returns from the \verb+on_created+ or \verb+on_modified+ - callback when it finds that the file is a backup/temporary file. -\item Dropbox client maintains the \verb+.dropbox.cache+ directory - under the root of the Dropbox directory. - - \begin{itemize} - \item When a file (shard) was created on another computer, the - Dropbox client pulls the new file (shard) to this computer into - \verb+.dropbox.cache+ as a temporary file and then moves the new - file (shard) to its respective location with the appropriate name. - \item When a file (shard) was modified on another computer, the - Dropbox client pulls the modified file (shard) to this computer - into the \verb+.dropbox.cache+ as a temporary file; moves the old - version of the file (shard) under the Dropbox directory into the - \verb+.dropbox.cache+; finally moves the updated copy of the file, - stored as a temporary file, into the Dropbox directory to its - respective location with the appropriate name. - \item When a file (shard) was deleted on another computer, the - Dropbox client moves the delete file into the - \verb+.dropbox.cache+ directory on this computer. - \end{itemize} - - All of the above behavior of the Dropbox client epically broke - combox. Commits \verb+3d714c5+ to - \verb+6e1133f+\cite{git:dropbox-fix} fixed combox by making it aware - of Dropbox's client behavior. -\end{itemize} - -\subsubsection{Demo} - -Demo of combox being used on two GNU/Linux machines can be viewed at -\url{https://ricketyspace.net/combox/combox-2-gnus.webm}. - -\verb+lyra+ (virtual machine) and \verb+grus+ (bare-metal) are the two -GNU/Linux machines being used for the demo. - -Description of what happens in the demo follows: - - - (lyra) install combox. - - - (lyra) run combox (test mode). - - - (lyra) create file \verb+walden.pond+ with content ``It must be - beautiful there''. - - - (lyra) sync Google Drive using \verb+rclone+. - - - (grus) sync Google Drive using \verb+rclone+. - - - (grus) git pull latest copy of combox. - - - (grus) install combox - - - (grus) run combox (testing mode). - - - (grus) verify that \verb+walden.pond+ was create on this machine. - - - (grus) append 'Peaceful too.' to \verb+walden.pond+. - - - (grus) sync Google Drive using \verb+rclone+. - - - (lyra) sync Google Drive using \verb+rclone+. - - - (lyra) verify that the latest copy of \verb+walden.pond+ is there - in the combox directory; it should contain 'Peaceful too.' in the - last line. - - - (lyra) append ``I've a dream'' to \verb+walden.pond+. - - - (lyra) sync Google Drive using \verb+rclone+. - - - (grus) sync Google Drive using \verb+rclone+. - - - (grus) verify that the latest copy of \verb+walden.pond+ is there - in the combox directory; it should contain ``I've a dream'' in the - last line. - - - (grus) remove \verb+walden.pond+ from combox directory. - - - (grus) sync Google Drive using \verb+rclone+. - - - (lyra) sync Google Drive using \verb+rclone+. - - - (lyra) verify that \verb+walden.pond+ is removed from the combox - directory. - - - (grus) open dropbox and Google drive accounts from the web browser. - - - (lyra) create file \verb+manufacturing.consent.+ with content ``Chomsky stuff?''. - - - (lyra) sync Google Drive using \verb+rclone+. - - - (grus) sync Google Drive using \verb+rclone+. - - - (grus) verify that \verb+manufacturing.consent+ was created in the - combox directory. - - - (grus) verify that the shards of \verb+manufacturing.consent+ were - created on Dropbox and Google Drive through the web browser. - -\subsection{Testing on a GNU/Linux and an OS X machine} - -combox was run on a GNU/Linux machine and an OS X machine and a file -was alternatively created/modified/renamed/deleted on one of the -machine and it was verified if the respective file was also -created/modified/renamed/deleted on the other machine. The GNU/Linux -machine was a virtual machine (\verb+lyra+) running Debian GNU/Linux -stable; the OS X machine was on Mavericks (10.9) during the initial -stage of testing, later it was upgraded to Yosemite (10.10). The node -directories to scatter files' shards were the Dropbox directory and -the Google Drive directory. The official Dropbox client was used to -automatically sync files from the Dropbox directory to the Dropbox' -server on both the GNU/Linux machine and the OS X machine; the -official Google Drive client was used to automatically sync files from -the Google Drive directory to Google Drive' server on OS X and -\verb+rclone+\cite{program:rclone} was used to sync files from the -Google Drive directory to Google Drive's server on GNU/Linux. - -\subsubsection{Issues found} - -\begin{itemize} -\item When a file was modified on another computer, on this computer - combox assumed that first shard (shard0) will be updated first and - also counted on the existence of the first shard (shard0). It was - observed that the order in which the shards were updated were - unpredictable on this computer and if the first shard (shard0) was - stored in the Dropbox directory, it will momentarily disappear - before the most updated shard becomes available in the Dropbox - directory; this broke combox. This issue was fixed on - 2015-08-25\cite{git:bug-four-fix}. This issue is not got to do with - the nature of the setup but it is related to the Dropbox's behavior - elaborated in section \ref{ch-5-2gnus-issues}. -\item The official Google Drive client when it pulls an updated - version of the file from Google Drive' server, instead directly - updating the respective file on the computer, it deletes the older - version of the file and creates the latest version of the file at - the respective location in the Google Drive directory; this behavior - of the Google Drive confused and broke combox. This issue was fixed - 2015-09-06 by making combox under the official Google Client's - behavior\cite{git:bug-googledc-fix}. -\item When a non-empty directory was move/renamed on another computer, - the old directory was not getting properly deleted on this computer; - this was happening because the files under the directory being - renamed were not deleted when it was time for \verb+NodeDirMonitor+ - to \verb+rmdir+ the old directory. This issue again is not specific - to the nature of the setup but was found while testing combox on - this setup. This issue was fixed on - 2015-09-12\cite{git:bug-six-fix}. -\item It was found that \verb+combox.file.rm_path+ function failed - when it was given a non-existent path to remove; this issue was - fixed on 2015-09-12\cite{git:bug-seven-fix}. -\end{itemize} - -\subsubsection{Demo} - -Demo of combox being used on a GNU/Linux machine and OS X machine can -be viewed at \url{https://ricketyspace.net/combox/combox-gnu-osx.webm} - -\verb+lyra+ is the GNU/Linux (virtual) machine and -\verb+dhcp-129-1-66-1+ is the OS X machine that is being used for the -demo. The OS X machine is accessed through VNC\cite{article:vnc}. - -Description of what happens in the demo follows: - - - (\verb+lyra+) create file \verb+cat.stevens+ with content ``peace train''. - - - (\verb+lyra+) sync Google Drive using \verb+rclone+. - - - (\verb+dhcp-129-1-66-1+) verify that file \verb+cat.stevens+ is - created with content ``peace train''. - - - (\verb+dhcp-129-1-66-1+) append string ``moonshadow'' to file - \verb+cat.stevens+. - - - (\verb+lyra+) sync Google Drive using \verb+rclone+. - - - (\verb+lyra+) verify that the file \verb+cat.stevens+ was updated - (modified); last line must have the string ``moonshadow''. - - - (\verb+lyra+) append string ``father and son'' to the file - \verb+cat.stevens+. - - - (\verb+lyra+) sync Google Drive using \verb+rclone+. - - - (\verb+dhcp-129-1-66-1+) verify that the file \verb+cat.stevens+ - was updated (modified); last line must have the string ``father and - son''. - - - (\verb+dhcp-129-1-66-1+) rename file \verb+cat.stevens+ to - \verb+yusuf.islam+ - - - (\verb+lyra+) sync Google Drive using \verb+rclone+. - - - (\verb+lyra+) verify that the file \verb+cat.stevens+ was renamed - to \verb+yusuf.islam+. - -\subsection{Testing with a USB stick as a node} - -combox was run on a GNU/Linux machine and an OS X machine and a file -was alternatively created/modified/deleted on one of the machine and -it was verified if the repsective file was also -create/modified/deleted on the other machine. The GNU/Linux machine -was a physical machine (\verb+grus+) running Debian GNU/Linux stable; -The OS X machine was on Mavericks (10.9). The node directories to -scatter files' shards were the Dropbox directory, Google Drive -directory and the USB stick (\verb+ZAPHOD+, FAT filesystem). The -official Dropbox client was used to automatically sync files from -Dropbox directory to Dropbox' server on both the GNU/Linux machine and -OS X machine; the official Google Drive client was used to -automatically sync files from the Google Drive directory to Google -Drive' server on OS X and \verb+rclone+\cite{program:rclone} was used -to sync files from the Google Drive directory to Google Drive's server -on GNU/Linux; the same USB stick (\verb+ZAPHOD+) was used on bothe -GNU/Linux and Dropbox to store the third shard (shard2) of a file. - -\subsubsection{Caveats} - -\begin{itemize} -\item When a removable USB disk is used as a node, combox must be - turned off before ejecting/unmounting the USB disk; combox does not - expect a node directory to disappear when it is running, if the USB - disk is removed when combox is running, then combox goes to a - undefined state. - -\item When a file modified on machine A is synced to machine B, combox - must be turned on first before turning on Dropbox and Google Drive - clients and the shard in the USB disk needs to be ``touched'' for - combox to detect that the file was modified on the remote computer - and update the file locally on this machine. - -\item File rename/move does not work. To make it work, core - functionality of combox must be re-written. -\end{itemize} - -\subsubsection{Demo} - -Demo of combox being used with a USB stick as the third node can be -view at \url{https://ricketyspace.net/combox/combox-usb-node-demo.webm} - -\verb+grus+ is the GNU/Linux machine and \verb+dhcp-129-1-66-1+ is the -OS X machine that is being used for the demo. \verb+ZAPHOD+ is the -FAT32 USB stick used as the third node. - -Description of what happens in the demo follows: - - - (\verb+grus+) start combox. - - - (\verb+grus+) create a file called \verb+simon.and.garfunkel+ with - content ``the boxer''. - - - (\verb+grus+) sync Google Drive using \verb+rclone+. - - - (\verb+grus+) stop combox. - - - (\verb+grus+) unmount USB stick (\verb+ZAPHOD+) from \verb+grus+. - - - (\verb+dhcp-129-1-66-1+) mount USB stick (\verb+ZAPHOD+) to - (\verb+dhcp-129-1-66-1+). - - - (\verb+dhcp-129-1-66-1+) start Dropbox client. - - - (\verb+dhcp-129-1-66-1+) start Google Drive client. - - - (\verb+dhcp-129-1-66-1+) start combox. - - - (\verb+dhcp-129-1-66-1+) verify that the file - \verb+simon.and.garfunkel+ with content ``the boxer'' was created. - - - (\verb+dhcp-129-1-66-1+) append string ``mrs. robinson'' to file - \verb+simon.and.garfunkel+. - - - (\verb+dhcp-129-1-66-1+) stop combox. - - - (\verb+dhcp-129-1-66-1+) stop Google Drive client. - - - (\verb+dhcp-129-1-66-1+) stop Dropbox client. - - - (\verb+dhcp-129-1-66-1+) unmount the USB stick (\verb+ZAPHOD+) - from (\verb+dhcp-129-1-66-1+). - - - (\verb+grus+) mount the USB stick (\verb+ZAPHOD+) to - (\verb+grus+). - - - (\verb+grus+) start combox. - - - (\verb+grus+) start Dropbox client. - - - (\verb+grus+) sync Google Drive using \verb+rclone+. - - - (\verb+grus+) touch \verb+simon.and.garfunkel.shard2+ in the USB - stick (\verb+ZAPHOD+). - - - (\verb+grus+) verify that the file \verb+simon.and.garfunkel+ is - updated; the last line must contain the string ``mrs. robinson''. - - - (\verb+grus+) remove the file \verb+simon.and.garfunkel+. - - - (\verb+grus+) sync Google Drive using \verb+rclone+. - - - (\verb+grus+) unmount the USB stick (\verb+ZAPHOD+) from - (\verb+grus+). - - - (\verb+grus+) stop Dropbox client. - - - (\verb+dhcp-129-1-66-1+) mount the USB stick (\verb+ZAPHOD+) to - (\verb+dhcp-129-1-66-1+). - - - (\verb+dhcp-129-1-66-1+) start Google Drive client. - - - (\verb+dhcp-129-1-66-1+) start Dropbox client. - - - (\verb+dhcp-129-1-66-1+) start combox. - - - (\verb+dhcp-129-1-66-1+) verify that the file - \verb+simon.and.garfunkel+ was deleted. - - -\section{Stress testing} - -Large number of files of different sizes were dumped to the combox -directory between an one second interval to see how combox responds to -high load. The file dump size was varied from \verb+424.798190MiB+ (27 -files) to \verb+10800.000000MiB+ (180 files); the average time taken -to split a file and the total time to process all files were -calculated for each dump. - -Stress testing was first done on \verb+2015-11-08+. In mid November -the \verb+ComboxDirMonitor+ was drastically modified to make it use -the file Lock shared the instances of -\verb+NodeDirMonitor+\cite{git:bug-eleven-fix}; my hunch was that this -change in \verb+ComboxDirMonitor+ directly affected the performance of -combox and therefore the results that were got from stress testing on -\verb+2015-11-08+ would no longer be valid. Stress testing was again -done on \verb+2016-01-16+; the results of this stress test are in -sections \ref{5-st-424} to \ref{5-st-10800}, section \ref{5-st-tu} -gives information about the tools used for stress testing, section -\ref{5-st-o} contains the observations and comparisons between this -stress test and the one done on \verb+2015-11-08+, lastly section -\ref{5-st-if} reveals the issues that were found with combox by virtue -of doing the stress tests. - -\subsection{flac dump (27 files - 424.798190MiB)}\label{5-st-424} - -\begin{center} -\begin{tabular}{ll} -field & value\\ -\hline -delay between a file dump & 1s\\ -start time of processing & 11:00:54\\ -end time of processing & 11:01:38\\ -total time taken to process all files & 00:00:44\\ -no. of files & 27\\ -total size of all files & 445433187.000000 bytes (424.798190MiB)\\ -avg. file size & 16497525.000000 bytes (15.733266MiB)\\ -avg. time to split and encrypt a file & 352.583370 ms\\ -\end{tabular} -\end{center} - -\subsubsection{Differences from previous stress test (2015-11-08)} - -\begin{itemize} -\item Total time to process all files was faster by 1min3secs. -\item Average time to split and encrypt a file reduced by - 28.337963000000002ms. -\end{itemize} - -\subsection{20MiB - 90MiB dump (27 files - 1620.000000MiB)}\label{5-st-1620} - -\begin{center} -\begin{tabular}{ll} -field & value\\ -\hline -delay between a file dump & 1s\\ -start time of processing & 12:26:45\\ -end time of processing & 12:29:07\\ -total time taken to process all files & 00:02:22\\ -no. of files & 27\\ -total size of all files & 1698693120.000000 bytes (1620.000000MiB)\\ -avg. file size & 62914560.000000 bytes (60.000000MiB)\\ -avg. time to split and encrypt a file & 2670.596556ms\\ -\end{tabular} -\end{center} - -\subsubsection{Differences from previous stress test (2015-11-08)} - -\begin{itemize} -\item Total time to process all files was slower by 4secs. -\item Average time to split and encrypt a file reduced by - 25.52536999999984ms. -\end{itemize} - -\subsection{20MiB - 90MiB dump (99 files - 5940.000000MiB)}\label{5-st-5940} - -\begin{center} -\begin{tabular}{ll} -field & value\\ -\hline -delay between a file dump & 1s\\ -start time of processing & 13:10:16\\ -end time of processing & 13:19:26\\ -total time taken to process all files & 00:09:10\\ -no. of files & 99\\ -total size of all files & 6228541440.000000 bytes (5940.000000MiB)\\ -avg. file size & 62914560.000000 bytes (60.000000MiB)\\ -avg. time to split and encrypt a file & 2979.647586ms\\ -\end{tabular} -\end{center} - -\subsubsection{Differences from previous stress test (2015-11-08)} - -\begin{itemize} -\item Total time to process all files was faster by 59secs. -\item Average time to split and encrypt a file increased by - 206.20906100000002ms. -\end{itemize} - -\subsection{20MiB - 90MiB dump (180 files - 10800.000000MiB)}\label{5-st-10800} - -\begin{center} -\begin{tabular}{ll} -field & value\\ -\hline -delay between a file dump & 1s\\ -start time of processing & 13:42:06\\ -end time of processing & 14:00:10\\ -total time taken to process all files & 00:18:04\\ -no. of files & 180\\ -total size of all files & 11324620800.000000 bytes (10800.000000MiB)\\ -avg. file size & 62914560.000000 bytes (60.000000MiB)\\ -avg. time to split and encrypt a file & 3423.087539ms\\ -\end{tabular} -\end{center} - -\subsubsection{Differences from previous stress test (2015-11-08)} - -\begin{itemize} -\item Total time to process all files was slower by 1min2secs -\item Average time to split and encrypt a file increased by - 399.87623299999996ms. -\end{itemize} - -\subsection{Tools used}\label{5-st-tu} - -The \verb+dump+ script\cite{program:dump} was used to dump files to -the combox directory between one second intervals; a night of Emacs -Lisp indulgence made it possible to quickly slurp the required data -from the combox output and calculate the average time to split and -encrypt a file and the total amount of time taken to process the files -for a given dump\cite{program:dumps.el}; lastly \verb+org-mode+ was -used to document all data gathered during stress -testing\cite{doc:benchmarks.org}. - -\subsection{Observations}\label{5-st-o} - -\begin{figure}[h] -\centering -\input{graphs/tot-time.tex} -\caption{time to process all files} -\label{fig:5-st-tt} -\end{figure} - -\begin{figure}[h] -\centering -\input{graphs/avg-time-sae.tex} -\caption{avg. time to split and encrypt} -\label{fig:5-st-atsae} -\end{figure} - - -\begin{itemize} -\item Figure \ref{fig:5-st-tt} shows the time it takes combox to - process files for a given file dump\footnote{A ``file dump'' here - means a bunch of files copied to the combox directory between 1 - sec intervals.}. As can be observed from the graph, the total time - taken to process all the files tends almost linearly increase with - the increase in the size of the file dump\footnote{The ``size of the - file dump'' is the total size of all files in a given file dump.}. -\item Figure \ref{fig:5-st-atsae} show the average time it takes - combox to split and encrypt a file for a given file dump. There is a - steep increase in the average time from the \verb+424.798190MiB+ - dump and the \verb+1620.000000MiB+ dump, after which the average - time to split and encrypt a file seems to almost linearly increase; - The main reason for this is that the average file size for dumps - from \verb+1620.000000MiB+ to \verb+10800.000000MiB+ are the same. -\end{itemize} - -\begin{figure}[h] -\centering -\input{graphs/tot-time-diff.tex} -\caption{time to process all files - difference between 2015 and 2016} -\label{fig:5-st-tt-diff} -\end{figure} - -\begin{figure}[h] -\centering -\input{graphs/avg-time-sae-diff.tex} -\caption{avg. time to split and encrypt - difference between 2015 and 2016} -\label{fig:5-st-atsae-diff} -\end{figure} - -\begin{itemize} -\item Figure \ref{fig:5-st-tt-diff} shows the graphs for the total - amount of time taken to process all files for a given file dump in - the \verb+2016-01-16+ and \verb+2015-11-8+ stress test. The amount - of time needed to process all fills seems to be reduced for the - \verb+5940.000000MiB+ file dump when compared to the \verb+2015+ - stress test results and it seems to be slightly higher for the - \verb+10800.000000MiB+ file dump when compared to the \verb+2015+ - stress test. -\item Similarly, figure \ref{fig:5-st-atsae-diff} shows the graphs for - the average time to split and encrypt for a given file dump in the - \verb+2016-01-16+ and the \verb+2015-11-8+ stress test. The average - time taken seems to able almost the same for the - \verb+424.798190MiB+ and the \verb+1620.000000+ dump, but for the - \verb+5940.000000MiB+ and the \verb+10800.000000MiB+ dump the - average time taken seems to higher for the \verb+2016+ stress test - when compared to the \verb+2015+ stress test. -\end{itemize} - -\subsection{Issues found}\label{5-st-if} - -\begin{itemize} -\item Initially when combox was stress tested with huge files, combox - would get overwhelmed leading to the computer running out of memory - and the load average sometimes peaking at \verb+8+. At first, it was - assumed that there was a bug in combox which caused this to happen, - but later it was found that \verb+watchdog+\cite{pylib:watchdog} was - generating a large number ``file modified'' events when a huge file - (\verb+~500MiB+ was modified). To prevent \verb+watchdog+ from - generating a large number ``file modified'' events for a single - modification of a huge file, a delay proportional to the size of the - file was created in the \verb+on_modified+ callback methods in both - \verb+ComboxDirMonitor+ and - \verb+NodeDirMonitor+\cite{git:bug-ten-fix}, this fixed the - issue. Also, this it might be useful to note here that this was - ``the'' hardest issue I dealt with in working on combox. -\end{itemize}
\ No newline at end of file |