summaryrefslogtreecommitdiffstats
path: root/report
diff options
context:
space:
mode:
authorSiddharth Ravikumar <sravik@bgsu.edu>2016-01-22 16:22:08 -0500
committerSiddharth Ravikumar <sravik@bgsu.edu>2016-01-22 16:22:08 -0500
commit917d6d61eb3a2c0ef075e51ad5803d72602a2b56 (patch)
tree49a564b6aa50114f06d41f416b8281595e4c01b3 /report
parent83d1ee510bb038df98a040d766acd1abd1c2cfb4 (diff)
Started fleshing out chapter 5.
Diffstat (limited to 'report')
-rw-r--r--report/Makefile12
-rw-r--r--report/bib/combox.bib29
-rw-r--r--report/chapters/5-ucases-benchmarks.tex73
-rw-r--r--report/combox-report.pdfbin90607 -> 119399 bytes
-rw-r--r--report/combox.tex6
5 files changed, 113 insertions, 7 deletions
diff --git a/report/Makefile b/report/Makefile
index 6b58488..2b1a083 100644
--- a/report/Makefile
+++ b/report/Makefile
@@ -10,8 +10,9 @@
# <https://creativecommons.org/publicdomain/zero/1.0/>
PDF = combox-report.pdf
-REPORT = combox.tex
-REPORT_INCLUDES = graphs/*.tex chapters/*.tex
+REPORT_TEX = combox.tex
+REPORT_AUX = combox.aux
+REPORT_INCLUDES = graphs/*.tex chapters/*.tex bib/combox.bib
DVI = combox.dvi
all: $(PDF)
@@ -20,8 +21,11 @@ all: $(PDF)
$(PDF): $(DVI)
@dvipdf $< $@
-$(DVI): $(REPORT) $(REPORT_INCLUDES)
- @latex $<; latex $<; latex $<
+$(DVI): $(REPORT_TEX) $(REPORT_INCLUDES)
+ @latex $<
+ @bibtex $(REPORT_AUX)
+ @latex $<
+ @latex $<
clean:
rm -f $(PDF) *.aux *.dvi *.log *~
diff --git a/report/bib/combox.bib b/report/bib/combox.bib
new file mode 100644
index 0000000..cc47d6d
--- /dev/null
+++ b/report/bib/combox.bib
@@ -0,0 +1,29 @@
+@misc{pylib:nose,
+title = "Nose - a nicer testing for python",
+url = "https://nose.readthedocs.org/en/latest/",
+}
+
+@misc{combox-issue-tracker,
+title = "combox issue tracker (org-mode)",
+url = "https://git.ricketyspace.net/combox/plain/TODO.org",
+}
+
+@misc{combox-wd-fix,
+title = "combox - watchdog 'file create event' bug fix",
+url = "https://git.ricketyspace.net/combox/commit/?id=8c86e7c28738c66c0e04ae7886b44dbcdfc6369e",
+
+}
+
+@misc{pylib:watchdog,
+title = "Watchdog - Python API library and shell utilities to monitor file system events.",
+url = "https://pythonhosted.org/watchdog/",
+}
+
+@techreport{dijkstra69,
+title = "Software Engineering Techniques",
+author = "J.N. Buxton and B. Randell",
+year = 1969,
+institution = "NATO Science Committee",
+number = "p. 16",
+url = "http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF"
+} \ No newline at end of file
diff --git a/report/chapters/5-ucases-benchmarks.tex b/report/chapters/5-ucases-benchmarks.tex
index 78e9e66..08421a3 100644
--- a/report/chapters/5-ucases-benchmarks.tex
+++ b/report/chapters/5-ucases-benchmarks.tex
@@ -1,4 +1,75 @@
-\chapter{Use Cases, Benchmarks, and Demonstration}
+\chapter{Testing}
+
+\section{Unit testing}
+
+The \verb+nose+\cite{pylib:nose} testing framework was used to
+write unit tests for the functions and classes part of the
+\verb+combox.config+, \verb+combox.crypto+, \verb+combox.events+,
+\verb+combox.file+, \verb+combox.silo+ \verb+combox._version+
+modules. Unit tests were not written for \verb+combox.cbox+,
+\verb+combox.gui+, \verb+combox.combox.log+ modules.
+
+Unit tests for combox become reality by pure serendipity. During the
+time, when I started working on combox, I was learning to use the
+\verb+nose+ library to unit test python code. Since, \verb+combox+ was
+being written in python, I started making it a norm to write unit
+tests for functions and classes in combox modules.
+
+As mentioned before, unit tests were not written for some modules
+either because it would make no sense to write one (for the
+\verb+combox.cbox+ module, for instance, which basically uses
+functions and classes defined in other modules to run combox) or it
+was not clear how to write unit tests it (the \verb+combox.gui+
+contains just the \verb+ComboxConfigDialog+ a graphical front-end
+which uses the configuration function defined in the
+\verb+combox.config+ module to complete the combox configuration based
+on the user input).
+
+It must be noted here that pure Test Driven Development (TDD) was not
+observed -- most of the time the function/class was written before the
+its corresponding test was written.
+
+\subsection{Benefits}
+
+While writing unit tests definitely increased the time to write a
+particular feature, it enabled me to immediately check if a feature
+worked as it should for the given use case or given set of inputs.
+
+With the benefit of hindsight, unit tests greatly helped in testing
+the compatibility of combox on OSX. Before the \verb+v0.1.0+ release,
+combox's node directory monitor always assumed that a file's first
+shard (\verb+shard0+) is always available; while this assumption did
+not create any problems on GNU/Linux, on OS X, this assumption made
+the node directory monitor to behave erraticly -- this issue (bug
+\#4\cite{combox-issue-tracker} was immediately found when the unit
+tests were run for the first time on OS X. Another instance where unit
+tests helped was just before the \verb+v0.2.0+ release; major changes,
+including the introduction of file locks in the
+\verb+ComboxDirMonitor+, were made to the \verb+combox.events+. When
+the unit tests were run OS X, two tests failed, revealing a difference
+in behaviour of watchdog\cite{pylib:watchdog} on GNU/Linux and OS X on
+file creation\cite{combox-wd-fix}; without unit tests, there is a high
+probability that this bug would never have been found by now.
+
+\subsection{Caveats}
+
+Unit tests are helpful in testing the correctness of a feature for
+\verb+N+ number of use cases but it does not necessarily mean the
+written feature correctly behaves for use cases that the author of the
+feature did not consider or did not think about while writing the
+respective feature. As Dijkstra correctly observed\cite{dijkstra69}:
+
+\begin{quote}
+Testing shows the presence, not the absence of bugs
+\end{quote}
+
+Unit tests failed reveal bugs \#4, \#5 \#6 \#7 \#5 \#10
+\#11\cite{combox-issue-tracker}; these bugs were found when manually
+testing combox.
+
+\section{Manual testing}
+
+\section{Stress testing}
\begin{figure}[h]
\centering
diff --git a/report/combox-report.pdf b/report/combox-report.pdf
index b5e41f1..23cd75d 100644
--- a/report/combox-report.pdf
+++ b/report/combox-report.pdf
Binary files differ
diff --git a/report/combox.tex b/report/combox.tex
index 48aa2c6..a787815 100644
--- a/report/combox.tex
+++ b/report/combox.tex
@@ -1,5 +1,7 @@
\documentclass[12pt]{uthesis-v12} %---> DO NOT ALTER THIS COMMAND
+\usepackage{hyperref}
+
\begin{document} %---> %---> %---> %---> DO NOT ALTER THIS COMMAND
%--------+----------------------------------------------------------+
@@ -348,7 +350,7 @@
\chapter{Literature Review}
%% 4
-\chapter{combox Architecture and Design}
+\chapter{Architecture and Design}
%% 5
\include{chapters/5-ucases-benchmarks}
@@ -357,7 +359,7 @@
\chapter{Conclusion and Future Work}
\bibliographystyle{IEEEtran}
-\bibliography{bib}
+\bibliography{bib/combox}
\end{document} %---> ---> ---> ---> DO NOT ALTER THIS COMMAND