What does reliability means for a computer program?
It means that it has to behave exactly as its designers expect it to.
One way to achieve it is to test every part of the program with a given input, against the expected results. Those tests should be automated and run regularly during the development process to make sure that no regression is introduced (what works today doesn’t break tomorrow). The rule is to break down the program into simple units and test them separately in order to identify precisely the source of a problem.
This technique is called unit testing and most languages are offering a framework to write, run and produce a visual feedback for the test (green for pass and red for fail).
Consider a function myCamelCase( sentence ) designed to turn any given string into my very own kind of camel case word. To make sure that this function is reliable, we need to be able to assert that, for each potential type of input string, its actual output will match the expected result:
| input string |
expected output |
| “word” |
“Word” |
| “WORD” |
“Word” |
| “two words” |
“TwoWords” |
| “A single letter” |
“aSingleLetter” |
| “this is a letter” |
“ThisIsAletter” |
| “remove+special_chars!” |
“RemoveSpecialChars” |
| “save 123 numbers” |
“Save123numbers” |
| “+ trim me _” |
“TrimMe” |
| ” + _ ! “ |
false |
Using QUnit
The unit test framework of choice for a jQuery related piece of code is QUnit. It offers a reduced number of assertion methods, allows to group them in tests and modules and display verbose feedback in HTML.
The assertions
QUnit offers three basic assertion functions: ok, equals, same
ok( state, message );
Should be used to assert a Boolean output for a function.
test("'ok' is not only meant to be used for Boolean values", function() {
ok(123, "123 is ok");
ok("abc", "Any string is ok");
ok([], "An empty array is ok");
ok(!"", "An empty string is not ok");
ok(!0, "0 is not ok");
ok(!null, "null is not ok");
ok(!undefined, "undefined is not ok");
});
equals( actual, expected, message );
Should be used for a comparison assertion.
test("Cases where 'equals' passes for two values not strictly identical", function() {
equals(1, true, "1 equals true");
equals([1], true, "An array with a single 1 value equals true");
equals("1", true, "A string with only the number 1 equals true");
equals(0, false, "0 equals false");
equals([0], false, "An array with a single 0 value equals false");
equals("0", false, "A string with only the number 0 equals false");
equals("012.345", 12.345, "A number and this number turned into a string are equal");
});
same( actual, expected, message );
Should be used for strict comparison assertion, works recursively on arrays and objects.
test("The actual and expected values have to be strictly identical", function() {
same([{a: 2}, false, null], [{a: 2}, false, null], "Same is recursive");
});
Writing tests and modules
A test is composed of an homogeneous set of assertions for one unit of the program: In our example, we should write one assertion for each potential type of input string. The first line of a test state the number of assertions that are expected to pass, it’s optional.
test("Test that each type of input string produces the expected output", function() {
expect(9);
equals( camelCase("word"), "Word", "First letter of a word is capitalized");
equals( camelCase("WORD"), "Word", "Other letters are lower-case");
equals( camelCase("two words"), "TwoWords", "Spaces between two words are removed");
equals( camelCase("A single letter"), "aSingleLetter", "If the first letter is a single letter, it is turned to lower-case");
equals( camelCase("this is a letter"), "ThisIsAletter", "If there is a single letter in the middle of a sentence, the following letter is turned to lower-case);
equals( camelCase("remove+special_chars!"), "RemoveSpecialChars", "Special characters are removed");
equals( camelCase("save 123 numbers"), "Save123numbers", "Numbers are saved and the following letter is turned to lower-case");
equals( camelCase("+ trim me _"), "TrimMe", "Trailing spaces are always removed");
ok( !camelCase(" + _ ! "), "Return false instead of an empty camel case word");
});
QUnit offers the module method to help you visually identify related tests:
module("Camel Case");
Every test following this module will be prefixed by the module name.
synchronizing tests
QUnit offers two other methods to pause the tests for example when an Ajax call was made and the client is waiting for the answer. It is possible to make a pause in the sequence using the stop() and the start() methods.
Making your life easier.
If you happen to write a long batch of tests, you’re likely to realise that you often write similar lines. Being, like the vast majority of us, a lazy developer, you may start copy/paste large amount of code and write additional logic in order to factorise your test code. By doing this way, you might end up chasing bug that are not in the code you are testing, but right in your tests!
On the other hand, there are several ways to write DRY tests safely.
In-line function
If your function accept a second parameter that is not supposed to change the output of the function, it’s possible to create an additional set of test with few lines of code by using an in-line function. Imagine that we add a second Boolean parameter to make the camel case algorithm go faster (I’d love to see such parameter for every algorithm): myCamelCase( sentence, faster). The code would be:
var camelCaseTest = function(faster) {
expect(9);
equals( camelCase("word", faster), "Word", "First letter of a word is capitalized");
equals( camelCase("WORD", faster), "Word", "Other letters are lower-case");
equals( camelCase("two words", faster), "TwoWords", "Spaces between two words are removed");
...
});
test("Test that each type of input string produces slowly the expected output", function() {
camelCaseTest(false);
});
test("Test that each type of input string produces quickly the expected output", function() {
camelCaseTest(true);
});
We’ve just written two tests for the same price!
Setup and Teardown
module() also offers a second parameter to register a setup and teardown callback: pieces of code that will be called respectively before and after each test of this module. For example if you want to reset a number of variables that have been modified during the test:
module("Test camel case", {
teardown: function() {
a = b = c = d = undefined;
}
});
Breaking the rule
The main rule in unit testing is to test your units of code separately. The smaller the units, the easier it will be to locate the source of an assertion failure. It is actually possible to break this rule and test multiple units at the same time ONCE those units have been tested enough to appear reliable.
Limit of unit tests
The first limit of unit testing is… the writer himself. If you don’t cause any failure just by writing tests, then I’m afraid you haven’t written enough tests. The second limit of unit tests in JavaScript is the number of configuration and what is actually testable.
This has been explained in details by John Resig in JavaScript Testing Does Not Scale. In brief, there is too much browsers and OS and it’s not possible to use unit testing to check the visual appearance of a page, or the correct behaviour of an animation for example.
Going further
During this post I have written a complete test for myCamelCase(), a method that I haven’t actually implemented. The test that I’ve written can actually be seen as a specification of this method, and I will only be able to claim that its implementation is complete once it successfully pass those tests. This software design method is called Test-driven Development and is one of the best way to write code that is reliable from the very beginning throughout the entire life cycle of the project.