On javascript, and its testing

Lately I run into requirements that need an intensive javascript logic, including Ajax, jQueryUI and dynamic DOM manipulation logic. It makes me ponder over issues I have had with this whole thing. 

I hate it. 

Why? 

Javascript itself is a nice little language, completely object oriented, on prototyping paradigm. But when it comes to Ajax and DOM manipulation, well, there is no other term more appropriate than shit hits the fan. 

jQueryUI is a great tool. You select the DOM elements with the CSS3 selector, you attach things to it. Job's done. I can add a calendar widget, a date picker, and a rich text editor to my site in no more than a line of javascript code each. The hard part, though, begins when I have to begin handling the events these widgets fire. 

When I want to handle a hover event on a calendar entry, which element am I talking to? There is 'this' variable which is the reference to the object firing the event. But I want the event to affect other elements of the same kind. Dig through the 5000 lines of library code (which is not the cleanest code in the world, and no test to refer to either) is not an easy task. How is jQueryUI object defined? How do I access the methods inside the object? It is either my lack of understanding of the prototyping OO, or jQuery was not designed with the OO in mind. I suspect it's the former. 

Stuff like this reminds me how valuable it is to stick to Test-Driven-Development. The test lets you make sure that new code you add in will not break original functionality. The new test lets you make sure you are coding to the requirement, nothing more. It assures you that any refactoring you do maintains the same functionality and purpose, and nothing more. 

So how do we go about writing tests for these Javascript code? 

Well, not easy. The one word you need to keep reminding yourself: discipline. A strict one.

First let me explain on the technical side how javascript testing is done. There are two ways you can test javascript code. Standalone javascript unit testing, and high level functional testing.

Standalone javascript unit testing is a fine grain level of testing, focusing on the javascript code itself, and nothing else. The tool available is Jasmine. Jasmine runs anywhere javascript code can run. V8, Rubyracer, Rhino, Firefox, Chrome, IE, Opera, etc. All you need to do is to make the Jasmine standalone package accessible in the scope of the code, and you can harness all the testing facility it provides. To test the DOM manipulation logic, you need HTML fixture. The fixture needs to have the DOM you want to manipulate, and all the condition you expect from the 'real' page. When the test is run, the assertion should be able to assert that the DOM with the particular CSS matching condition should change in the way you have expected. 

You should see a problem by now. Do you? 

The problem is, we create the 'real' page with the server side code, with dynamic content depending on the states of objects related. Unless you are on node.js code, you cannot effectively invoke those 'real' pages with the plain Jasmine standalone. The fixture, then, has to significantly match the 'real' page. Once the 'real' page changes, chances are you forget to update your fixture. The tests pass, but somehow there are bugs in the system. Also, as soon as you fall in the pit of 'we can just have the server side code generate the javascript for us,' the ability to test the javascript loses its coverage. If that dynamic javascript includes crucial part of the logic, such as initialization variables, then the whole suite breaks down. As soon as you have Ajax code in the path of the testing logic, the suite breaks down. A particular problem related to running the javascript unit testing on check in time is automation. If you want to run it as a check in script, you need some javascript engine which is separated from the browser's engine. You have the same problem as the fixture problem. They are not the same engine, and they may be different in functionality. Rhino, for example, does not update the DOM element width and height properties without having to manually calling style update functions. You can manually fire up the browser and run the suite there. But if I get $20 each time we forget, I won't be running around offering my service to feed my wife and child.

There are merits to javascript unit testing, no doubt. The limitation, however, is significant. You need to be careful to make sure that the DOM manipulation logic is safe, static, and as isolated as possible.

On the other hand, we have the high level functional testing. 

In this approach, we have the browser adapter that takes macros to run automatically, then after the macro is performed, assertions are made on the resulting page. Selenium is the prime example of this kind of test. You start out with a Selenium adapter opening up a browser, most of the time Firefox. Tell it to visit a URL, may be enter some text in the login field, and password field, click the login button. Then the test suite asserts that the current URL of the window is the home page, with a particular DOM element saying "Welcome to my site, #{username}", etc. 

This approach, of course, overcome all the limitation we have in the other approach. We can rely on anything that results in the page of interest. There is no fixture to worry about. There are adapters to allow high level functionality on all kinds of programming language. The test suites themselves are written in any language of choice. 

Problem within this approach is all the steps you need to achieve just to be able to start testing. You need a server up and running, with everything setup according to the fixture specification, a user in the database, the date and the time for time-sensitive data, etc. At the end of every single testcase we have to reset the whole thing. The server process is separated from the test suite, as the suite runs on the client side, but contradicts itself and go ahead and resets the state of the app on the server side as well. Transactional fixture cannot be performed, and data cannot be rolled back instead of deleted. Performance is a major concern. The more we cover, the polynomially longer the suite takes to perform. If the test is badly designed, the degree of increasing complexity becomes exponential. Without enough coverage, there is no use in investing in functional suite. With high enough coverage, and a big enough codebase, the time to run the test is actually longer than it to be of any use. There's a balance we need to strike. 

We can run the functional suite at check in time, but you have to think about how many times you are checking in a day, and how much time is spent waiting. We can run it as a CI suite, but how much do we really care about it? A mysterious bug that ends up being a wrong testcase, or a change in a tiny DOM element id, or class, will remind you of a shepherd boy tale. Soon enough, you will lose interest in maintaining it. At that point it ceases to be useful. 

Now, back to our one word, discipline.

Problems we have with both approaches can be mitigated to a certain degree if the whole team accepts some discipline terms. When it comes to Javascript, it must be considered as a separated set of logic from the server side code. Do not rely on the server side to provide anything. It's irony that Javascript asynchronous behavior, which the client-server HTTP paradigm lacks, requires us to think about Javascript logic very differently from how we think about the web application client-server logic. Javascript logic fits much more to the .NET analogy than we anti-M$ monkeys dare admit. With that understanding in mind, we have a better clue of how the code executes, and how the logic are supposed to be tested. We need to design our Javascript logic so that the Ajax facility, the jQueryUI facility, and any facilities that is hard or impossible to unit test are contained, isolated, and tested by the high level functional suite. The rest can then be tested with the unit test for acceptable performance reasons. 

This discipline is not achievable overnight. Especially when you have existing code base that has javascript logic splattered all over the place, intermixing Ajax, DOM manipulation, and server side code. The amount of time and effort it takes to fix this, and the business value it brings make it really hard to justify the investment from the stakeholders. 

Either you do it right from the very start, or you need one hell of influence. 

Start with discipline, good things will then follow.