Archive for October, 2009

Relief for the pain of waiting for AJAX elements on the page?

October 21, 2009

More and more web applications are taking advantage of asynchronous JavaScript (or AJAX) – and with good reason. It gives tremendous power to a web application, allowing the app to respond fluidly to the user without requiring a full page reload and making it feel more like a native desktop app. At the same time, it makes the application trickier to automate.

When I first started writing browser-based test automation, it was safe to assume that once the page loaded, if an element wasn’t there yet, it wasn’t going to be. With AJAX, that assumption goes out the window. Let’s take an example:

Someone’s registering for our website. The site prompts her for a username, which she enters. Now, in the olden days, she would fill out the rest of the page and then submit. The page would refresh…and often tell her that the username she selected is unavailable. Rinse and repeat until you finally find an available name.

With AJAX, the moment she clicks away from the username field, we can have feedback appear on the page, letting her know whether or not that name has been taken. She doesn’t need to wait for a page refresh before each attempt, the feedback may be nearly instantaneous.

Sounds good, right? Now I come along to automate a test of this page. If I write something like (in pseudocode):

    set username = reserved_name
    assert response_text says reserved_name is taken.

…much of the time, I’ll get an error saying that the response_text isn’t there on the screen.

How can I tell my script not to progress to the next step until the application is ready? I have a range of options here. At the cringe-inspiring end of the spectrum, I can write:

    set username = reserved_name
    sleep 10 seconds
    assert response_text says reserved_name is taken.

The problem here is that one either sets the sleep too short, and the script errors out when the test server is under a bit of extra load — or one sets it too long, and the scripts begin to spend 90% of their time sleeping.

Somewhat better is to do like so:

    set username = reserved_name
    wait( up to 20 seconds) for response_text #then
    assert text in page says reserved_name is taken.

The issue here is that you need to know exactly what you’re waiting for, may need to list multiple triggers to wait for, and your script risks of aging poorly as these triggers change.

To get better than this likely requires testability features in the application under test. One thing that’s worked well for me is having the application set a flag whenever it starts to execute AJAX, and unset that flag when the AJAX completes. At Freebase.com, we chose to have that flag be an attribute on the body tag, like so:

Whenever AJAX is kicked off, this is set to ajaxStart and whenever the last asynchronous process completes, it sets it to

Now I can say something like:

    set username = reserved_name
    wait_until_loaded
    assert text in page says reserved_name is taken.

Where wait_until_loaded is defined like:
Wait( up to 20 seconds) while body.attribute(ajax) == ‘ajaxStart’

This pushes the burden of knowing when the page is ready for the next step back to the application under test. The downsides of this approach are: (1) it requires code in the application under text to make it work. Sometimes that’s difficult (politically or otherwise). (2) this solution’s only as robust as the code that sets to ajax flag. If the application under test can’t or won’t report this reliably, this solution will only cause you grief.

That said, the upside of this approach is potentially huge: It frees the tester from writing a custom wait trigger after every AJAX step, speeding up test script writing and often making the scripts more robust as well.

Depending on how familiar you are with the testing library you’re using, it may be worth considering putting this check deeper in your code. For example, one might change the click method to always wait_until_loaded before clicking. This has the potential to streamline test scripts that much more.

In fact, the Windmill test automation library has already done this on trunk. As of the next release of Windmill, the default behavior for everything other than asserts is to wait for that element for a set number of seconds — progressing immediately if it’s there right away, but only throwing an error if the timeout is reached. I suspect as web applications become more AJAX-heavy, this will become the standard in test automation libraries.

There isn’t a One Right Answer for any context, but I’ve been very happy with having the application under test set an ajax flag of some sort. If you’re testing an application where the page changes asynchronously, I encourage you to give it a try.

Advertisements

When (Broken) Software Inspires New Language

October 15, 2009

Software Testing has many joys. Most have to do with solving tricky puzzles, collaborating with brilliant minds, or contributing to the creation of elegant software…but sometimes there are simpler pleasures as well.

Today a colleague sent around a prototype of an application that takes any string and attempts to pluralize it. It handles a lot of tricky cases, e.g. correctly pluralizing “surgeon general” as “surgeons general”. On the other hand, it makes many mistakes, and I suspect will always do so. One example: it currently assumes that any input is singular. When run against one of the data sets it was designed for (a list of Types Freebase.com users have created) I noticed it turned “Most Disliked Things” into “Most Disliked Thingses”. At that point I couldn’t resist watching it turn “Hobbits” into “Hobbitses”…and then coining a new term:

Smeagolize: To alter a word or phrase, making it sound like something the character Smeagol from the Lord of the Rings would say, e.g. “Most Disliked Things” into “Most Disliked Thingses”

This reminded me of one of my favorite examples of a coder solving a problem in an amusing way. This was many years and companies ago. A harried coder was asked to display some (potentially long) URLs in a very small space on a home page. As he worked on it, he discovered that some URLs didn’t fit the space. What to do? A less intrepid coder might have asked for help with the design problem, but he tackled it himself…by writing code to remove all the vowels from any URL over 20 characters.  This led us to coin the term:

Disemvowel: To remove the vowels from a URL or other word, e.g. changing
http://www.associationforsoftwaretesting.org/ to
http://www.ssctnfrsftwrtstng.rg/

Needless to say, this particular code didn’t live to see the light of day…but its memory lives on in the term Disemvowel.

Has broken software inspired new words for you as well?