Holy Watir!

I was in a tight spot last week. As a middle manager at a small software company, I was in charge of starting the load test practice. We set the goal and plan of actions. The major problem was that our software was designed in a way that prevented any load testing tool to function properly without any major effort. I tried WAPT, jMeter, and Grinder. While some worked better than others, all of them failed due to some fundamental problems with the user interface layer design. I pushed the development manager in charge to make the changes. But it was difficult, time-consuming and risky to change, and the manager was already stressed out over a couple of other projects.

The deadline was approaching.

Watir was designed to do single user automation testing. It just invoked an IE instance and drove the UI elements in a web page. One could always create multiple IE instances to simulate multiple users. This was not a preferred way to do load test. But I was desperate.

I wanted to conduct three kind of test:

• Longevity test: to make sure our server can stay healthy under heavy load for many hours.
• Capacity test: to measure the number of concurrent users one server node can support for more an extended period of time.
• Response time test: to measure the response time under typical scenarios.

I figured that Watir could do the first two kind of tests. So I spent about two hours modifying an existing script that a colleague of mine had written last Friday. I used Firebug to identify the dynamically generated HTML elements. Then I installed Ruby and Watir on a VMWare image that I allocated enough memory. We were ready to go.

On Saturday morning, when I got the green light from our QA team in India that it was all clear, I restarted the QA server and started my longevity test. The test script fired up and closed about fifteen IE windows. It paused then different users logged in and logged out. The screens were flashing. The VM image was just fine, no memory leak after a long period of time. Ruby and Watir seemed to be much improved. I managed to go to hiking with Victor while starting and stopping multiple load tests.

Ultimately the longevity tests failed. And we are still debugging the problem. The poor man's stress testing tool was just amazing; I could never imaging that it just works out-of-box. Thanks to Yun-Ping who introduced me to this tool.

A side note: So far Tomcat lasted ten times as long as WS in my longevity tests.