J-Meter, first impressions

Today I had a chance to take a look at J-Meter, a load tests tool, which most important feature is that it's free.

It took up to 10 minutes of my time to generate a simple HTTP request load test. In this sense, I would not call the tool UI intuitive. But once you've done your first test you should have no problems in the future. It just takes time to get used to unusual way of handling test components. Instead of enabling or using controllers and entities you simply add them to your test plan.

I liked the way you add more summary and reporting to your test. You may enable only those reporting ways that you prefer. However I found the charting slightly difficult to use and cheap-looking. But to me this is not a big problem as I usually value table output over charts.

Overall impression is that tool is capable of helping you out with simple load tests. But I would not recommend it for testing big and complex systems. You better look at commercial competitors.

In case when your financial restrictions prevail this is a good choice. But be prepared to spend more time for generating and supporting the tests than it would have been should you use a more advanced tool.

Load testing

Today I have written to a customer on what we usually do about load/performance testing. I decided to put it here should I need it again as well as to help you organize our load testing activities.

***

Load testing usually start from learning about the customer's problem. Every load testing session is unique and this is very important to start moving right direction from the beginning.

After learning the purpose of testing we develop testing strategy. Test strategy include the definition of load scenarios (how many virtual users are to be involved, where to put the load in the system (at which hierarchy level), what type of scripts to use (simulative or fast), etc.)

Then we start working on scenarios. Scenarios are the individual scripts which will be played back by virtual users. Customer input is vital because domain knowledge is the key to creating the right set of scenarios.

Then we execute several test runs with different load level to see how server reacts and to make sure that load is adequate and test results are not affected by configuration or communication issues. Usually it takes from 5 to 10 runs to finalize test plans and conditions.

A very tight communication with development representative and deployment team is very important. We have to make sure the testing is performed in clean conditions, whereas no one else can interfere and alter test results.

During the testing we usually set up server-side monitors to find the resource-bottleneck, if any.

***

Having done all above you will make sure that the problem is solved, not just a load test performed.

LoadRunner crashes on recording?

Here is the cure...

1. Control Panel / System
2. Advanced
3. Performance Settings
4. Data Execution Prevention
5. Turn on DEP for essential Windows programs and services only.
6. Restart!

That's it :)

Load testing is not easy

Recently I have convinced in it myself yet another time. The task looked as simple as generating load on a site accepting several files for parameters. As usual it was easy to record and parametrize scripts, debug and plan the first load.

The difficulties appear during the testing. Every time when I approach load testing the results kind of amaze me. They are not what I would expect. This time was no different. I was amazed and did at least 5 times more test execution than planned originally.

Thanks to my managerial experience I have lend myself some time for contingencies. So it worked well and I met the deadline :)

Every time you plan for load testing keep in mind that this is investigation rather than determined work in sense that we put in it when we approach planning.