Sunday, October 21, 2012

J-Meter, first impressions

Today I had a chance to take a look at J-Meter, a load tests tool, which most important feature is that it's free.

It took up to 10 minutes of my time to generate a simple HTTP request load test. In this sense, I would not call the tool UI intuitive. But once you've done your first test you should have no problems in the future. It just takes time to get used to unusual way of handling test components. Instead of enabling or using controllers and entities you simply add them to your test plan.

I liked the way you add more summary and reporting to your test. You may enable only those reporting ways that you prefer. However I found the charting slightly difficult to use and cheap-looking. But to me this is not a big problem as I usually value table output over charts.

Overall impression is that tool is capable of helping you out with simple load tests. But I would not recommend it for testing big and complex systems. You better look at commercial competitors.

In case when your financial restrictions prevail this is a good choice. But be prepared to spend more time for generating and supporting the tests than it would have been should you use a more advanced tool.

Automated testing (to be or not to be)

Many want to do automated testing. Not many start doing so. Few succeed. The reason why hides in the way we used to do our work and how we assess the performance. We prefer to see the results straight away, on the spot, now. Whereas it takes a while for automation to start paying back. The difference in time and efforts before it starts doing so can be recognized as a waste. In other words you would get an immediate result if you'd manually test your application instead of re-writing map files after UI change. Defects found and removed from the system is what is being scaled against investment in automation that will start paying back in months.

On the other hand, having automated tests saved many projects putting them back on track timely. The beauty of automation is that you so not need to make trade-offs, select regression tests carefully, adjust plans, and  procure more resources if you found that more testing is required. You just run them all (except for the situation when there are too many or your resources are limited).

The dilemma grows even bigger when you project size grows. For big projects this is even more difficult to find resources for automation, the lead time is even longer, however the outcome is usually more tangible. You should have  firm intention, high motivation, and be prepared for a challenge if you want your automation to start paying back one day. Below are simple recommendation for those who started on this path:

- Get commitment from the management, sell off the idea.
- Set right expectations in stakeholders (automation will start paying back when we reach X% of tests automated; usually about 30 if you start from the most important ones).
- Plan efforts for automation (don't expect people to spend their free time due to pure enthusiasm).
- Stick to the plans, treat automation goals as project goals, equal in the priority.
- Allow enough contingency buffers in your plans, so automation will not suffer just because something happened and your plan does not have any fat to cope with the situation.
- Get skilled hands to work on automation (test automation is more of programming that testing, so make sure people who do it are rather developers).
- Follow best practices in code engineering. It will save you massive efforts in maintenance. Make test code isolated from accessibility functions and GUI maps.
- Run your tests often not to allow them get outdated, so as it is not reasonable to invest in the updating. A test that sits n the shelf a year will unlikely work any more. Now imagine that you have got thousands shelf-ware tests.
- Use data-driven and control-driven approach.
- Involve more people by implementing pseudo-languages (a command driven test with a  language that anyone can easily use to create test sequence; such a sequence drives the test code responding to the commands and data parameters in it).
- Consistently deliver the message that automation is not an immediate ROI. Nonetheless it's got one of the biggest ROI in the software industry.

If you follow this you would save yourself tonnes of nerves.

Good luck!

Tuesday, October 2, 2012

Meteor approach

Yesterday I've finally fulfilled my wish and tried out Meteor. This is a new framework for developing web applications using JavaScript as the main language.

I just loved how easy it is to create a site and how few lines of code it requires. I also liked the way how it makes the changes immediately reflected in the running application.

But the most valuable tome is the ability to move the functionality between client and server with no efforts. There are actually three folders: one for server, one for client, and one for both. Moving stuff between them you change the architecture of your system. This may be very interesting for balancing complex systems load. For example, you have a client-server system which should support 1000 simultaneous users. You are kind of limited on the amount of horse power under the hood, the server is rather weak. So you may decide to move as much of computation as reasonable to the client side. In the traditional system it would require a careful planning and decent programming. In Meteor this is just a matter of Cut/Paste the Try. So you can easily adopt trial and error method.

Meteor is built on templates. Everything is defined as a template and it requires quite a bit of abstract thinking if you want to design a new system from scratch. be prepared to start thinking in different categories.

But Meteor is not only a framework. It's a Cloud. It takes just one command line call to deploy your application in a cloud. very easy and robust.

Meteor uses a non-relational database in the back-end. I hope they will implement connection to other database engines in the future.

Of the disadvantages I would mention using JavaScript, which is not object-oriented (despite some may argue, f.e. Hopefully they will consider new replacement from Microsoft (TypeScript). Also there is only one database engine available yet (correct me if I am wrong). which also puts limitations on the system. We know that changing DB engine is not an option for many very complex systems. Allowing use of engines like Oracle, MS SQL Server, and DB2 would make that new framework more attractive to the enterprise software produces, including my company.

System is still in development, so don't rely on it if you need to make a commercial package. The documentation is scarce but you may find several examples and tutorial videos on the Internet. It is definitely good enough for evaluation and playing around with it. Believe me it's a pure fun :)

If you want to read more, just visit their site at