Category: Testing

SLAMD performance testing framework

I am currently working on implementing SLAMD Distributed Load Generation Engine in favour of JMeter for performance testing. JMeter has served us well and given us some very quick results to work with. Unfortunately we are now finding JMeter rather limiting since we need to test a networked application with several different custom built servers. We need to be able to monitor system load, such as CPU, memory and network load, in combination with tests data. SLAMD comes with such functionality out of the box.

SLAMD was initially developed at Sun to test LDAP servers (ldapd) but has since been open sourced and further developed to support different networked applications such as HTTP, SMTP, IMAP and POP.

It is an old and in certain aspects a very dated code base though. As an example the HTTP servlet presentation tier uses no JSP. Instead it creates all the view code in the servlets or servlet helpers.

Yet it does seem to be the best tool available on the market. It ships with web administration and reporting interface, test client performance monitoring, application server monitoring and extensible APIs.

Right now I am working on extensions that maps well to our domain and hope to have a running performance test system up soon with custom test jobs that are easy to administer and maintain. It is good to be back in the coding trenches again after to much manual performance testing using JMeter. And to be able to automate what was previously done manually is always a good feeling.

If you have any previous experience working with SLAMD that you would like to share, please post a comment/link.

Robotium testing for Android

Yesterday I went to a breakfast seminar at Jayway’s office here in Malmö. Hugo Josefson and Renas Reda showed us a new way to write tests for Android called Robotium.

The framework aims to remove the pain of writing traditional Android instrumentation tests. When demonstrated it is striking how easy it is to create UI tests using Robotium.

Hugo showed us an example test case written using normal Android instrumentation. He then proceeded to show a similar test written using Robotium. The Robotium test was about a tenth of the amount of code required to create a traditional Android test. It also removes the need to have any awareness of the underlying application being tested. This makes the framework useful both as a very powerful QA tool and for creating up front acceptance tests.

Robotium is also fully integrated with Maven using Maven for Android. This makes it easy to get up and running and to automate. It also makes it easy to work in other IDEs then Eclipse.

The one thing I missed was influence of BDD. I almost exclusively use the given-when-then format when I write tests. I find that it helps increase readability. It also helps me staying focused on the domain I work with. This is because it is closer to how the customer expresses them self when discussing features.

I have started to sketch on how the main class, Solo, would look given some given-when-then syntactical sugar. The Robotium source code is on GitHub so I forked it. Feel free to check out my progress. The changes are added in the same manner Mockito have added BDD, by extending the main framework class. In Robotiums case this means there is now a BDDSolo class that inherits from Solo. I will add methods to it as and when I have time.

Please come with feedback and ideas. What is currently there have been done from a very limited grasp of how Robotium works. I hope that I will get a chance to use it soon so I can add some more substance to it. In the mean time I would like to get others feedback. Both on Robotium specific things and on general phrasing.

Me and Hugo talked about how to design some lower level APIs for Robotium, and there will be a discussion on the Robotium Google group at some point. But in the mean time, I like to discuss APIs by actually designing them and GitHub makes it easy to maintain such a discussion in code. So feel free to join in.

Writing unit tests using Groovy

Groovy can greatly decrease the level of work involved in creating unit tests for your Java code. I find that this is especially evident in the number of code lines I need to write to create the data objects that are needed for the tests. I often end up writing up towards two hundred lines of Java to create a decent amount of test data. Using Groovy I can reduce this to around 50.

This article will show how to create a unit test in Groovy that is automatically integrated into the Maven test suite and executed when running the build. I will also give a real life example to illustrate how much less code is needed when creating the supporting data for a test.

To complete the toolbox I will also illustrate how to use EasyMock to mock dependencies in a Groovy test case.

Configuring Maven2 to compile and run your Groovy tests.

First of all, in order for the tests to execute we will need to configure the Maven2 configuration to use the Groovy plug-in in it’s test phase.

To do this, add the Groovy jar dependency to the pom file and assign it the correct scope. Add the below snippet in-between the <dependencies> tags:

Then add the following snippet in between the <build> tags:

We also need to create the source folder where Maven can find the Groovy source files. The path, relative to the projects root folder, should be src/test/groovy. This is where the plug-in will look for source files by default. Maven will then put the binary files in target/test-classes.

To run a specific test with Maven, use the normal command:

Some basic language constructs

Before creating the first test case I should introduce some important language constructs. Writing Groovy don’t need to be any different then writing Java, as a matter of fact it’s possible to C&P valid Java code into a Groovy class. This would how ever not provide any benefits, so lets look at what makes Groovy so attractive.

Since Groovy is a dynamically typed language it’s not necessary to specify a data type for a variable. Instead the keyword def can be used to define it.

This can offer less typing and for narrow scopes makes the code easier to read. Watch out for doing this when working with wider scope variables however since it makes the code less understandable.

Three major differences with Java are the string, list and map handling. Strings, lists (java.util.ArrayList) and maps (java.util.HashMap) are built in to the language so creating and working with them is very simple.

First something that will be noticed in the code examples below and may be confusing. Its the use of semicolons. In Groovy semicolons are optional and I have chosen not to use them.

Strings are defined as in Java, but with the option of using def. The real benefits is when referencing other variables within a string:

The above statement would look like this in Java:

To create a new list all you need to do is:

What goes in between [ ] will build up the list, each item split with a comma. To sort the list just:

To access an item in the list:

There are several other such shorthand commands, all listed in the reference summary (see link in Further reading),

Maps works in exactly the same manner:

The key assignment is before the : and the value is after.

To access items in the map:

There are two ways to create a set:

The second species the specific set implementation rather then leaving it to Groovy.

Groovy supports properties in the language. This means that instead of creating a bean like in Java:

You write this:

Groovy then provides two constructors to all objects, the default constructor and a constructor that takes a map like structure with name/value pairs to populate the objects selected properties.

This creates a new person with the name and age properties set without the need to create a constructor that takes the specific arguments.

This sets the height property. And the same syntax is used to get a property:

The last important thing that I will bring up here is method return statements. All methods in Groovy returns something, namely the last executed statement. This can be a bit confusing at times but is quite useful as well. Look at the following code:

No return statement is needed since the last executed statement was a call to the constructor of Person. The return keyword can still be used to return values, especially if this needs to happen within a section of conditional logic or similar.

Creating your first Groovy test case

Now that the first introduction to Groovy is done it’s time to look at how to create a basic unit test in Groovy. It’s actually very simple since the base test class for Groovy, GroovyTestCase, extends JUnit’s TestCase. So the code needed to create the first test with a test method is:

The accessor for tests methods is removed since the default scope for Groovy is public. The class will be compiled using the Groovy compiler when the Maven2 test target is invoked.

The same conventions are used in Groovy as in JUnit. To create a setUp or a tearDown use this code:

Everything else with a Groovy test case is pretty much the same as with a JUnit test case.

How Groovy can help speed up test creation

I recently wrote a test case for our service/menu system The system assigns a set of services to a user. Some of the services has menu items. Two services are available to clients. One is a service resolver that resolves a users services dependent on the roles assigned to the user. The other builds a menu using the services. When I am going to test this I need to create a set of services and menu items that I can then inject into the objects that are used to provide the services. Below is a snippet of what it would look like in Java:

This is only a bit of the actual code. Now lets look at what it’s like in Groovy:

So 22 lines of Java code have been reduced to 8 lines of Groovy code. This will of cause reduce the time it takes to write said code as well. Not to mention how much easier the Groovy code communicates the intent.

Mocking with EasyMock

The above is fairly basic test writing and data building. When working with creating self testing code it’s important to also be able to mock dependencies. I always use EasyMock to mock dependencies since it’s easy to use. It is also maintained within the test cases which creates a conceptually clean design. Since it’s a technology I am familiar with I have chosen to use it in Groovy as well.

It’s easy to work with EasyMock in groovy although since static imports aren’t supported yet it’s a bit more convoluted then in Java.

In order to work with EasyMock we need to add the dependency to the pom.xml file like below:

I will now return to the tests from the example above where I created test code for the service/menu system we use. Since it’s a web based system it has dependencies on the HttpServletRequest, HttpServletResponse and the HttpSession interfaces. In order to mock the three interfaces I first need to create three mock controls:

I then create the mocked implementations like this:

Now I need to record what behaviour is expected of the mock objects:

Next is to replay the mock object:

Now I can use the objects to test against. When I have tested against them I can also verify that the expected calls were made to the interfaces by using the verify operation:

To understand more about how EasyMock works for both Java and Groovy there are good documentation at the EasyMock site at http://www.easymock.org. The instructions are easy to understand, especially if you follow the examples in detail first.

Further reading

The Groovy Reference Summary is a handy PDF document that is well worth printing and have on the desk when working with Groovy: http://docs.codehaus.org/download/attachments/2715/groovy-reference-card.pdf.

The Groovy homepage is where Groovy lives and where you will find most documentation you need: http://groovy.codehaus.org/.

The EasyMock website for all things to do with the framework: http://www.easymock.org.

P.S I have, after writing this article, found out that Groovy now supports static imports. This is great news and I am looking forward to using it!