Category: Testing

Exploratory testing to save time

I am currently working with a rather complex e-commerce system. It is a web application archive which is using Spring 2.5 throughout to load wire the application. I need to expose some of its functionalty as a REST API to other applications. I am using Spring 3, Jersey and javax.inject to make this possible.

But to understand and verify that my interaction with the underlying e-commerce system works I have to redeploy the whole application to a server. This is very expensive. Even when using such a brilliant tool as JRebel from Zero Turnaround, which saves me from restarting the server. Many of the integration points are built up of several small steps that needs to be done in order to ensure that the function works. To debug errors, which happens often, I have to redeploy, test and redeploy.

To speed this process I am using a technique called exploratory testing. Instead of writing production code which runs on a server and then test this production code I create unit tests that acts against the e-commerce APIs I need to use. This removes the need to redeploy and restart a web server every time I discover an issue with the integration.

This is made possible using two simple constructs. One is the @RunWith annotation provided in JUnit 4. It enables the tests to run with a different test runner. Spring provides a SpringJUnit4ClassRunner that can load a Spring application context into the normal JVM. The other is the Spring annotation @ContextConfiguration which is used to point the test runner to the correct Spring configuration to use.

Exploratory testing has other advantages too.

The tests are not throw away tests. They live their own life in a maven project which is part of the larger reactor build. It is executed automatically in our CI server prior to all other projects. If the e-commerce system is changed, or we need to upgrade it, we only need to run the exploratory tests to verify if our application will work.

Another great benefit is that the tests also allows provides executable documentation. They help clarify assumptions that are used in the production code and tests which otherwise would have to be written down as text documentation or comments. And we all know what usually happens with comments and text documentation.

The conclusion is that whenever you are faced with understanding a complex third party API exploratory testing almost always pays of. If not for time saved due to redeployment and restating of an enterprise server so at the very least for regression testing and documentation.

AMQP Hello World as an Exploratory test

We have taken the decision to use Rabbit MQ as our internal message queue. To understand the way that Rabbit MQ talks with Java applications I decided to create a little Hello World application. Using Rabbit MQ’s Java API guide I created the exploratory test below.

The complete code, in a Maven project, can be found on my github repository here.

The test starts with setting up the connection to the Rabbit MQ broker which is running on localhost. Since the client comes with sensible defaults the connection does not need to be configured with anything but the name, type and durability of the exchange. A queue is then bound to the channel to enable us to send messages.

@Before
public void setUpConnectionAndChannelAndExchange() throws IOException {
  ConnectionFactory factory = new ConnectionFactory();
  connection = factory.newConnection();
  channel = connection.createChannel();
  channel.exchangeDeclare(EXCHANGE_NAME, TYPE, DURABLILTY);
  queueName = channel.queueDeclare().getQueue();
  channel.queueBind(queueName, EXCHANGE_NAME, ROUTING_KEY);
}

The actual test only details the steps needed to verify that the message sent matches the messages received. The implementation will be discussed in each of the supporting methods:

@Test
public void receiveSentMessage() throws InterruptedException, IOException {
  givenMessageIsSentTrhoughConfiguredChannel();
  whenTheMessageIsRetrivedFromTheChannel();
  thenTheReceivedMessageShouldMatchTheSentMessage();
}

To send a message to the MQ we simple publish it to the given exchange using the channel:

private void givenMessageIsSentTrhoughConfiguredChannel() throws IOException {
  channel.basicPublish(EXCHANGE_NAME, ROUTING_KEY, null, MESSAGE.getBytes());
}

We then pick up the message using a consumer. When implementing this in production code the producer and the consumer should not be running in the same thread since both are blocking. But since we are only testing the availability of the MQ the blocking is actually working or out advantage.

private void whenTheMessageIsRetrivedFromTheChannel() throws IOException, InterruptedException {
  QueueingConsumer consumer = new QueueingConsumer(channel);
  channel.basicConsume(queueName, AUTO_ACK, consumer);
  QueueingConsumer.Delivery delivery = consumer.nextDelivery();
  message = new String(delivery.getBody());
  channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}

The last step of the test is to assert that we received the same message as we sent:

private void thenTheReceivedMessageShouldMatchTheSentMessage() {
  assertEquals(MESSAGE, message);
}

When the test is done we need to close the channel and the connection:

@After
public void closeChannelAndConnection() throws IOException {
  channel.close();
  connection.close();
}

The benefits of using this approach to understand new technology is that the tests can be put to use to verify the integrity of the libraries and the MQ used when running automated integration tests rather then just writing throw away code.

If you find the structure of the test code different you can find an explanation in this article which discusses the drivers behind the design as well as the given-when-then pattern used in the test method.

Test driving Tornado – and getting instant feedback

A little while back I wrote about test driving Tornado development. This is all good and well, but how about getting instant feedback? If you are developing on a Mac or a Linux machine this is certainly possible. I found this great blog post on how to set up a change monitor on a directory that will alert you on test failures using Growl or Notify. Thanks Rodrigo Pimentel for the write up.

Test driving Tornado development

The past week I have had the pleasure to write a small web based system in python. Being a devout TDD practitioner the first step in learning python was of cause to understand the unittest module. Next thing was to see what mocking frameworks are available. To my delight I found that mockito is available for python as well (here).

The application I’m working on is a web service API to access our application using Cloud to device messaging (c2dm) from Google. To provide the web API I am using the Tornado framework.

As web frameworks goes they are usually tricky to test drive. Not so with Tornado. It ships with it’s own testing framework that allows the developer to set up a round trip request/response cycle within a unit test using the tornado.testing module. Given clear module boundaries and mockito to stub out dependencies it’s an easy thing to test drive Tornado web application development.

The tornado.testing module comes with a couple of very useful classes. I used AsyncHTTPTestCase to test the RequestHandler classes. To make sure that only failing tests writes to the log I also inherited LogTrapTestCase.

Below is some sample code written to allow device registration with our application server. First is the test in it’s entirety and then the implementation.

The tests are in the module c2dm_registration_service_test.py:

import unittest
from mockito import mock, verify
from tornado.testing import AsyncHTTPTestCase, LogTrapTestCase
from c2dm_registration_service import C2DMRegistrationService

class  C2DMRegistrationServiceTest(AsyncHTTPTestCase, LogTrapTestCase):
    _deviceid = 'deviceid'
    _registrationid = 'registrationid'
    _http_success_code = 200
    _successfull_new_registration = '/register_device?deviceid=' + _deviceid + '&registrationid=' + _registrationid
    _successfull_update_registration = '/update_device?deviceid=' + _deviceid + '&registrationid=' + _registrationid

    def setUp(self):
        self._registration_handler = mock()
        AsyncHTTPTestCase.setUp(self)

    def get_app(self):
        return C2DMRegistrationService(self._registration_handler)

    def test_register_new_device(self):
        self.http_client.fetch(self.get_url(self._successfull_new_registration),
            self.stop)
        response = self.wait()
        self.assertEqual(self._http_success_code, response.code)
        verify(self._registration_handler).handle_registration(self._deviceid, self._registrationid)

    def test_update_registration_for_registered_device(self):
        self.http_client.fetch(self.get_url(self._successfull_update_registration),
            self.stop)
        response = self.wait()
        self.assertEquals(self._http_success_code, response.code)
        verify(self._registration_handler).handle_registration_id_change_for_device(self._deviceid, self._registrationid)

if __name__ == '__main__':
    unittest.main()

The production code is in the module c2dm_registration_service.py:

import tornado.web
from c2dm import RegistrationHandler

class C2DMRegistrationService(tornado.web.Application):

    def __init__(self, registration_handler = RegistrationHandler()):
        handlers = [
            (r'/register_device', RegisterNewDeviceHandler),
            (r'/update_device', UpdateDeviceRegistrationHandler)        ]

        tornado.web.Application.__init__(self, handlers)

        self.registration_handler = registration_handler

class BaseHandler(tornado.web.RequestHandler):

    @property
    def registration_handler(self):
        return self.application.registration_handler

class RegisterNewDeviceHandler(BaseHandler):

    def get(self):
        self.registration_handler.handle_registration(self.get_argument('deviceid'), self.get_argument('registrationid'))

class UpdateDeviceRegistrationHandler(BaseHandler):

    def get(self):
        self.registration_handler.handle_registration_id_change_for_device(self.get_argument('deviceid'), self.get_argument('registrationid'))

Added to this one more class in the mondule c2dm.py. So far it is just a stub:

class RegistrationHandler():

    def dummy(self):
        print("hello")

With this knowledge I can’t find any excuses not to test drive web development in Python. It is such an easy task and takes no time at all. It is well worth the effort even when creating pilots, which most likely will end up becoming production code in the not to distant future.

Mocking with ease using mockito

I have been using mock objects for a very long time now. I started back in the day of EasyMock 1.x. This was a great help when removing the extra complexity of an enterprise server and the libraries it used when creating unit tests. And EasyMock has evolved tremendously since then. I followed this development closely using it for several year.

However, about two years back a colleague of mine suggested I try out something new. A mocking framework that tasted as good as a well shaken cocktail. The idea spoke to me. So I decided to try out mockito.

I have never turned back since. Mockito offers a very simple way to create and verify mocks. It is transparent to if I am mocking classes or interfaces. And it provides a readable easy to use API.

I find that I mock away far more boilerplate code in my tests now then I ever used to do with EasyMock. This is since Mockito provides the flexibility to only verify the methods calls that needs to be verified and will not reports errors on methods calls that are not recorded unless told otherwise. This makes it easy to mock away even value objects and data transfer objects when this makes the tests easer to read.

To create a mock object without recoded behaviour three lines of code is required. Annotate the class with @RunWith(MockitoJUnitRunner.class). Create a field for the mock object with an annotation to define that it is a mock:

@Mock
private ClassToMock response;

That’s it, it’s ready to use. And any call to it will succeed unless specific behaviour is needed.

Mockito also has support for BDD like syntax with the BBDMock class. Since I use given-when-then as a way to create readable tests this is a great help.

The framework is also available for Python, Flex, Javascript, Scala and Perl. The list with links can be found here. How good the frameworks are for those languages I don’t know. If you do I would be interested to hear.