Category: Technology

Choosing between table mappers or DeclerativeBase when working with SQLAlchemy

October 18th, 2011 · · Python No comments - Tags: , , , , ,

I have recently been looking how to create a clean Domain Driven Design architecture using SQLAlchemy and specifically how to map domain classes to tables. This post will discuss the two techniques available and why to choose one over the other.

Lets start with looking at the two strategies available. SQLAlchemy offers a Data Mapper out of the box. It is also possible to use an Active Record approach by using the Declarative Base ORM extension.

The Active Record approach is a very attractive way of working. It adds metadata to the domain objects that tells the engine how to save them to the database. It’s quick and easy to use and avoids extra classes or configurations that handles the database mapping. It’s a perfect fit for most web applications. The main drawback is that storage logic is tightly coupled with the domain objects. This can make the domain difficult to work with over time. It also means that any changes required in storage logic will force changes into the domain breaking the Single Responsibility Principle. Active Record will also prove very costly if the project finds that a change of storage technology would be beneficial. A not to unlikely event with such a variation in industry strength storage technologies available.

Data Mappers provides high decoupling by keeping all meta data for mapping objects to tables separate from the domain. The mapping can be placed in its own module which makes it easy to divide the domain into coherent modules with no need to cater for storage concerns. It also makes it easy to model more advanced relational concepts in a large database. Since it allows for complete separation of concerns for domain and data access it is also easy to change or swap out the data storage technology completely without any needs to change the domain. It does however increase the complexity of the application since it adds an extra module/package.

If the project in question is going to use a clean implementation of Domain Driven Design only the Data Mapper works. It is also the only possible choice if it is likely that the project will be changing storage technology.

If, however, the project is a traditional web application with a domain model that is simple or close to non-existing then Active Record is a good choice. It will provide more speed of development. Making a clear distinction between domain logic tests and data access tests will make it easier to later extract data access out into Data Mappers if needed.

Exploratory testing to save time

July 25th, 2011 · · Testing One comment - Tags: , , ,

I am currently working with a rather complex e-commerce system. It is a web application archive which is using Spring 2.5 throughout to load wire the application. I need to expose some of its functionalty as a REST API to other applications. I am using Spring 3, Jersey and javax.inject to make this possible.

But to understand and verify that my interaction with the underlying e-commerce system works I have to redeploy the whole application to a server. This is very expensive. Even when using such a brilliant tool as JRebel from Zero Turnaround, which saves me from restarting the server. Many of the integration points are built up of several small steps that needs to be done in order to ensure that the function works. To debug errors, which happens often, I have to redeploy, test and redeploy.

To speed this process I am using a technique called exploratory testing. Instead of writing production code which runs on a server and then test this production code I create unit tests that acts against the e-commerce APIs I need to use. This removes the need to redeploy and restart a web server every time I discover an issue with the integration.

This is made possible using two simple constructs. One is the @RunWith annotation provided in JUnit 4. It enables the tests to run with a different test runner. Spring provides a SpringJUnit4ClassRunner that can load a Spring application context into the normal JVM. The other is the Spring annotation @ContextConfiguration which is used to point the test runner to the correct Spring configuration to use.

Exploratory testing has other advantages too.

The tests are not throw away tests. They live their own life in a maven project which is part of the larger reactor build. It is executed automatically in our CI server prior to all other projects. If the e-commerce system is changed, or we need to upgrade it, we only need to run the exploratory tests to verify if our application will work.

Another great benefit is that the tests also allows provides executable documentation. They help clarify assumptions that are used in the production code and tests which otherwise would have to be written down as text documentation or comments. And we all know what usually happens with comments and text documentation.

The conclusion is that whenever you are faced with understanding a complex third party API exploratory testing almost always pays of. If not for time saved due to redeployment and restating of an enterprise server so at the very least for regression testing and documentation.

Managing Transitive Dependencies in Maven

May 4th, 2011 · · Java No comments - Tags: , ,

Have you ever got a ClassNotFoundException when running tests and yet the dependency that contains the jar is clearly in your pom file? This is most commonly due to collations in transitive dependencies. If Maven cannot decide which version of a dependency to use it will not import it. It will expect you to sort the problem out first.

Handling transitive dependencies should always be done pro-actively. And there is some really good tooling for it. Being a IntelliJ/IDEA developer my self it saddens me to say that in this instance my favourite IDE is outdone by NetBeans. NetBeans uses Maven pom files as it’s project descriptors which makes it very easy to open the project.

When the project is opened in NetBeans you will find the pom.xml file located under project files in the project navigator. Once opened you will find an item named “Show Dependency Graph” in the context menu. This will render a graphical representation of all the projects transitive dependencies.

If any of the dependencies has a red top left corner there is a version conflict. Resolving it is done quickest by opening the context menu for the dependency and selecting “Fix Version Conflict”. Unless you have a specific preference that differs from the default suggestion in the dialogue it is usually fine to got with the default solution.

If any of the dependencies has a yellow top left corner there are more then one version of the dependency requested. Maven will always choose the latest version in the list. This can, and should, however be overridden using the exclusions tag. I tend to take control over all such dependencies buy making sure I clearly choose which one should be included. You have to do this manually by excluding the dependency from each of the dependencies where the version you do not want is included and then ensure that the dependency requiring the correct version is the only one still including it. Sometimes it’s better to exclude it as a transitive dependency altogether and add it as a direct dependency.

Whenever a new dependency is added to the project I make sure to perform the same procedure to ensure that I have full control of the dependencies in my project.

I would be very pleased to find such a great tool in IDEA as well, then I wouldn’t have to run two IDEs at the same time, they do require a lot of resources.

Making Jersey and Spring play together

April 4th, 2011 · · Java No comments - Tags: , ,

I am currently working with a REST application that uses JSON as it’s data format. The Jersey framework offers this very nicely out of the box using annotations. The application is a front end that will have several different back ends depending on deployment. To simplify this I have chosen to use JSR-330, dependency injection from Java. The implementation is the Spring Framework.

It took me some time however to find out how to get Spring and Jersey to play together. I had to follow the following steps to get it all to run. I am using Maven 3 as the build and dependency manager.

To instantiate the Spring application context a specific servlet provided by Jersey is needed. To get the jar file we need to add the following dependency to our pom file.

Now we can replace the traditional Jersey servlet, com.sun.jersey.spi.container.servlet.ServletContainer, with one that also creates an application context, com.sun.jersey.spi.spring.container.servlet.SpringServlet in web.xml. Now the standard web.xml context configuration will be picked up:

With these changes done the Spring framework works as it always does.

AMQP Hello World as an Exploratory test

February 15th, 2011 · · Testing No comments - Tags: , , , ,

We have taken the decision to use Rabbit MQ as our internal message queue. To understand the way that Rabbit MQ talks with Java applications I decided to create a little Hello World application. Using Rabbit MQ’s Java API guide I created the exploratory test below.

The complete code, in a Maven project, can be found on my github repository here.

The test starts with setting up the connection to the Rabbit MQ broker which is running on localhost. Since the client comes with sensible defaults the connection does not need to be configured with anything but the name, type and durability of the exchange. A queue is then bound to the channel to enable us to send messages.

The actual test only details the steps needed to verify that the message sent matches the messages received. The implementation will be discussed in each of the supporting methods:

To send a message to the MQ we simple publish it to the given exchange using the channel:

We then pick up the message using a consumer. When implementing this in production code the producer and the consumer should not be running in the same thread since both are blocking. But since we are only testing the availability of the MQ the blocking is actually working or out advantage.

The last step of the test is to assert that we received the same message as we sent:

When the test is done we need to close the channel and the connection:

The benefits of using this approach to understand new technology is that the tests can be put to use to verify the integrity of the libraries and the MQ used when running automated integration tests rather then just writing throw away code.

If you find the structure of the test code different you can find an explanation in this article which discusses the drivers behind the design as well as the given-when-then pattern used in the test method.