The Covariant Return Type Abyssal

The possibility to define a more specific return type when overriding a method has been introduced together with generics long time ago. The number of casts needed has dimished dramatically since these days, and our code is now a lot more readable. But there are also some nasty pitfalls you won't recognize before you fall down. In this article on DZone I will describe some of those traps we stumbled about lately. 

Give me a ping Vasily, Part I

Writing integration tests is not an easy task. Besides the business complexity, it is also often not easy to set up a suitable test scenario. This article will show an easy way to write integration tests for common problems in Eclipse based client-server projects.

It started in a meeting with project management. The leader of a large-scale SmartClient project asked: "How come we have so many unit tests, but only a few integration tests?". I guess the main problem was that the application was based on large, interweaved (host) database tables, and we were not allowed to alter any data...even in the test environment. So in fact, we could not set up any test data, we were forced to set up our test cases on the existing data.

But it was living data, and so it was subject to change. Means: even if you had set up a test based on that data, it was very expensive to maintain. The project leader claimed: "We need a kind of integration test that is easy to write, needs no maintenance and runs in any stage from development to production". That's three wishes at once. I ain't no Jeannie in a bottle, man ;-)

But let's be serious for a second. What kind of common problems do we have in large multi-tier client-server projects?

• The components of the application are loosely coupled, so a lot of problems do not occur until runtime. Especially in a distributed development environment.
  
• The environment-specific part of the application is extracted into stages. So if a developer is not careful, he might forget to provide the needed information in all stages.
• Infrastructure: In a large distributed system there are a whole bunch of things that can go wrong, e.g. firewall problems, missing entries in the host file, etc.

So, if business-motivated integration tests are hard to write, maybe there's a way to write tests for the problems mentioned above? Well, the core of current multi-tier architectures is usually based on (local and remote) services. By successfully calling a service we can prove that...er, yo what?!? Well for local services it proves that the service is registered and the corresponding plugin is active. For remote service it also means, that we can connect from client to server. Services often need other resources to fulfill there job e.g. other services, databases, etc. If these resources are not available, the service itself is also not - or only partially - available. So our call to the service should also call all resources it depends on. And finally, if any call fails, the failure message will give you a hint on the cause.

The Eclipse Riena Project provides a little framework for writing and running such tests: the Ping API. The main interface is IPingable which defines a non-business service dedicated for implementing the test described above:

public interface IPingable {

 PingVisitor ping(PingVisitor visitor);

 PingFingerprint getPingFingerprint();
}

The ping() method defines the dedicated service call. The PingFingerprint is needed for reporting and to avoid cycles. So, any services that wants to get pinged has to implement that interface. The implementation is quite easy:

public PingVisitor ping(final PingVisitor visitor) {
 return visitor.visit(this);
}

public PingFingerprint getPingFingerprint() {
 return new PingFingerprint(this);
}

Not that hard, is it? And for the lazy ones (like me ;-) there is the class DefaultPingable which you can derive from. But here comes the tedious job: all service that wanna get pinged (that's usually ALL services) have to implement IPingable. Means all service interfaces must extends IPingable:

public interface IRidiculousService extends IPingable {

We already discussed the implementation. If you can derive, it is just

public class RidiculousServiceImpl extends DefaultPingable implements IRidiculousService {

Until now we've only pinged a single service. What about it's dependencies? What about resources like databases or backend server? Let's take the following silly architecture as an example:


On the client side we have a couple of service. Some of them are local, others (dashed) are stubs for remote services. On the server side there are the service implementation for the client stubs, which itself may call other services, a database or other backend resources like a mailserver. How are they getting pinged? That's the PingVisitors job. Remember the implementation of ping:

public PingVisitor ping(final PingVisitor visitor) {
 return visitor.visit(this);
}

When visitor.visit(this) is called, the PingVisitor inspects the service for member variables that implement IPingable (using introspection), collects and then pings 'em.

Let's take the SeriousService from our silly architecture for example:

public class SeriousServiceImpl extends DefaultPingable implements ISeriousService {

 private IRidiculousService ridiculousService;

 @InjectService
 public void bind(IRidiculousService ridiculousService) {
    this.ridiculousService = ridiculousService;
 }

 public void unbind(IRidiculousService ridiculousService) {
    this.ridiculousService = null;
 }
 ...
}

The IRidiculousService is injected and stored in a member variable. On ping() the visitor will find the ridiculousService and pings it also. Means: ping() is called recursive on all IPingables found.

But I don't use injection, I don't keep services in member variables. That's evil. I fetch service when I need 'em:

public void doSomeRidiculousStuff() {
 IRidiculousService ridiculousService = Service.get(IRidiculousService.class);
 ridiculousService.dontWorryBeHappy();
}

No problem. Just provide a method getAdditionalPingables() that returns all IPingables you are interested in:

private Iterable getAdditionalPingables() {
 List pingables = new ArrayList();
 pingables.add(Service.get(IRidiculousService.class));
 ...
 return pingables;
}

If the PingVisitor finds a method matching that signature, it will ping all IPingables in that Iterable. The drawback of this way is that you have to maintain this list of services.

Databases and other resources
By now we can ping all client and server side services. But what about other resources like databases or e.g. the mail server. Wouldn't it be nice if you could ping them also? Sure you can: The PingVisitor inspects the IPingable if there are methods called void ping...(), where the first character after ping must be an upper case letter e.g.

private void pingDatabase() {
 ...
}

In pingDatabase() you have to check if the database is available, e.g. make a select for a row with a certain ID. It doesn't even matter if that ID exists: If the select returns a result (that might be empty), it proves that:

• the database exists
• you can connect to it
• the table exists

If that's not enough, you can write a stored procedure (named ping ;-) that can perform all kinds of checks. And your pingDatabase() method just calls the stored proc.

In the same way, you can check all kind of resources e.g. for a mail server you could check if a helo/quit succeeds.

Ping'em all
So now all services implement IPingable... but we haven't pinged anything yet. How do we do that? Well, you could write a JUnit test that collects all pingable services, creates a PingVisitor and calls ping() on them. Or you could use the helper class Sonar which does exactly that for you. Or... you could use the Sonar User Interface which will be introduced in the next part.

Regards
Ralf

Give me a ping Vasily, Part II

This is the second part of a two-part article on Ping. The first part gave you an instroduction to the Riena Ping API. This second part will show you the usage of the Sonar UI to run ping tests.

The first intention was to use ping for automated, JUnit-driven, integration tests. But that did not work out for two reasons:

• The build server supposed to run the tests was a unix system, but the client was targeted for windows.
• The build server was located in the intranet and was not allowed to connect to the application server.

During discussion of these problems, one of the infrastructure guys said: "Wouldn't it be cool, if we could use those ping tests as part of our daily system check?" Bingo! That's the idea. Instead of preparing a client-like test setup, just integrate the tests into the client. And that's what Sonar is all about: A UI dedicated for running ping tests, that could be easily integrated into any (Riena-based) client product:


No need to say that it was inspired by the JUnit-Runner, eh? ;-) It consists of a single plugin (org.eclipse.riena.example.ping.client) which provides the Sonar Submodule and the main logic, and also menu and command handler for bringing up the module. The plugin is currently part of the Riena example application.

So that's what Sonar basically does: If you press the start button...

• it collects all services that implement the IPingable interface
• it creates a new PingVisitor and pings all services
• it renders the result tree and provides any failure messages

So usage is quite simple: Just run the tests. If everything is green, your system is elementary ok and you can start functional testing. If it's red, you have to analyze the failure message and see what's wrong. Let's do this by example. Take the following silly architecture from the first part:

On the client side we have a couple of services. Some of them are local, others are stubs for remote services (dashed). On the server side, there are the service implementations for the client stubs, which itself may call other services, databases or other backend systems like e.g. a mail server. Now let's start a ping and see what happens:


Oops, all remote services are red. What's wrong?!? Let's have a look at the failure message:

org.eclipse.riena.communication.core.RemoteFailure:
Error while invoking remote service at
...
Caused by: java.net.ConnectException: Connection refused

Alright, in our silly example I just forgot about to start the server. But in reality, a couple of things could be the cause: Firewalls, wrong host names, network failure, whatever. But here the solution is quite simple: just start the server and try again:


Hmm, the remote services have been called successfully, but all pings to the database failed. What does the failure message say?

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at
...
Caused by: java.sql.SQLNonTransientConnectionException:
Connection authentication failure occurred. Reason: Invalid authentication..

Oh, we've used a wrong database user. Smells like a stages problem. The developer has used the correct database user in development (stage), but forgot to set up the appropriate user for test, resp. productions. So let's fix the stage by setting up the correct DB user and try again:


Aah, nicely green at last. So all system components from client to database are basically available now ;-)

Conclusion
Sonar provides an easy way to run ping tests directly form your client product. The ping tests doesn't help you on testing business functionality, but give you a tool for checking basic integration and infrastructure problems... at almost no cost :-)

Give me a ping, Vasily.
One ping only, please!
Captain Ramius
Boat Red October

A bad name will kill

Javadoc is quite helpful if you have to deal with a new API. It explains what methods are doing, and the role each class is playing. But let's be honest: How many times have you ever read the javadoc upfront? After studying some basic examples I usually choose classes and methods by name: Hey, this sounds like the functionality I need. Then I check the javadoc to make sure that it actually is what I want. But this requires that names are well chosen...

Bad names do not communicate a method's purpose or, even worse, they communicate the wrong intention. So what are the reasons for poor naming? Well, one reason is for sure careless naming. The one who designs an API knows exactly what it does, so, with a certain lack of sense, he won't even recognize that it's badly named. Another reason is refactoring or redesign that changed the intention of a method. Means: the name was wisely choosen in the first place, but after changing the methods semantics, the name has not been adapted in order to reflect its new meaning :-|

You will find poorly named methods all over the place, so let's take a look at a prominent example: Thread.interrupted(). So what does this method do? It queries the thread's interrupted state, doesn't it? No, that's what method Thread.isInterrupted() does. Yeah no, is clear... so what the heck is interrupted() doing? Have a look at the javadoc:

public static boolean interrupted()

Tests whether the current thread has been interrupted. The interrupted status of the thread is cleared by this method. In other words, if this method were to be called twice in succession, the second call would return false (unless the current thread were interrupted again, after the first call had cleared its interrupted status and before the second call had examined it).
A thread interruption ignored because a thread was not alive at the time of the interrupt will be reflected by this method returning false.

So basically it resets the threads interrupted state, and returns the value it had before reset. This kind of indivisible operation is used in operating systems (e.g. to implement mutual exclusion) and is usually called testAndSet(). Java's AtomicX classes also have such a method called getAndSet().

Our interrupted() method is a derivate which does not set but clear the state. This operation is usually called testAndClear(). Now that we are aware of what the method really does, we can give it a better name. Intuitionally one would call it testAndClearInterrupted(), but in analogy to AtomicX.getAndSet() we better call it

  public static boolean getAndClearInterrupted()

Now the method communicates what it does. No misunderstanding, no need to look at the javadoc.

So here is my recipe on naming methods: If a method's name is not crystal clear...

1. give the method any (foolish) name, e.g. foo or yetToName
   
2. write javadoc for the method and describe what it does
3. read your own doc and find a name (if it's still not clear, go back to 2.)
4. rename your method

I can hear someone whine: My API is already widely used, changing the name will break existing code. That's a lame excuse. Keep your old bad named method, make it use your new one and mark it deprecated, e.g.

  /**
  ....
  * @deprecated use {@link #getAndClearInterrupted() getAndClearInterrupted()} instead
  */
 public static boolean interrupted() {
    return getAndClearInterrupted();
 }

Didn't hurt that much, did it? So, there is no reason to live with bad named methods.

Best regards
Ralf

A bad wound may heal, but a bad name will kill.
-- Scottish Proverb

Typesafe maps for arbitrary content

Data containers like Vectors and Hashtable are part of Java from the very early beginning. The problem is that they are bags for arbitrary data, means the type of data they deal with, is the overall super class Object. That's no problem if you store values, but on retrieval you need to now the type you put in and cast it back to the original type. Then Generics had been introduced, and all collection classes were refactored to be usable in a type-safe manner. You were now able to restrict the collection to one (or, in the case of maps two) types, and the number of cast reduced dramatically.

But sometimes you need a container for arbitrary data. Typical usage scenarios for this are context- or configuration-data. An example for that would be OSGi's BundleContext.registerService() method, which uses an untyped Dictionary to provide config data. So the one who provides, and the other who consumes the data, need to know about the type of each date. This is usually some kind of contract: Parameter host has type String, port is an Integer. A neat API will at least provide constants for the keys, like e.g.

public static final String PARAM_HOST = "host";
public static final String PARAM_PORT = "port";

So providing and consuming a value would look like something like that:

Dictionary properties = new Properties();
properties.put(PARAM_HOST, "www.blogger.com");
...
String host = (String)properties.get(PARAM_HOST);

But if we know that each key is associated with a certain type, why not let the key provide that type? With the help of Generics we can define a typed Key-class:

public class TypedKey <T> {} 

Now we are able to define typed keys:

public static TypedKey<String> PARAM_HOST = new TypedKey<String>();
public static TypedKey<Integer> PARAM_PORT = new TypedKey<Integer>();

What's left is a kind of container that deals with this kind of key. Since there's no need to create a full-blown map, we'll create some tiny class called TypedData. What would getValue() look like? Well, what we'd like to have is that the return type is specified by the type of the key:

public class TypedData {
...
public <T> T getValue(TypedKey<T> key) { ... }

It's the same thing for setValue(). The type of the value to set is given by the key:

    public <T> void setValue(TypedKey<T> key, T value) { ... }

Here comes the implementation:

public class TypedData {
private Map<TypedKey<?>, Object> map = new HashMap<TypedKey<?>, Object>();

public <T> void setValue(TypedKey<T> key, T value) {
   map.put(key, value);
}

@SuppressWarnings("unchecked")
public <T> T getValue(TypedKey<T> key) {
    return (T)map.get(key);
}
}

Since we want to store multiple types, the Map used must accept any kind of TypedKey and Object as values. The implementation of the setValue() is easy. In getValue() we have to cast the value (that's why we need to suppress the unckecked warning), but it's safe to do so, since our implementation of setValue() guarantees that the type always matches the one associated with the key. Now let's try it:

TypedData data = new TypedData();
data.setValue(PARAM_HOST, "www.blogger.com");
data.setValue(PARAM_PORT, 80);
...
String value = data.getValue(PARAM_HOST);

Feels good, eh? No cast necessary as if the map was fully typed. If we accidently use the wrong type, the compiler complains:

And what's even more useful is that your IDE now can deduce the correct type and give you content assist:

So, it looks good, it feels good. And it is type-safe! Sure about that? You better doubt it, there is still a weakness we have to eliminate. Remember: we made a cast in getValue() which we considered to be safe, since the type is associated with the key. We rely on the fact that two keys with different types retrieve different values, with other words: that keys are unique. But what if not? Hash keys are based on there hashCode() and equals() method, now let's make a naive implementation:

public class TypedKey<T> {

@Override
public boolean equals(Object obj) {
    return true;
}

@Override
public int hashCode() {
   return 3;
}
}

Ridicolous, I know. But until now, nothing prevents us from inheriting from TypedKey and do something funny like that. So let's give it a try:

TypedData data = new TypedData();
data.setValue(PARAM_HOST, "www.blogger.com");
data.setValue(PARAM_PORT, 80);
String value = data.getValue(PARAM_HOST);

What will now happen if we run that code? Tada:

Exception in thread "main" java.lang.ClassCastException:
java.lang.Integer cannot be cast to java.lang.String

Ooops. So we have to make sure that every key is unique. The implementations of Object.hashCode() and equals() are based on object-identy, that's quite perfect for us. So let's make sure nobody can tweak that by making them final:

public class TypedKey<T> {

@Override
final public int hashCode() {
    return super.hashCode();
}

@Override
final public boolean equals(Object obj) {
    return super.equals(obj);
}
}

Now we are done :-)

That's it for today
Ralf


P.S. Some remarks on the implementation:
My first idea was to use an interface for the key definition and enums (which implement that interface) for key constants. I like enums as constants since they are lightweight and type-safe. Alas, you can't use generics with enums. Since the sole reason for the interface was that you cannot inherit from enums, I dropped the interface and used a class instead.