TDD adapted for mere mortals

I've been teaching and practicing agile for several years and there is definitely a problem with TDD: People find it very difficult to use. I believe there are certain points, either in the TDD itself or in people's interpretation of it, that should adapted (at least for mere mortals):

Writing test before code

It is definitely a very good practice to interweave coding and testing. This is what we programmers want to do; we feel the urge to test run a certain piece of code as it feels complicated. However, writing the test before code is not a natural way in many cases. For example, let's consider a BookService class. You'd like to implement its borrow(String borrowerId, String callNo) method. If you insists on writing the test first, you'll have to think very hard what collaborators the BookService object will use. It is not only difficult, but most likely incorrect. A much more effective way is to write the borrow() method first, then you can see what collaborators it needs (e.g., a BookDAO, a BorrowerDAO, a system clock and etc).

Most TDD demos don't have this problem because they work on classes that need no collaborators, for example, stacks, calculators.

Note that I am not advocating writing the complete code before writing the test; we should build the functionality in suitable steps. For mere mortals, try to implement the basic functionality first, then test it, then write more code and then more test.

My suggestion is to replace "writing test before code" with "interweaving coding and testing".

Take the smallest step that makes the test pass

I agree that we shouldn't write too much code without test-running the code. If we do, it's difficult to isolate the bug. But why always take the smallest step if we are pretty sure that it is going to work? The size of the step depends on the complexity of the code. We shouldn't take too large a step (hard to isolate a bug), nor too small a step (waste of time).

The whole idea is a well-established principle in testing: Risk-based testing. That is, we should put more effort on testing high-risk code, and less on low-risk code. Programmer's effort is the most scarce resource in software development projects. So, we should prioritize its use wisely.

My suggestion is to replace "take the smallest step possible" with "take the smallest step before you're worried with the correctness of the code".

If you aren't doing TDD, you aren't professional

This is not defined in TDD, but many people believe it. I think this is against the agile manifesto which says we should value people over process. Forcing TDD into people's throats is exactly the opposite. If people have tried but it doesn't help them, they will simply not use it. It's that simple. People should have every right to use whatever works best for them.

In fact, most programmers like testing: they feel the urge to test run the code if it gets complicated. It's just that writing the test before code is so difficult and against their nature. Therefore, our process should work with their nature, not against it.

My suggestion is to replace "every professional programmer should do TDD" with "every professional programmer should keep looking for their own best practices".

TDD helps you design the API of your code

This doesn't make much sense to me at all. The user requirement guides you in implementing your UI classes. When implementing the UI classes, you are guided to design the API of your service classes. When implementing your service classes, you are guided to design the API of your DAO classes.

The real design aspect of TDD is not about the design of the API, but to make sure your code is loosely-coupled and thus is easy to test.

My suggestion is to replace "TDD helps you design the API" with "testing helps make your code loosely-coupled".

 

Making your manual mocks resistant to interface changes

Suppose that you'd like to unit test the code below:

 public class BookService {
    private Books books;
    private Patrons patrons;
    
    public void borrow(String patronCode, String callNo) {
        Book b = books.get(callNo);
        Patron p = patrons.get(patronCode);
        if (b.isOnLoan()) {
            throw new RuntimeException("already on loan");
        }
        b.addBorrowRecords(new BorrowRecord(p, new Date()));
    }
}

public interface Books {
    Book get(String callNo);
}

public interface Patrons {
    Patron get(String patronCode);
}

You'll need to mock the two DAO objects: Books and Patrons. Usually I will hand-create these mock objects instead of using mock frameworks (in order to access the fields of the test case in a mocked method. For the other reasons, see Uncle Bob's article). So, the test may look like:

 public class BookServiceTest extends TestCase {
    private Book b123 = new Book("123", "Java programming");
    private Patron kent = new Patron("001", "Kent");

    public void testBorrow() throws Exception {
        BookService bs = new BookService();
        bs.setBooks(new Books() {
            @Override
            public Book get(String callNo) {
                return callNo.equals("123") ? b123 : null;
            }
        });
        bs.setPatrons(new Patrons() {
            @Override
            public Patron get(String patronCode) {
                return patronCode.equals("001") ? kent : null;
            }
        });
        bs.borrow("001", "123");
        List<BorrowRecord> records = b123.getBorrowRecords();
        assertEquals(records.size(), 1);
        BorrowRecord record = records.get(0);
        assertEquals(record.getPatron().getName(), "Kent");
    }
}

The problem is that if you later add a method to the DAO interfaces such as:

public interface Books {
    Book get(String callNo);
    void add(Book b);
}

Your unit test code will break because you aren't implementing the add() method. To avoid this problem,  the idea is to first create an (abstract) mock class implementing only the needed methods such as get(). Then use an automatic way to further create a subclass that provide all the dummy methods. To do the latter, one can use cglib. Here is an example:

public class BookServiceTest extends TestCase {
    private Book b123 = new Book("123", "Java programming");
    private Patron kent = new Patron("001", "Kent");

     public abstract class MockedBooks implements Books {
        @Override
        public Book get(String callNo) {
            return callNo.equals("123") ? b123 : null;
        }
    }
    public void testBorrow() throws Exception {
        BookService bs = new BookService();
        bs.setBooks(mock(MockedBooks.class));
        bs.setPatrons(new Patrons() {
            @Override
            public Patron get(String patronCode) {
                return patronCode.equals("001") ? kent : null;
            }
        });
        bs.borrow("001", "123");
        List<BorrowRecord> records = b123.getBorrowRecords();
        assertEquals(records.size(), 1);
        BorrowRecord record = records.get(0);
        assertEquals(record.getPatron().getName(), "Kent");
    }
    @SuppressWarnings("unchecked")
    private <T> T mock(Class<T> c) {
        Enhancer enhancer = new Enhancer();
        enhancer.setSuperclass(c);
        enhancer.setCallback(NoOp.INSTANCE);
        //Because MockedBooks is a non-static inner class, need to provide the outer instance
        return (T) enhancer.create(new Class[] { getClass() },
                new Object[] { this });
    }
}

 To make the code reusable in multiple test cases, just extract it into a base class:

public class ChangeResistantMockTest extends TestCase {
    @SuppressWarnings("unchecked")
    public <T> T mock(Class<T> c) {
        Enhancer enhancer = new Enhancer();
        enhancer.setSuperclass(c);
        enhancer.setCallback(NoOp.INSTANCE);
        return (T) enhancer.create(new Class[] { getClass() },
                new Object[] { this });
    }
}

public class BookServiceTest extends ChangeResistantMockTest {
    private Book b123 = new Book("123", "Java programming");
    private Patron kent = new Patron("001", "Kent");

     public abstract class MockedBooks implements Books {
        @Override
        public Book get(String callNo) {
            return callNo.equals("123") ? b123 : null;
        }
    }
&
nbsp;   public void testBorrow() throws Exception {
        BookService bs = new BookService();
        bs.setBooks(mock(MockedBooks.class));
        ...
    }
}

Acknowledgement: I got this idea from the Scala mailing list.

Applying scala to solving real world problems: Say bye bye to boring constructors and getters/setters

In Java, it is very common and boring to create constructors and getters/setters like:

public class Foo {
  private String x;
  private String y;

  public Foo(String x, String y) {
    this.x = x;
    this.y = y;
  }
  public String getX() {
    return x;
  }
  public String getY() {
    return y;
  }
  public void setX(String x) {
    this.x=x;
  }
  public void setY(String y) {
    this.y=y;
  }
}

Yes, Eclipse provides the "Generate constructor using fields" and "Generate getters and setters" commands to do that, but it is still very boring and requires work from us. In Scala, all this boring work is no longer required:

class Foo(x: String, y: String) {
}

Due to the closure support, x and y will be available to all methods inside the Foo class just like Java fields:

class Foo(x: String, y: String) {
  def someMethod {
    println(x+y)
  }
}

To create getters, use to the "val" keyword:

class Foo(val x: String, val y: String) {
  ...
}

The Scala compile will create a getter methods named "x" and "y" automatically. To create setters in addition to getters, use "var" instead of "val":

class Foo(var x: String, var y: String) {
  ...
}

Then the compiler will create setter methods named "x =" and "y =" for you (Yes, the method name contains a space and then an equal sign, which are allowed in Scala). To call the getters and setters, you may:

val f = new Foo("a", "b")
f.x            //In Scala you don't need to use () to call a method
f.x = ("c")  //This is OK
f.x = "c"    //Again, don't have to use ()

Pretty neat, isn't it?

Installing MIT Scratch on Kubuntu 9.10

Below are the steps to get Scratch working, including playing audio (recording quality is still quite poor, but you can always record outside of Scratch).

Install PulseAudio

$ aptitide install pulseaudio pulseaudio-utils

Install the latest squeak

The Squeak package included in Ubuntu 9.10 can't play audio (wav). Fortunately, the latest version (3.11.3.2135) works. So, go to http://www.squeakvm.org/unix to download the binary package. However, if you try to download the .deb package, it will say that you don't have the permission. So, download the rpm install and then convert it:

$ alien Squeak-3.11.3.2135-linux_i386.rpm
$ dpkg -i squeak_3.11.3.2135-2_i386.deb

Install Scratch 1.4

Download the Scratch source from http://info.scratch.mit.edu/Source_Code. Unzip it and you will get the VM file ScratchSourceCode1.4.image. Before you can run it, you need to know that it relies on some "plugins" that are written in C language for each platform. So, download the source code to the plugins from that same page. Then compile them. The package contains three plugins: take one of the plugin called ScratchPlugin as an example:

$ cd ScratchPluginSrc1.4
$ cd ScratchPlugin/ScratchPlugin-linux/
$  ./build.sh
$ sudo cp ScratchPlugin /usr/lib/squeak/3.11.3-2135/so.ScratchPlugin

For the UnicodePlugin, you need to do some extra steps:

$ su aptitude install libpangomm-1.4-dev
$ su aptitude install libcairo2-dev
$ add an option no-stack-protector to the gcc command in the unixBuild.sh file:
       ....
       gcc -fno-stack-protector -fPIC -Wall -c `pkg-config --cflags pangocairo` *.c
       ....
$  ./unixBuild.sh
$ sudo cp UnicodePlugin /usr/lib/squeak/3.11.3-2135/so.UnicodePlugin

Running it

To run Scratch, type:

$ squeak ScratchSourceCode1.4.image

It should work.

open source web-based centralized console for tripwire

When using tripwire to monitor changes on multiple servers, it is common to have to review and accept changes on a daily basis. Logging into multiple servers to accept the changes is troublesome. So, I've created a  web-based centralized console to review and accept changes. It is GPL licensed and can be downloaded from http://centralwire.sourceforge.net. Hope it is useful to others.

Handling BLOB/CLOB in postgreSQL with JDBC/Hibernate

What is a BLOB/CLOB? It is a large object (binary for BLOB and char for CLOB). It is commonly used when you're storing large files into the database as column values.

It was really a challenge to work with BLOB/CLOB and in the process I almost pulled my hair out. Below are the hard lessons learned:

• Avoid using LOB if possible! If say, it is just a few KB in size, just treat it as a string or byte array.
  
• You must NOT try to read the stream outside the transaction. Note that using the open session in view is NOT enough. Once the transaction is ended, even if the Hibernate session is still open, you still can't read it (you'll get the "invalid large object descriptor" exception).
• You must NOT try to read a LOB twice in the transaction. The "pointer" can't seem to be reset so you read past some portion, you can't read it again.
• I still haven't figured this one out: If you repeatedly read a row containing a LOB in different transactions, it may cause the same "invalid large object descriptor" when you try to read it in a new Hibernate session.

Better approach to unit testing pages in web applications

It's common to see two approaches to unit testing pages in web applications: in-container (real container and browser) and out of container (mocked container and no browser). The former is good in that the tests can check the HTML DOM element and thus works with AJAX, but it is difficult to mock the services as the web application is running "on the other side". The latter is the opposite: Easy to mock the services but usually the tests can only check the internal state of the program, not the end HTML code.

In fact, a better approach is to combine them together: Run the application in the container and run the container in-process so that the test code can mock the services. This way you can control the right thing (user input on the web page and the data your pages get, from the services), and then observe the right thing (HTML DOM elements, possible manipulated by Javascript/AJAX). In order to run the container in-process, we can, eg, run an embedded Jetty. As it is run in the same process, the test code can access the servlet context and thus can get the opportunity to replace the services with mocks.

I've created a proof of concept library for unit testing Wicket pages. The mocking is done by using a chain of component injectors. The first one keeps a map of mock objects. The second one is the normal one to inject Spring beans. Usually the map in the first injector is empty so it has no practical effect in production. But your tests can put mock objects into there to perform mocking.

The library is released as open source software (LGPL). You're welcome to check it out at http://wicketpagetest.sourceforge.net. There is a step-by-step tutorial.

Finally, potentially this approach can be applied to other web frameworks like JSF, Tapestry and etc.