I have always been a fan of testing and blogged about it in the past. Software testing is a complex discipline. Most developers admit it’s important, but I think we are often mistaken about the true purpose of testing. It already starts with the word ‘test’, which has a wider range of meanings in software than the dictionary definition [to] take measures to check the quality, performance, or reliability of (something), especially before putting it into widespread use or practice. What is this something? The software of course! Ah, so we apply the test to something that’s already there. We test to check the work we have done. That not quite TDD.

Test Driven Development builds test and production code in micro-increments, thinking first about how you are going to test the code under construction. The biggest benefit of unit tests and TDD is that they instill a commitment to work thoroughly and not rush to get something working. It makes sure that code is always testable. Untestable code is by its nature unreliable and probably a maintenance nightmare. It works great at the level of small units, but TDD is no guarantee that the end product works, much less that it does what the stakeholders expect. I’m still not telling you anything new.

The trouble with writing automated test code at any level (from unit- to integration to end-to-end testing) is that it can always be added after the fact. As long as we think of testing in the dictionary meaning of the word there is always the lure of cutting corners and adding it after the fact. By doing so we not only run the risk of degrading the quality of our work (doing things right), but also of not building the right thing. Tests should also act as an opportunity to verify that we are building the right thing.

Acceptance Test Driven Development (ATDD) solves many of the problems I just sketched. Acceptance tests emerge from acceptance criteria, which result from discussions between all stakeholders. They focus on WHAT to build before it has been built. That makes it different from all other modes of testing. It must start before a single line of code is written and must involve other people than just the developers.

Acceptance tests are commonly expressed as a collection of Given-When-Then scenarios: Given a certain situation A, when user B does C, then we expect B to happen. You can start these out on a whiteboard. The important thing to remember is that they are owned (in a Scrum setting) by the Product Owner and that only at a later stage they are fleshed out with backing code that does the actual verification, typically in a framework like Cucumber or Robot Framework.

The ATDD test scenarios that the Product Owner helped to create are turned into executable code by the developers. The original scenarios from the whiteboard specify what to build and the backing code verifies that this is so. Once in place these ATDD tests are a living body of documentation and – more importantly – a piece of high-level code that you cannot alter without breaking things. In a culture where documentation is considered optional (even unimportant) and all code is prime, what better way to fix it than to turn the documentation into code?

Writing great ATDD scenarios is hard and paradoxically it does NOT require coding skills. It takes writing skills. Since scenarios are intended for all stakeholders they must be phrased in the ubiquitous language shared by all stakeholders and not expose the technical internals of the system. It is hard to strike the right balance between too much and too little abstraction and this can take several iterations. I recommend the excellent book Writing Great Specifications by Kamil Nicieja for a really thorough discussion.

To close, let me summarize why ATDD is a radically different way of testing and well worth the investment:

• Since the scenarios actually tell the developers what to build, it must start before writing any code
• Test scenarios are owned by the Product Owner and therefore more likely to express what the stakeholders need, in non-technical terms.
• The plain-text scenarios are turned into executable code, so they are tightly coupled to the production code, which is a good thing for a change.
• Think of ATDD as All Tests Drive Development.