In this article, we take an objective look at test driven development, it's benefits, drawbacks, best use cases, and associated costs. Along the way, we'll look at concerns for over testing, maintenance, and why TDD isn't practiced more in real life.
Test Driven Development Overview
Have you ever argued about the efficiency of test-driven development in your day job? I certainly have, and on both sides of the argument. I have worked in the software development sector for 9 years as a developer and an architect on seven different projects ranging from mobile apps to a custom made telecom self services. I’m fed up with the pointless arguments - the ones based on solo experiences and anecdotal evidence.
Arguments should be based on a common understanding of what TDD is and your current project situation at hand: people involved, technology, goal, deadline, etc.
With that in mind, we’ll discuss the following topics:
- Benefits of Test Driven Development
- Graphical User Interface and Test-Driven Development
- Against Test-Driven Development
- Why isn’t TDD Practiced More in Real Life?
Benefits of Test Driven Development
Using test driven development:
- Reduces bugs when applied to legacy projects
- Decreases bug to code ratio in new projects
- Increases overall software quality
- Increases code quality
There are more reasons to use TDD, but, as we outline below, these benefits are supported by long-term industry studies.
Why Use Test Driven Development?
Test-driven development is increasingly widespread and there is good empirical evidence that it’s a beneficial practice. TDD reduces the number of bugs in production and improves code quality. In other words it makes code easier to maintain and understand. Also, it provides automated tests for regression testing. The evidence about TDD from the industry:
- Applying TDD by a single developer to legacy codebase over years reduced bugs by 50% -- OMG, Test Driven Development Actually Works! (September 2012)
- New projects built with TDD approach had a smaller bugs to lines of code ratio than non-TDD ones -- Realizing quality improvement through test driven development: results and experiences of four industrial teams (February 2008)
- 5 year study with developers inexperienced in TDD showed better software quality using TDD than the industry average -- A longitudinal Study of the Use of a Test-Driven Development Practice in Industry (2007)
- Study on open source projects found that TDD increased code quality 20% -- Quantitatively Evaluating Test-Driven Development by Applying Object-Oriented Quality Metrics to Open Source Projects (December 2009)
TDD Isn't Right For All Situations
Just because there are benefits for TDD, it doesn’t mean that it should be used by everyone everywhere in every situation. This will be discussed further in chapter "Against Test-Driven Development" of this post.
Moreover, it is not advised to develop a graphical user interface (GUI) in a test-driven way. One could look at this topic in more depth on StackOverflow, like in this question about Does TDD apply well when developing an UI? or the academic community. Both agree that developing GUIs in a test-driven way is not straightforward.
TDD Can Be Costly for GUI Testing
According to the aforementioned sources, automating GUI tests during the development phase costs more than manual testing. It’s even costlier if one tries to automate in a TDD way because GUI test automation tools are focused on existing systems. It would be pragmatic to automate GUI regression tests after the development phase is finished. In general, GUI test automation is made costly by these aspects:
- Complexity - the number of possible user interactions from different application states, grows exponentially as the GUI functionality increases.
- Verification - as the complexity increases it becomes harder to create automated test scripts that cover a sufficient number of cases.
- Change - GUIs change very often during development, so test script maintenance becomes a burden.
This brings us to the next chapter, how to incorporate TDD and GUI development.
👉🏽 Looking for other resources related to testing? Check out Testing Microservices in Java and TestContainers: Making Java Integration Tests Easy
Developing GUI With Test Driven Development
If developing a GUI in a test-driven way is not beneficial, then how should one use TDD with an application that have a GUI? One of the leading proponents of TDD, Kent Beck, suggests to create a thin view layer together with underlying model and develop application in a TDD way from that model instead. Some software patterns have spawned from this idea:
- Presentation Model
- Model-View-ViewModel. Here are a couple of useful links that describe the summary, background, and pros and cons of this approach.
- Model-View-Binder, basically MVVM outside of the .NET world
The goal of these patterns is to make the view layer as dumb as possible, so testing it isn’t required. All of these patterns work similarly: the view will have an underlying model that has fields for each view element or state. The view will also delegate all the actions to the model. If an update to the view is needed, the model will request it. This allows developing the application from that model using TDD.
This approach covers most of the code with automated tests, although, doubts remain if the actual view is integrated with the underlying model correctly. To overcome this uncertainty, a few tests are recommended that invoke the GUI, for example through the browser for web applications, and verify that the connections with the underlying model are correct. The fact that there are only a few tests that invoke the GUI means that maintaining those are easy. Moreover, because there are only a few GUI tests, they don’t slow down the application test cycle. These tests don’t have to be written in a TDD fashion of course.
When Not to Use Test Driven Development
Test-driven development can backfire when the environment is not suitable or it is used incorrectly. One should consider these topics when planning to use TDD:
- Cost of implementing functionality
- Test code requires maintenance as well as production code
- Too much TDD makes code more complicated than necessary
Let’s look at them in more details.
Cost of Implementing Functionality
The main drawback is that it takes significantly more time to implement functionality in a test-driven way than in conventional manual testing way. The cost depends on a couple of things. Consider for example these issues:
- Experience with TDD among developers - less experience means more cost.
- Existing code support for automated testing - trying to automate testing on a framework not meant for that, will probably cost more than the benefit it provides.
- Organizational support for TDD - It might be easy to convince your department to start using TDD, but if your project depends from another department you might not reap all the benefits of TDD. For example, if you need a running copy of their component with all the test data for running integration tests, but they are not willing to provide it to you, then you simply cannot run integration tests. Organizational support also means resources for setting up an initial TDD scaffold.
When creating a proof of concept it is usually faster and cheaper to use manual testing. Furthermore, when time to market is important, then it’s worth considering creating a quick version without automated tests. It is a slippery road, of course, because future changes will be more error prone, thus harder and slower to implement because of a lack of automated regression tests.
Test Code Requires Maintenance as Well as Production Code
The second problem of TDD is that test code grows linearly with production code. All lines of code require maintenance, which means cost. The cost is easiest to acknowledge when a change to existing functionality is made. The tests have to be modified as well as production code.
This becomes a problem when multiple tests execute the same line of production code. This usually happens when there are too many high-level tests. The highest level test is an end-to-end test, which starts by invoking the GUI and goes all the way down to the data storage level. But it’s not only the highest-level test that can cause problems.
Tests that execute a whole component, for example a web service, are quite high-level too, compared to a unit test. The ice cream cone anti-pattern is a nice illustration of TDD gone wrong. An ice cream cone emerges when you invert the ideal test automation pyramid. You can make ice cream by creating more integration tests than unit tests and more GUI tests than integration tests and even more manual regression tests than your automated GUI tests.
Too Much Test Driven Development Can Be a bad Thing
A third problem emerges when one applies too much TDD - so much so that the tests will make the code more complex than necessary. There are good examples, albeit from the Ruby world of how this can happen. Here are main points from that article:
- It’s wrong to decouple the application from the Rails framework, so that tests could run faster and code could be easier to mock. This looks good from the testing perspective, but from the production code perspective the code gets more complex. It contradicts one of the TDD goals - to design better production code.
- Unit testing MVC controllers doesn’t make sense. These should be integration tested instead.
- Don’t try to eliminate databases from model tests. Booting the database takes only 1.2 seconds and you can use transactional text fixtures to generate different scenarios.
- “The answer to how can I make it better, is how can I make it clearer, not how can I test it faster or more isolated” -- D.H.Hansson
Why Isn't TDD Practiced More in Real Life?
In short, it’s impossible to show connection with TDD and improved productivity. That’s because measuring software creation productivity is impossible. There is a conflict between the long term and short term goals for software creation teams. In the short term the fastest way is to develop functionality, test it manually, fix the bugs, test again and so on, until the functionality is ready. There are no arguments that TDD would be faster in the short term. However, in the long term some problems arise with the manual testing method:
- Time for manual regression testing increases as the project grows, which makes release cycles longer.
- One could freeze regression testing time in order to freeze release cycle time, but then more and more bugs go to production.
- Code quality suffers which makes adding new functionality slower as the project grows.
Even though the benefits are clear in the long term, these benefits don’t run companies - deadlines do. When an upcoming hard deadline approaches, the development team will be focused on the short term. At that time the inexperienced team members hope to fix things in the future, experienced members have already abandoned that hope.
Conclusion
In this post we’ve discussed how TDD is becoming more and more widespread for the following reasons:
- reducing the number of bugs in production
- improving code quality
- providing you with the automated tests for regression testing
However, before choosing to use TDD in an existing project, you have to consider the unique context of your project in detail. Ask yourself these questions and then decide:
- How much will TDD increase the cost of implementing your functionality?
- What kind and how many automated tests do you plan to create?
- Will TDD be used correctly? Will TDD simplify your production code or make it more complex?
After all, here’s a short rule of thumb: A GUI shouldn’t be built with TDD. A thin view layer between a GUI and application logic should be created instead. If automated regression tests are needed, GUI test automation tools can be used after development.
Additional Resources
Get more tips and tricks for Java development by checking out our resources.
Interested in more tips to boost your application performance? Check out our webinar, "Eight Ways to Improve Java Microservices Performance."