American software engineer Kent Beck is credited with having developed or "rediscovered" the technique of Test-Driven Development, suggesting that TDD encourages simple designs and inspires confidence. In 2011, James W Grenning published a book entitled “Test-Driven Development for Embedded C”, showing that Test-Driven Development can also be performed on embedded code. He is one of the authors of CppUTest, an open source test harness for C and C++.

The TDD paradigm requires a set of tests to be created before each function is developed. Next, the function is written as a stub that compiles, but serves no useful purpose. An initial set of tests is then executed, most or all of which will fail. An iterative process then follows such that code is developed to ensure that all tests pass, and new tests are added until the full functionality of the function is in place. The code may then be refactored to make it more efficient, and the tests re-run.

This iterative cycle is known as “red, green, refactor”. It involves writing tests first, making them pass, and then refactoring the code to improve the finished article.

Because the tests are constantly regressed, it is essential to have a mechanism in place for their automation.

Mission-critical Test-Driven Development

Mission-critical TDD adheres to similar principles, with some variation to accommodate the demands of functional safety standards. For example, it is necessary to execute tests in an environment to closely resemble that of the complete product (usually on the target hardware, or perhaps a simulator). When refactoring the code it is important to ensure that it is compliant to a coding standard such as MISRA C, and that the code is not more complex than it needs to be by monitoring metrics such as the cyclomatic complexity. And in executing the final tests of that refactored code, it is essential to check structural coverage to ensure that all the code has been exercised, it is all traceable to requirements, and that there is no dead code.

Once again, the automation of test regression is essential. The use of continuous integration management tools such as Jenkins is popular in this regard.

Test automation

Test automation is key to the successful deployment of TDD. Suitable tools fall into two categories.

For lower integrity software there are many free or low cost frameworks that can be deployed, including CppUTest and Unity. These tools provide a testing framework that allows tests to be written in C++, which makes it easy for developers to learn how to use them. “Mocks” (or “stubs”) need to be created manually for any missing functions.

Mission-critical embedded software for safety critical sectors such as industrial, automotive, and aerospace are usually expected to comply with functional safety (IEC 61508, ISO 26262 and DO178C respectively) and/or cybersecurity standards (IEC 6244341, SAE J3061, and DO-326A).

One popular approach is to use a low-cost framework in tandem with a “home grown” test framework to address the features needed when testing high integrity applications. Such a set-up reduces capital investment which might be important especially when test is to begin so early in a project. However, the standards demand rigorous evidence that any tools used are qualified as fit for purpose, and that applies to in-house tools like any other. Such a process can be very labour intensive.

The TBrun component of the LDRA tool suite has been chosen for the worked example to exemplify a second group of tools that are designed for use with high-integrity software and are TÜV certified to ensure that tool qualification is as seamless as possible. TBrun can automate the creation of stubs, the execution on targets (even when resources are constrained), and the measurement of structural coverage.

Mission-critical TDD: A worked example

Consider the requirements shown in Figure 1 for an example function that forms part of a mission-critical embedded software project.

Figure 1: Requirements for an example function

It is typical of embedded systems that some functions are called very frequently to sample data and that they must complete their execution within an allocated time budget. In this example the sine must be computed for an angle in less than 75 CPU cycles. Other requirements specify what should happen if the requested angle is out of range, the required accuracy of the result, and the scaling to be deployed to avoid the use of floating point.

It is assumed that the application code will be executed on an STMicroelectronics STM32 target, and that the code is to comply with a functional safety standard that requires the tests to be executed on that same target.