Last time I touched on why relying on manual testing of your medical device is a brittle, unsound strategy to ensure safety.

Automated tests are run completely, consistently, and (nearly) instantly, giving rapid feedback on the quality of your software.

However, it’s not all roses and puppies in automated testing land.

Here are some of the downsides to automated testing.

Writing good tests is hard

When I first started writing tests, I made all the classic beginner mistakes.

I tested implementation details rather than testing the interface.

I slaved on a single module to achieve 100% code coverage, so much so that I ran out of time and didn’t write any tests at all for my other modules. Most of the testing code I wrote did not add much value.

I didn’t refactor as I went, so when I made mistakes with my code architecture that made it hard to test, I wasted time working around those issues instead of solving them.

Speaking of which…

Writing code that is testable is even harder

This is probably the #1 reason why software developers give up on writing unit tests. It’s especially a problem when you first learn about unit testing, get really excited, and dive into testing with gusto.

It’s not enough to write code that is modular. To write testable code, you must ruthlessly reduce inter-dependencies between your code modules. Embedded software is especially tricky because you are interacting directly with the hardware. A solid Hardware Abstraction Layer (HAL) is an absolute must.

It’s so easy to look at a codebase and think to yourself that it is well-structured. You learn how wrong you are when you try to write tests for it. Sure, the variables and functions are nicely named, but in order to test the simplest component, you’d have to mock ten others!

Writing unit tests is the fastest way to reveal flaws in your code architecture, and force you to fix them.

It’s hard to test legacy code

Yep. True statement.

Sometimes it’s worth it, especially when making modifications to that legacy code is critical to your business, and you can’t easily make modifications because it’s so brittle. In that case, suck it up, refactor the legacy code, and test the hell out of it.

In other cases, it’s not worth it to spend time testing the legacy code comprehensively. If it doesn’t change often, and it’s already been de facto tested by being in production for some time, then it’s best to leave it alone and focus your limited energy elsewhere.

It’s hard to do the right thing

As with anything else, if you want human beings to follow a process, you’d better make it really easy to follow the process.

If you call a team meeting and tell your software team, “start testing”, you probably won’t get the result you want out the other side.

Invest in continuous integration infrastructure to make running tests automatic.

Invest in training your developers on how to write tests, how to write testable code, and how to work with each other to create a culture of testing.

Invest in developing internal tools (scripts, makefiles, IDE plugins, etc) to help your developers do the right thing.

Your best developers should be spending time creating the tooling and infrastructure to make your entire team more productive. This isn’t a job for an intern. This is a job for your most seasoned engineers.

Want to learn how to write units tests for embedded systems? Even better, want to learn how to write code that is testable?

Get this book.

Or, reach out to me.

Happy developing!