Test first, test last – and in between

These are, of course, the very first basic questions to be asked when planning a new product.

“However, one doesn’t always realize that product testing is critically related to those questions”, says Mr. Esa Kivioja, Etteplan’s Software Testing Department Manager.

Testing is the way to verify that the product meets the specified requirements and the product features are working as described in the specifications.

The features of the product more or less define the type of tests to be run on it. Three of the key factors to be determined in the test plan is what, why and how to test.

“Should the testing be split into different phases of the product development? On which features should it be allocated? These are the kind of questions you have to ask yourself”, Kivioja says.
 

In addition to functional requirements set by the product owner itself, there are additional requirements from law and regulation, concerning safety, security, or performance, for example.

“These external requirements broaden the testing scope and emphasize the importance of the planning phase even more. There is a whole lot to be taken into consideration”, Kivioja says.

Testing should be planned and included already in the earliest phases of the product development process. In fact, testability may set requirements for the product design itself. That’s design for testability.

“It may be challenging to test certain product parameters if testability is not being taken into account already in the product design”, says Mr. Niko Auvinen, Senior Test Manager at Etteplan.

Unfortunately, according to Mr. Kivioja, testing on the whole gets often neglected in the product development. For example, testing might get done only if someone remembers to “run some tests” and there is still budget left. This may result in bad product quality which may cause reputational damage and recall risk.
 

“It should be kept in mind that quality assurance is just as important as product development function”, Kivioja says. This holds true especially in the world of embedded products.

“If there’s a bug in a finished embedded product, you may not fix that easily after the product has hit the market”, Auvinen says.

There are other reasons to emphasize the importance of testing in the embedded product environment.  For example, embedded solutions typically require one or more radio modules for wireless communication. That’s where requirements of the radio equipment directive step in.

“Whenever there is a radio in an embedded product, you have to make sure it meets the safety requirements in addition to requirements to radios set by the regulation”, Auvinen says.  

If any major failures is being found in the later phases when the product was thought to be ready and finished, the required changes in electronics design will cost a lot. So, by testing as early, continuously and in small steps as possible, you can avoid unpleasant surprises. It’s always good to fail fast.

“Testing generates valuable information for project management. By running appropriate tests at the end of each iteration we know how far we are from the goals set to the product”, Auvinen says.

The coverage and quality of the testing can be followed and measured by setting up a testing dashboard.

“There is absolutely no reason to consider testing as an abstract concept. The automated tools can, for example, monitor and report the testing coverage of the currently running test set”, Kivioja says. The dashboard results may help in deciding whether the product is ready for the next phase in product development or even the market.
 

Should a product owner have challenges in planning the testing and placing the tests in the product development process, it doesn’t necessarily matter. There is help available.

Etteplan has the expertise and tools to help product owners increase testing quality in all phases and fields of the product development process, from planning to production and finally maintenance.

“Electronic device product development takes time and hardware is not at hand right away. It poses its own challenges to testing software in parallel with hardware”, Kivioja says.

However, you don’t have to wait until a physical product exists to start implementing the tests. Involving testing already in simulated environments reduces lead time significantly.

“You can run plenty of virtual tests on a digital twin of a product before the physical version even exists”, Kivioja says. This speeds up the development process a lot.