The software of embedded systems in the aerospace industry as well as the automotive area must fulfill high safety requirements. Hence, already in the development process, suitable steps have to be to carry out to guarantee that these requirements are kept. One of these steps is the system test which is performed after the integration of the software in the overall system. However, the software to be tested can not be segregated from the remaining system. The test must enclose the interaction with the hardware and the physical environment.
The usual approach for such tests is the HiL (Hardware in the Loop) test. The completely integrated system, consisting of hardware and software, is coupled with a simulation of the environment and is executed on a real-time basis. Unfortunately, most of the hardware components needed for the development process are available quite late. This is the primary disadvantage of the HiL approach. In particular, specialized hardware for the simulation of the environment as well as the electrical interfaces are resource bottlenecks.
An alternative approach is the SiL (Software in the Loop) test which couples partially integrated software with an environment simulation. Instead of the usage of electrical interfaces, software interfaces provided by the operating system are used here which allows a direct information-technical communication with the simulation. This results in a higher flexibility for the execution of tests and does not require special hardware. Therefore, SiL tests can be introduced early during the software development and offers the possibility to execute tests before the hardware is available. The disadvantage of the SiL test is the changed timing behavior of the software which, however, can be compensated by a synchronization protocol with a simulated real-time clock.
To enhance the SiL test approach, the DLR institute “Simulation and Software Technology” has developed a test framework called SiLEST funded by BMBF (German Ministry of Education and Research) and performed within the research program “Software Engineering 2006”. Different X-in-the-Loop test approaches were compared and a test automation environment for this field of applications was developed.
The SiLEST test automation environment offers support for the independent execution of test cases collected in a test suite without manual interventions. The support is independent of the selected X-in-the-Loop test approach. In particular, the test automation is bound to arbitrary test management tools and to a configuration management system which is implemented by a plug-in mechanism. For the test, test cases and a test object can be autonomously loaded from the configuration management system. Then, the produced test results and the test report can be stored again in the configuration management system. However, the test results will also be passed automatically to the test management system for further human analyses.
An XML-based data format has been developed to describe test cases. This comes along with suitable tools which are in general independent of the actually used X-in-the-Loop test approach. Moreover, a mapping is provided between the interfaces in the test case, in the test object, and in the environment simulation. This guarantees a high reusability of test cases during the complete life cycle of software as well as the run time of projects.
Special attention was laid on the possibility to select arbitrary time-continuous signals for test cases. These can be used as test stimuli or for the verification of the correct behavior. Additionally, the test case format offers ways to modify or replace signals within the simulation, which is needed during the verification of the system robustness against sensor and actuator faults.
For the simulation of such faults, a simulation library has been defined which provides appropriate signal management functionalities. As an example, the SiLEST test process was evaluated using the attitude control system of the compact satellite BIRD and an engine control unit employed in the automotive industry. The results showed that the automatic test execution mainly yielded, above all, a reduction of expenses for regression tests and relieved the test engineer considerably. Eventually, SiLEST cuts down the complexity of the X-in-the-Loop test preparation merely on the development of the required environment simulation.
More information to the project is to be found on the official SiLEST web page or the final report.
Project partners
Project duration
01.01.2004 - 30.04.2007