Carsten Emde, Open Source Automation Development Lab (OSADL) eG

The traditional gold standard to determine and to certify a system's real-time capabilities is path analysis. The recipe is easy: Study the machine instructions of the kernel in question, find the longest path with interrupts and preemption disabled, determine the number of machine cycles of this path and multiply the result with the clock cycle duration of the CPU. In reality, however, things are not that easy anymore: State-of-the-art CPUs with several levels of possibly shared caches, peripheral devices that are connected via an arbitrated bus and IRQ layouts that force controllers to share common interrupt lines may all cause latencies that are no longer evident from studying the source code. In addition, path analysis cannot be done - or at least is very difficult to do - if the source code of a driver is not available at the time of the path analysis. Last not least to err is human, so it is certainly a good idea to verify the theoretically derived result of a path analysis by an empirical approach to measure the system's worst-case latency. Thus, a suitable latency measurement concept is needed to verify the result of a path analysis and to determine the worst-case latency in cases when path analysis cannot be done. OSADL has developed a three-stage latency measurement concept that consists of

• Overall external latency measurement using the OSADL latency measurement box
• Internal registration of latencies using the latency histograms of the Linux kernel
• Internal simulation of latencies that may occur in a real-time task

In parallel with these measurement procedures, calibration and stress tools have been developed and defined to ensure that a realistic system load is created throughout the measurement. This concept is routinely used and all individual tests must have been passed successfully, before a PREEMPT_RT Linux kernel is declared "Latest Stable". The various test procedures, calibration and verification tools, stress scenarios and individual results are presented and discussed in detail.