You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-15 - 09:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6.osadl.org (updated Wed Jan 15, 2025 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22851999875,15cyclictest0-21swapper/020:15:120
2240329614,35sleep10-21swapper/119:07:041
296232940,5sleep129627-21cut19:25:261
2144829237,37sleep00-21swapper/019:05:160
22851998768,9cyclictest14-21ksoftirqd/023:50:080
22851998256,18cyclictest14-21ksoftirqd/023:30:090
259372790,4chrt0-21swapper/122:30:251
22851997956,15cyclictest14-21ksoftirqd/022:20:130
22851997849,18cyclictest14-21ksoftirqd/023:35:110
22851997747,16cyclictest14-21ksoftirqd/019:40:140
114342770,9chrt11433-21/usr/sbin/munin20:05:340
22851997620,12cyclictest22906-21apt-get20:40:150
22851997214,38cyclictest243-21systemd-journal23:15:000
22851997212,38cyclictest24948-21date19:15:010
2286599706,24cyclictest26-21ksoftirqd/121:05:011
22851997041,21cyclictest14-21ksoftirqd/019:20:130
22851997041,20cyclictest14-21ksoftirqd/021:20:130
22851997040,17cyclictest14-21ksoftirqd/019:15:130
22851997038,18cyclictest0-21swapper/023:00:110
77902690,10chrt0-21swapper/119:55:301
2285199699,51cyclictest20045-21netstat20:30:350
2285199699,18cyclictest0-21swapper/020:35:210
22851996921,37cyclictest15874-21apt-get20:20:150
22851996916,21cyclictest502-21avahi-daemon00:00:020
2285199685,36cyclictest0-21swapper/020:45:170
22851996838,17cyclictest0-21swapper/000:35:150
22851996736,20cyclictest7081-21apt-get19:55:150
22851996720,37cyclictest12412-21aten2.4-expect20:10:200
22851996716,25cyclictest29818-21runrttasks20:58:140
22851996716,25cyclictest24382-21runrttasks00:15:130
22851996716,21cyclictest26783-21turbostat22:35:010
22851996715,27cyclictest1696-21plymouth19:40:000
22865996645,9cyclictest26-21ksoftirqd/121:36:131
22851996646,11cyclictest14-21ksoftirqd/023:15:110
22851996638,16cyclictest0-21swapper/020:25:120
22851996635,20cyclictest19278-21apt-get22:10:100
22851996635,19cyclictest5343-21apt-get19:50:160
22851996631,21cyclictest15109-21apt-get21:55:140
22851996616,25cyclictest21802-21idleruntime-cro00:05:000
22851996616,20cyclictest30534-21runrttasks21:00:220
22851996615,27cyclictest10050-21runrttasks21:35:350
22851996615,22cyclictest243-21systemd-journal22:04:590
22851996611,30cyclictest550-21snmpd22:44:390
2285199659,32cyclictest16529-21idleruntime-cro23:50:000
22851996539,13cyclictest26948-21apt-get22:35:140
22851996523,37cyclictest31241-21aten2.4-expect22:50:120
22851996514,27cyclictest31027-21latency_hist22:50:010
22851996513,22cyclictest0-21swapper/023:26:120
22851996510,22cyclictest0-21swapper/019:30:320
314882640,5chrt251rcuc/119:30:381
22865996429,23cyclictest26-21ksoftirqd/121:50:261
22865996429,23cyclictest26-21ksoftirqd/121:50:261
2285199648,29cyclictest2016-21awk23:00:010
2285199648,29cyclictest2016-21awk23:00:000
2285199646,35cyclictest23102-21irqstats00:10:200
22851996441,13cyclictest9977-21seq23:25:000
22851996413,26cyclictest29655-21cut19:25:250
22851996413,26cyclictest27403-21sshd00:21:510
22851996413,23cyclictest4602-21taskset21:16:520
22851996413,22cyclictest29205-21awk00:30:240
22851996413,20cyclictest243-21systemd-journal22:10:010
22851996412,26cyclictest1421-21cron21:10:010
22851996412,21cyclictest28262-21basename20:55:000
22851996410,21cyclictest0-21swapper/020:00:330
22865996333,12cyclictest26-21ksoftirqd/100:29:141
22851996330,20cyclictest21075-21apt-get22:15:140
22851996316,19cyclictest1599-21apt-get21:10:150
22851996312,21cyclictest8481-21unixbench-2d21:30:340
22851996311,27cyclictest4756-21awk19:45:380
22851996311,27cyclictest4756-21awk19:45:380
22851996311,27cyclictest13399-21aten2.4-expect21:50:110
22851996311,27cyclictest13399-21aten2.4-expect21:50:110
2286599626,25cyclictest26-21ksoftirqd/100:30:211
2286599623,25cyclictest26-21ksoftirqd/120:50:221
22865996227,23cyclictest26-21ksoftirqd/122:55:201
22865996227,23cyclictest26-21ksoftirqd/122:55:201
22865996210,25cyclictest598-21runrttasks23:20:241
22851996214,24cyclictest14982-21runrttasks23:40:260
2286599616,14cyclictest0-21swapper/119:20:121
2286599615,24cyclictest26-21ksoftirqd/120:45:251
22865996137,12cyclictest26-21ksoftirqd/122:53:011
2286599613,24cyclictest26-21ksoftirqd/119:50:251
22865996127,22cyclictest26-21ksoftirqd/122:35:221
22865996126,23cyclictest26-21ksoftirqd/122:40:311
22865996126,23cyclictest26-21ksoftirqd/122:20:111
22865996126,23cyclictest26-21ksoftirqd/119:15:081
2286599612,43cyclictest26-21ksoftirqd/120:40:341
22865996118,14cyclictest26-21ksoftirqd/122:15:161
22851996120,35cyclictest243-21systemd-journal22:26:550
22851996113,23cyclictest7097-21runrttasks21:28:040
22851996112,25cyclictest4013-21runrttasks23:06:070
22851996111,20cyclictest13207-21idleruntime-cro21:50:010
22851996111,20cyclictest0-21swapper/021:40:180
2286599606,25cyclictest26-21ksoftirqd/120:10:571
22865996026,22cyclictest26-21ksoftirqd/123:50:171
22865996025,23cyclictest26-21ksoftirqd/120:35:301
22851996032,19cyclictest499-21dbus-daemon00:10:000
2286599597,23cyclictest26-21ksoftirqd/120:30:321
2286599593,26cyclictest26-21ksoftirqd/120:10:011
22865995925,22cyclictest26-21ksoftirqd/100:15:141
22851995910,19cyclictest0-21swapper/000:26:510
2286599585,24cyclictest26-21ksoftirqd/123:40:331
2286599584,23cyclictest26-21ksoftirqd/121:40:291
2286599583,25cyclictest26-21ksoftirqd/120:55:221
22865995827,14cyclictest26-21ksoftirqd/121:55:191
22865995827,14cyclictest26-21ksoftirqd/120:00:131
22865995827,12cyclictest26-21ksoftirqd/100:38:151
22865995826,20cyclictest26-21ksoftirqd/123:30:261
22865995824,22cyclictest26-21ksoftirqd/100:20:121
22865995824,20cyclictest26-21ksoftirqd/123:06:281
22865995820,12cyclictest26-21ksoftirqd/121:30:401
22865995813,19cyclictest0-21swapper/121:28:471
2286599574,23cyclictest26-21ksoftirqd/123:00:211
22865995727,13cyclictest26-21ksoftirqd/119:40:141
22865995724,21cyclictest26-21ksoftirqd/123:35:131
22865995724,21cyclictest26-21ksoftirqd/120:20:321
22865995723,22cyclictest26-21ksoftirqd/100:10:081
22865995719,25cyclictest26-21ksoftirqd/121:24:061
22865995718,25cyclictest17521-21strings22:00:281
22865995714,8cyclictest151rcu_preempt23:10:381
22865995627,12cyclictest26-21ksoftirqd/120:15:131
22865995625,14cyclictest26-21ksoftirqd/122:10:251
22865995620,12cyclictest26-21ksoftirqd/122:06:111
22865995616,18cyclictest2993-21runrttasks21:13:181
22865995614,9cyclictest151rcu_preempt21:05:081
22865995524,14cyclictest26-21ksoftirqd/121:15:121
22865995524,14cyclictest26-21ksoftirqd/100:00:131
22865995516,21cyclictest598-21runrttasks23:45:411
22865995514,18cyclictest18342-21ntpq20:25:291
22865995514,17cyclictest598-21runrttasks23:19:091
22865995513,23cyclictest26-21ksoftirqd/119:49:561
22865995513,23cyclictest26-21ksoftirqd/119:49:551
2286599542,26cyclictest26-21ksoftirqd/119:10:091
22865995419,15cyclictest26-21ksoftirqd/100:10:001
2286599536,24cyclictest26-21ksoftirqd/122:30:011
22865995323,14cyclictest26-21ksoftirqd/119:35:151
22865995323,13cyclictest26-21ksoftirqd/123:55:121
22865995322,12cyclictest26-21ksoftirqd/123:26:331
22865995217,23cyclictest26-21ksoftirqd/122:46:321
22865995116,22cyclictest598-21runrttasks21:45:261
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:513
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
none09:44:512
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional