You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Thu Apr 03, 2025 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
215250999117108,7cyclictest2215074-21gpgv22:40:011
215250999117105,9cyclictest2177844-21latency_hist20:30:011
21525069910793,11cyclictest2171737-21apt-get20:10:150
21525069910697,6cyclictest2242534-21kworker/u8:1+flush-179:000:30:100
21525069910695,7cyclictest2237027-21kworker/u8:3+events_unbound00:15:160
21525069910692,11cyclictest2192928-21apt-get21:15:110
21525099910497,5cyclictest2213321-21kworker/u8:1+flush-179:022:40:041
2152506999890,5cyclictest2206479-21kworker/u8:3+flush-179:022:10:160
2152506999584,8cyclictest2225288-21latency_hist23:20:020
2152506999583,9cyclictest2187905-21latency_hist21:00:010
2152509999489,4cyclictest2152637-21kworker/u8:1+flush-179:020:35:151
2152506999479,12cyclictest2177851-21apt-get20:30:000
2152506999479,12cyclictest2174932-21apt-get20:20:120
2152509999387,4cyclictest2181334-21kworker/u8:0+flush-179:021:00:141
2152506999284,5cyclictest1490643-21kworker/u8:4+events_unbound19:15:270
2152506999281,8cyclictest2211492-21latency_hist22:30:020
2152509999184,4cyclictest2187888-21kworker/u8:1+flush-179:021:50:001
2152509999178,10cyclictest2232411-21apt-get23:50:071
2152506999079,8cyclictest2191088-21latency_hist21:10:000
2152509998981,6cyclictest2242534-21kworker/u8:1+flush-179:000:40:001
2152509998883,4cyclictest2231886-21kworker/u8:0+flush-179:000:00:311
2152509998883,3cyclictest2202461-21kworker/u8:0+flush-179:022:00:301
2152509998880,5cyclictest2152637-21kworker/u8:1+events_unbound19:30:261
2152509998773,11cyclictest2228963-21apt-get23:35:111
2152506998781,3cyclictest2187888-21kworker/u8:1+flush-179:021:50:170
2152506998779,5cyclictest2214909-21kworker/u8:2+flush-179:022:40:010
2152509998579,4cyclictest2169932-21kworker/u8:0+events_unbound20:15:161
2152509998578,4cyclictest2214909-21kworker/u8:2+flush-179:023:15:121
2152506998478,4cyclictest2187888-21kworker/u8:1+flush-179:021:35:210
2152506998474,7cyclictest2181554-21apt-get20:40:130
2152509998375,4cyclictest2152637-21kworker/u8:1+flush-179:020:30:251
2152509998375,4cyclictest2152637-21kworker/u8:1+flush-179:020:30:241
2152506998376,5cyclictest2165158-21kworker/u8:2+flush-179:020:00:000
2152506998375,6cyclictest2211503-21apt-get22:30:030
2152509998273,6cyclictest2206479-21kworker/u8:3+flush-179:022:30:161
2152509998270,9cyclictest2168569-21apt-get20:00:111
2152506998275,5cyclictest2181334-21kworker/u8:0+flush-179:020:50:260
2152506998275,5cyclictest2181334-21kworker/u8:0+flush-179:020:50:260
2152506998272,7cyclictest2174671-21latency_hist20:20:010
2152506998171,7cyclictest2237252-21latency_hist00:10:020
2152506998171,7cyclictest2223702-21kworker/u8:1+events_unbound23:40:010
2152506998072,6cyclictest2181334-21kworker/u8:0+events_unbound20:45:220
2152509997973,4cyclictest2187888-21kworker/u8:1+flush-179:021:10:251
2152509997971,6cyclictest2152637-21kworker/u8:1+events_unbound20:45:041
2152509997869,5cyclictest2152637-21kworker/u8:1+flush-179:019:20:141
2152509997869,5cyclictest2152637-21kworker/u8:1+flush-179:019:20:141
2152509997770,5cyclictest2213316-21kworker/u8:0+flush-179:023:05:001
2152509997768,6cyclictest2231886-21kworker/u8:0+events_unbound23:50:001
2152509997673,2cyclictest2214909-21kworker/u8:2+events_unbound22:50:161
2152506997669,5cyclictest2152637-21kworker/u8:1+flush-179:020:35:160
2152506997669,4cyclictest2169932-21kworker/u8:0+events_unbound20:10:010
2152506997665,8cyclictest2189506-21latency_hist21:05:010
2152509997564,8cyclictest2184982-21apt-get20:50:161
2152509997564,8cyclictest2184982-21apt-get20:50:161
2152506997562,10cyclictest2163796-21apt-get19:45:120
2152506997561,10cyclictest2217305-21apt-get22:50:070
2152509997468,4cyclictest2152637-21kworker/u8:1+flush-179:019:15:251
2152509997464,7cyclictest2214909-21kworker/u8:2+flush-179:023:10:171
2152506997366,5cyclictest2213316-21kworker/u8:0+flush-179:022:55:210
2152506997360,10cyclictest3705854-21cups-browsed00:00:000
2152509997266,4cyclictest2206479-21kworker/u8:3+flush-179:022:10:171
2152509997265,5cyclictest2231886-21kworker/u8:0+events_unbound23:55:241
2152509997257,13cyclictest2206479-21kworker/u8:3+flush-179:022:30:001
2152506997265,5cyclictest2174721-21kworker/u8:3+flush-179:020:30:260
2152506997265,5cyclictest2174721-21kworker/u8:3+flush-179:020:30:260
2152506997260,9cyclictest2152370-21latency_hist19:10:020
2152510997137,26cyclictest0-21swapper/223:45:592
2152506997164,5cyclictest2157213-21kworker/u8:2+events_unbound19:35:140
2152509997063,5cyclictest2213316-21kworker/u8:0+events_unbound22:55:201
2152509997061,6cyclictest2206441-21kworker/u8:2+events_unbound22:20:191
2152509997056,11cyclictest2230567-21apt-get23:40:131
2152506997060,8cyclictest2202461-21kworker/u8:0+flush-179:022:05:010
2152509996963,4cyclictest2152637-21kworker/u8:1+flush-179:020:05:221
2152509996961,5cyclictest2223906-21kworker/u8:3+events_unbound23:20:131
2152509996935,32cyclictest722-21Xorg19:38:581
2152506996963,5cyclictest2214909-21kworker/u8:2+flush-179:023:30:250
2152506996962,5cyclictest2189742-21kworker/u8:2+events_unbound21:10:240
2152506996959,7cyclictest2233740-21latency_hist23:55:010
2152509996863,4cyclictest2152637-21kworker/u8:1+events_unbound19:15:011
2152509996861,5cyclictest2194325-21kworker/u8:0+events_unbound21:30:191
2152506996863,4cyclictest2235316-21kworker/u8:1+flush-179:000:10:180
2152506996863,4cyclictest2235316-21kworker/u8:1+flush-179:000:10:180
2152506996862,4cyclictest2152637-21kworker/u8:1+events_unbound19:50:180
2152506996858,8cyclictest2240439-21apt-get00:20:010
2152506996856,9cyclictest2201159-21latency_hist21:50:010
2152509996762,4cyclictest2235316-21kworker/u8:1+flush-179:000:20:221
2152509996760,5cyclictest2186292-21kworker/u8:3+flush-179:020:55:141
2152506996761,4cyclictest2206441-21kworker/u8:2+events_unbound22:15:220
2152506996761,4cyclictest2152637-21kworker/u8:1+events_unbound19:45:010
2152509996660,4cyclictest2152637-21kworker/u8:1+flush-179:020:44:551
2152509996660,4cyclictest2152637-21kworker/u8:1+events_unbound19:55:121
2152506996660,4cyclictest2206441-21kworker/u8:2+events_unbound22:20:280
2152509996555,8cyclictest2194537-21apt-get21:20:141
2152509996458,5cyclictest2202461-21kworker/u8:0+events_unbound21:55:191
2152509996454,8cyclictest2158760-21latency_hist19:30:001
2152509996256,4cyclictest2199351-21kworker/u8:2+flush-179:021:40:161
2152509996252,8cyclictest2237488-21/usr/sbin/munin00:10:191
2152509996252,8cyclictest2237488-21/usr/sbin/munin00:10:191
2152509996250,9cyclictest2162211-21apt-get19:40:121
2152509996055,4cyclictest2173050-21kworker/u8:2+flush-179:020:20:251
2152506996054,5cyclictest1490643-21kworker/u8:4+events_unbound19:10:140
2152509995954,3cyclictest2152637-21kworker/u8:1+events_unbound19:45:231
2152506995954,4cyclictest2231886-21kworker/u8:0+flush-179:000:00:250
2152506995953,5cyclictest2244150-21kworker/u8:2+flush-179:000:35:270
2152506995953,5cyclictest2196162-21kworker/u8:3+events_unbound21:30:240
2152506995952,6cyclictest2157213-21kworker/u8:2+events_unbound19:35:000
2152509995752,4cyclictest2213316-21kworker/u8:0+flush-179:023:09:551
2152506995750,5cyclictest2187888-21kworker/u8:1+flush-179:021:45:000
2152506995545,7cyclictest2196126-21latency_hist21:30:010
2152509995348,4cyclictest2206479-21kworker/u8:3+events_unbound22:19:311
2152506995345,6cyclictest2214909-21kworker/u8:2+flush-179:023:10:110
215250699530,50cyclictest2214932-21apt-get22:40:030
2152509995248,3cyclictest2231886-21kworker/u8:0+flush-179:000:09:551
2152509995247,4cyclictest2187888-21kworker/u8:1+events_unbound21:35:231
2152509995245,5cyclictest2187888-21kworker/u8:1+flush-179:021:05:211
2152509995243,7cyclictest2237027-21kworker/u8:3+events_unbound00:15:141
2152506995247,4cyclictest2223702-21kworker/u8:1+flush-179:023:40:210
2152506995042,6cyclictest2206456-21latency_hist22:10:020
2152506995039,8cyclictest2227121-21latency_hist23:30:000
2152509994943,4cyclictest2241246-21kworker/u8:0+flush-179:000:30:001
2152509994940,7cyclictest2242534-21kworker/u8:1+events_unbound00:30:101
4505291480,13ptp4l61790irq/134-eno1-TxRx-019:09:573
2152509994741,4cyclictest2214909-21kworker/u8:2+events_unbound22:48:321
2152515994539,4cyclictest2209654-21turbostat22:19:593
215251599450,4cyclictest2222091-21turbostat23:10:003
2152509994540,4cyclictest2187888-21kworker/u8:1+flush-179:021:17:301
2152506994540,4cyclictest2226307-21kworker/u8:0+flush-179:023:20:240
2152506994539,5cyclictest2152637-21kworker/u8:1+events_unbound19:20:280
2152506994539,5cyclictest2152637-21kworker/u8:1+events_unbound19:20:270
215250699451,40cyclictest2204651-21munin-run22:00:000
215251099440,41cyclictest0-21swapper/219:19:592
2152506994441,2cyclictest0-21swapper/022:49:580
2152506994439,4cyclictest2214909-21kworker/u8:2+events_unbound23:10:000
2152509994340,2cyclictest2223702-21kworker/u8:1+events_unbound23:30:301
2152509994336,6cyclictest2152637-21kworker/u8:1+events_unbound19:55:001
215251099420,40cyclictest0-21swapper/200:06:592
215251599410,39cyclictest2163554-21turbostat19:45:003
2152509994134,6cyclictest2161999-21kworker/u8:3+events_unbound20:10:151
2152506994138,2cyclictest2195904-21systemd-run21:24:590
2152506994137,3cyclictest2155829-21kworker/u8:0+events_unbound19:25:220
2152506994136,4cyclictest2152637-21kworker/u8:1+events_unbound20:02:270
215251599400,38cyclictest2237007-21turbostat00:05:003
2152509994034,4cyclictest2214909-21kworker/u8:2+flush-179:023:30:001
215251599390,38cyclictest2195880-21turbostat21:25:003
215251599390,38cyclictest2176276-21turbostat20:25:003
215251599390,37cyclictest2174664-21turbostat20:20:003
215251099390,36cyclictest0-21swapper/200:17:582
215250699390,2cyclictest722-21Xorg23:49:580
2152515993835,2cyclictest0-21swapper/322:13:003
2152515993833,4cyclictest2160360-21turbostat19:35:003
2152510993835,2cyclictest0-21swapper/221:31:592
215251099380,36cyclictest0-21swapper/221:38:592
215251099380,1cyclictest0-21swapper/221:50:592
2152509993834,2cyclictest2196132-21munin-plugin-st21:30:001
215250999380,37cyclictest0-21swapper/121:48:001
215250699380,1cyclictest0-21swapper/000:27:590
2152515993734,2cyclictest0-21swapper/323:22:593
2152515993734,2cyclictest0-21swapper/320:39:593
2152515993731,5cyclictest2238843-21turbostat00:15:003
2152515993731,5cyclictest2238843-21turbostat00:15:003
215251599370,36cyclictest0-21swapper/321:41:583
215251599370,36cyclictest0-21swapper/319:22:003
215251599370,36cyclictest0-21swapper/319:21:593
2152510993735,1cyclictest0-21swapper/220:59:592
2152510993734,2cyclictest0-21swapper/223:41:002
2152510993734,2cyclictest0-21swapper/222:31:002
2152510993734,2cyclictest0-21swapper/220:14:552
2152510993733,2cyclictest0-21swapper/222:09:552
215251099370,2cyclictest0-21swapper/200:11:592
215251099370,2cyclictest0-21swapper/200:11:592
2152515993634,1cyclictest0-21swapper/323:59:553
215251599360,35cyclictest0-21swapper/323:17:003
215251599360,35cyclictest0-21swapper/322:22:593
215251599360,1cyclictest0-21swapper/321:46:003
215251599360,1cyclictest0-21swapper/320:02:593
215251599360,1cyclictest0-21swapper/319:49:553
2152510993634,1cyclictest0-21swapper/219:34:592
2152510993633,2cyclictest0-21swapper/222:02:592
2152515993533,1cyclictest0-21swapper/322:25:593
2152515993533,1cyclictest0-21swapper/321:19:003
2152509993530,4cyclictest2187888-21kworker/u8:1+events_unbound22:08:311
215250699350,33cyclictest2220499-21http23:00:170
2152515993432,1cyclictest0-21swapper/323:03:593
2152515993432,1cyclictest0-21swapper/322:40:153
215251599340,32cyclictest0-21swapper/323:45:593
2152510993432,1cyclictest0-21swapper/222:47:592
2152510993432,1cyclictest0-21swapper/200:24:592
2152510993431,2cyclictest0-21swapper/221:06:592
2152510993431,2cyclictest0-21swapper/219:48:592
2152515993330,2cyclictest0-21swapper/320:25:143
2152510993330,2cyclictest0-21swapper/221:10:582
2152510993330,2cyclictest0-21swapper/221:04:002
2152515993230,1cyclictest0-21swapper/322:06:593
2152515993230,1cyclictest0-21swapper/321:31:003
2152515993229,2cyclictest0-21swapper/322:31:593
2152510993229,2cyclictest0-21swapper/221:24:592
215251099320,2cyclictest0-21swapper/222:10:592
215251099320,2cyclictest0-21swapper/221:59:002
215251599314,1cyclictest0-21swapper/319:50:163
2152515993129,1cyclictest0-21swapper/322:53:583
2152515993129,1cyclictest0-21swapper/321:52:593
2152510993129,1cyclictest0-21swapper/223:26:592
2152510993128,2cyclictest0-21swapper/219:59:002
4505291300,6ptp4l61790irq/134-eno1-TxRx-000:10:023
2152515993028,1cyclictest0-21swapper/300:39:003
2152515993027,2cyclictest0-21swapper/320:10:163
215251599300,29cyclictest0-21swapper/319:35:143
2152515992927,1cyclictest0-21swapper/323:10:083
2152515992926,2cyclictest0-21swapper/322:00:193
215251599290,21cyclictest0-21swapper/319:10:143
2152510992927,1cyclictest0-21swapper/221:40:162
2152510992926,2cyclictest0-21swapper/223:10:082
2152510992925,2cyclictest0-21swapper/220:04:552
215251099290,27cyclictest0-21swapper/223:55:152
215251099290,27cyclictest0-21swapper/222:35:122
215251099290,27cyclictest0-21swapper/220:30:022
215251099290,27cyclictest0-21swapper/220:30:022
215251099290,27cyclictest0-21swapper/219:50:162
4505291280,6ptp4l61790irq/134-eno1-TxRx-023:35:013
4505291280,6ptp4l441ktimers/322:50:003
2152515992826,1cyclictest0-21swapper/320:58:593
2152515992826,1cyclictest0-21swapper/300:20:023
2152515992826,1cyclictest0-21swapper/300:15:143
215251599280,2cyclictest0-21swapper/320:30:023
215251599280,2cyclictest0-21swapper/320:30:023
215251599280,26cyclictest0-21swapper/320:45:043
2152510992826,1cyclictest0-21swapper/222:43:592
2152510992825,2cyclictest0-21swapper/219:41:592
215251099280,2cyclictest0-21swapper/223:15:112
215251099280,26cyclictest0-21swapper/221:28:592
215251099280,26cyclictest0-21swapper/220:35:142
215251099280,26cyclictest0-21swapper/219:20:152
215251099280,26cyclictest0-21swapper/219:20:142
4505291270,6ptp4l61790irq/134-eno1-TxRx-021:39:563
4505291270,6ptp4l61790irq/134-eno1-TxRx-000:29:563
2152515992725,1cyclictest0-21swapper/319:18:593
215251599270,1cyclictest0-21swapper/322:55:123
215251599270,1cyclictest0-21swapper/321:55:113
2152510992725,1cyclictest0-21swapper/200:39:592
2152510992724,2cyclictest0-21swapper/223:37:002
215251099270,25cyclictest0-21swapper/223:24:002
215251099270,24cyclictest0-21swapper/222:23:582
4505291260,6ptp4l441ktimers/323:39:563
4505291260,6ptp4l441ktimers/323:29:553
4505291260,5ptp4l441ktimers/323:44:563
4505291260,5ptp4l441ktimers/300:35:013
2152515992624,1cyclictest0-21swapper/323:53:003
215251599260,25cyclictest0-21swapper/320:42:593
2152510992623,2cyclictest0-21swapper/221:47:592
2152510992623,2cyclictest0-21swapper/220:19:562
215251099260,2cyclictest0-21swapper/223:30:162
215251099260,2cyclictest0-21swapper/222:30:002
215251099260,24cyclictest0-21swapper/220:44:592
4505291250,5ptp4l441ktimers/321:29:563
4505291250,4ptp4l441ktimers/320:10:013
2152515992522,2cyclictest0-21swapper/319:57:593
2152510992523,1cyclictest0-21swapper/219:14:562
2152510992523,1cyclictest0-21swapper/200:29:002
4505291240,4ptp4l61790irq/134-eno1-TxRx-022:39:553
2152515992422,1cyclictest0-21swapper/320:54:593
2152515992422,1cyclictest0-21swapper/320:54:593
2152515992422,1cyclictest0-21swapper/319:29:553
2152510992422,1cyclictest0-21swapper/222:15:152
2152509992419,3cyclictest0-21swapper/119:10:011
4505291230,4ptp4l61790irq/134-eno1-TxRx-021:04:553
2152515992321,1cyclictest0-21swapper/321:12:593
2152510992320,2cyclictest0-21swapper/220:26:592
2152510992320,2cyclictest0-21swapper/220:05:592
2152510992319,2cyclictest0-21swapper/219:26:592
2152515992220,1cyclictest0-21swapper/321:08:583
2152510992220,1cyclictest0-21swapper/223:50:082
2152510992220,1cyclictest0-21swapper/200:32:592
2152510992219,2cyclictest0-21swapper/220:20:142
2152510992119,1cyclictest0-21swapper/221:18:002
2152510992119,1cyclictest0-21swapper/220:47:592
2152510991815,2cyclictest0-21swapper/223:09:002
2152510991714,2cyclictest0-21swapper/200:00:592
2152510991713,2cyclictest0-21swapper/222:59:552
2152510991614,1cyclictest0-21swapper/222:52:592
2152510991513,1cyclictest0-21swapper/223:05:002
215251099129,2cyclictest0-21swapper/220:51:002
215251099129,2cyclictest0-21swapper/220:51:002
215251099119,1cyclictest0-21swapper/219:39:592
21525109930,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional