You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 04:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Wed Apr 02, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1002626359,0sleep00-21swapper/018:30:390
336723826,0sleep20-21swapper/218:29:122
1017523827,0sleep10-21swapper/118:32:241
1001423727,0sleep30-21swapper/318:30:303
329990,0watchdog/30-21swapper/322:52:063
29925080,0irq/32-eth0-rx-0-21swapper/323:04:243
29925070,0irq/32-eth0-rx-0-21swapper/323:54:323
29925070,0irq/32-eth0-rx-0-21swapper/322:13:563
29925070,0irq/32-eth0-rx-0-21swapper/321:26:243
8315060,0irq/26-0000:00:17202-21unixbench_singl19:19:333
8315060,0irq/26-0000:00:0-21swapper/318:34:053
339960,0migration/34502-21kthreadcore23:34:203
339960,0migration/325691-21switchtime22:14:263
329960,0watchdog/328945-21tr19:09:183
329960,0watchdog/30-21swapper/323:49:223
329960,0watchdog/30-21swapper/323:29:303
329960,0watchdog/30-21swapper/323:17:463
329960,0watchdog/30-21swapper/323:17:463
329960,0watchdog/30-21swapper/323:11:263
329960,0watchdog/30-21swapper/322:59:063
329960,0watchdog/30-21swapper/322:41:023
329960,0watchdog/30-21swapper/322:36:423
329960,0watchdog/30-21swapper/322:24:543
329960,0watchdog/30-21swapper/322:00:063
329960,0watchdog/30-21swapper/321:31:463
329960,0watchdog/30-21swapper/321:12:463
329960,0watchdog/30-21swapper/321:04:223
329960,0watchdog/30-21swapper/320:59:183
329960,0watchdog/30-21swapper/320:54:183
329960,0watchdog/30-21swapper/320:53:103
329960,0watchdog/30-21swapper/320:45:343
329960,0watchdog/30-21swapper/320:43:223
329960,0watchdog/30-21swapper/320:36:063
329960,0watchdog/30-21swapper/320:23:143
329960,0watchdog/30-21swapper/319:27:143
329960,0watchdog/30-21swapper/318:54:263
29935060,0irq/33-eth0-tx-30125-21munin-node20:04:200
29925060,0irq/32-eth0-rx-29055-21perl21:34:163
29925060,0irq/32-eth0-rx-25451-21diskstats18:44:203
29925060,0irq/32-eth0-rx-0-21swapper/323:40:583
29925060,0irq/32-eth0-rx-0-21swapper/323:23:513
29925060,0irq/32-eth0-rx-0-21swapper/322:46:163
29925060,0irq/32-eth0-rx-0-21swapper/322:21:163
29925060,0irq/32-eth0-rx-0-21swapper/321:56:533
29925060,0irq/32-eth0-rx-0-21swapper/321:51:213
29925060,0irq/32-eth0-rx-0-21swapper/321:45:203
29925060,0irq/32-eth0-rx-0-21swapper/321:15:133
29925060,0irq/32-eth0-rx-0-21swapper/300:03:323
149960,0migration/08421-21sh22:04:060
149960,0migration/06693-21munin-node21:14:220
149960,0migration/017357-21kthreadcore19:54:190
103899965,0cyclictest31657-21sadc22:44:061
103899960,0cyclictest7037-21kthreadcore19:14:311
103899960,0cyclictest0-21swapper/121:04:521
103889965,0cyclictest26601-21munin-node20:24:340
8315050,0irq/26-0000:00:0-21swapper/318:39:443
339950,0migration/37404-21kthreadcore21:39:243
339950,0migration/32189-21fschecks_time20:29:203
339950,0migration/318123-21kthreadcore18:59:253
339950,0migration/313998-21kthreadcore20:14:303
329950,0watchdog/30-21swapper/322:54:103
329950,0watchdog/30-21swapper/320:24:383
329950,0watchdog/30-21swapper/320:11:223
329950,0watchdog/30-21swapper/320:05:143
329950,0watchdog/30-21swapper/319:59:063
329950,0watchdog/30-21swapper/319:54:543
329950,0watchdog/30-21swapper/319:49:183
329950,0watchdog/30-21swapper/319:44:383
329950,0watchdog/30-21swapper/319:39:223
329950,0watchdog/30-21swapper/319:34:183
329950,0watchdog/30-21swapper/319:29:463
329950,0watchdog/30-21swapper/319:16:383
329950,0watchdog/30-21swapper/318:50:583
29935050,0irq/33-eth0-tx-7071-21munin-node18:54:230
29935050,0irq/33-eth0-tx-6786-21rt-features21:14:222
29935050,0irq/33-eth0-tx-4792-21munin-node20:54:232
29935050,0irq/33-eth0-tx-32446-21munin-node21:59:192
29935050,0irq/33-eth0-tx-32180-21memory20:44:272
29935050,0irq/33-eth0-tx-24376-21fschecks_count21:54:162
29935050,0irq/33-eth0-tx-24144-21pmu-power23:44:272
29935050,0irq/33-eth0-tx-23324-21ssh21:53:272
29935050,0irq/33-eth0-tx-19424-21munin-node20:19:190
29935050,0irq/33-eth0-tx-1422-21munin-node20:29:220
29935050,0irq/33-eth0-tx-12834-21chrt22:29:242
29935050,0irq/33-eth0-tx-0-21swapper/223:06:272
29935050,0irq/33-eth0-tx-0-21swapper/222:46:142
29935050,0irq/33-eth0-tx-0-21swapper/222:05:122
29935050,0irq/33-eth0-tx-0-21swapper/221:10:082
29935050,0irq/33-eth0-tx-0-21swapper/220:49:282
29935050,0irq/33-eth0-tx-0-21swapper/020:34:090
29935050,0irq/33-eth0-tx-0-21swapper/018:50:200
29925050,0irq/32-eth0-rx-9798-21if_err_eth1.322:04:223
29925050,0irq/32-eth0-rx-29301-21smartctl23:24:313
29925050,0irq/32-eth0-rx-21784-21df19:04:203
29925050,0irq/32-eth0-rx-12665-21kthreadcore21:19:263
29925050,0irq/32-eth0-rx-0-21swapper/323:44:103
29925050,0irq/32-eth0-rx-0-21swapper/322:29:243
259950,0migration/24791-21fschecks_count23:59:222
259950,0migration/230312-21kthreadcore19:09:232
259950,0migration/226526-21latency_hist20:24:062
259950,0migration/225051-21kthreadcore22:39:242
259950,0migration/224399-21munin-node21:24:302
259950,0migration/223052-21sh23:24:182
259950,0migration/217218-21sh21:23:282
259950,0migration/210538-21kthreadcore18:54:282
249950,0watchdog/29984-21fschecks_count23:39:222
249950,0watchdog/214773-21kthreadcore20:14:302
179950,0migration/17385-21taskset22:47:341
179950,0migration/128147-21taskset19:06:031
179950,0migration/114572-21kthreadcore21:19:261
179950,0migration/112402-21kthreadcore18:34:251
169950,0watchdog/123537-21ntpdc20:39:261
149950,0migration/09978-21kthreadcore19:44:240
149950,0migration/09417-21missed_timers19:14:320
149950,0migration/07013-21kthreadcore22:24:240
149950,0migration/06790-21if_eth023:14:170
149950,0migration/06790-21if_eth023:14:170
149950,0migration/06284-21irqrtprio20:54:240
149950,0migration/04683-21sh23:12:080
149950,0migration/0442-21latency_hist19:39:060
149950,0migration/03983-21kthreadcore22:44:270
149950,0migration/028279-21taskset23:54:060
149950,0migration/027972-21sh23:53:510
149950,0migration/027909-21taskset22:19:140
149950,0migration/026972-21kthreadcore21:54:200
149950,0migration/025429-21taskset19:32:560
149950,0migration/024721-21ksmtuned21:25:120
149950,0migration/023113-21sh21:52:510
149950,0migration/022169-21latency_hist21:04:060
149950,0migration/017978-21sh23:44:140
149950,0migration/01754-21sh20:47:450
149950,0migration/015084-21munin-node22:34:140
149950,0migration/01381-21switchtime23:04:290
149950,0migration/012540-21kthreadcore22:54:180
149950,0migration/011966-21taskset00:02:580
149950,0migration/010904-21irqcore23:39:250
149950,0migration/010542-21kthreadcore22:04:240
149950,0migration/010135-21munin-node21:19:230
103899955,0cyclictest0-21swapper/123:46:091
103899955,0cyclictest0-21swapper/121:45:421
103899953,0cyclictest3599-21lldpd22:12:401
103899950,0cyclictest0-21swapper/123:55:091
103899950,0cyclictest0-21swapper/123:49:091
103899950,0cyclictest0-21swapper/123:34:101
103899950,0cyclictest0-21swapper/123:27:331
103899950,0cyclictest0-21swapper/123:01:511
103899950,0cyclictest0-21swapper/122:29:041
103899950,0cyclictest0-21swapper/122:15:591
103899950,0cyclictest0-21swapper/121:29:001
103889955,0cyclictest0-21swapper/023:26:200
103889955,0cyclictest0-21swapper/022:51:580
103889954,0cyclictest18959-21awk19:24:230
103889950,0cyclictest0-21swapper/023:19:430
103889950,0cyclictest0-21swapper/021:41:580
103889950,0cyclictest0-21swapper/021:33:180
29935040,0irq/33-eth0-tx-0-21swapper/223:54:272
29935040,0irq/33-eth0-tx-0-21swapper/223:51:062
29935040,0irq/33-eth0-tx-0-21swapper/223:19:092
29935040,0irq/33-eth0-tx-0-21swapper/223:09:372
29935040,0irq/33-eth0-tx-0-21swapper/222:54:152
29935040,0irq/33-eth0-tx-0-21swapper/222:49:422
29935040,0irq/33-eth0-tx-0-21swapper/222:19:102
29935040,0irq/33-eth0-tx-0-21swapper/222:15:432
29935040,0irq/33-eth0-tx-0-21swapper/221:44:252
29935040,0irq/33-eth0-tx-0-21swapper/221:34:232
29935040,0irq/33-eth0-tx-0-21swapper/221:30:032
29935040,0irq/33-eth0-tx-0-21swapper/220:59:182
29935040,0irq/33-eth0-tx-0-21swapper/019:09:190
29935040,0irq/33-eth0-tx-0-21swapper/018:39:180
259940,0migration/29538-21fschecks_time20:34:232
259940,0migration/26706-21munin-node23:14:162
259940,0migration/26706-21munin-node23:14:162
259940,0migration/24837-21kthreadcore20:29:242
259940,0migration/24449-21latency_hist21:39:062
259940,0migration/220251-21fschecks_time20:19:202
259940,0migration/21981-21kthreadcore20:04:252
259940,0migration/215603-21latency_hist19:54:062
259940,0migration/214341-21kthreadcore19:19:252
259940,0migration/210934-21diskmemload22:24:372
259940,0migration/210109-21runrttasks19:14:452
249940,0watchdog/2953-21telnet23:34:142
179940,0migration/19755-21kthreadcore20:09:251
179940,0migration/131389-21kthreadcore22:19:251
179940,0migration/130888-21missed_timers18:44:261
179940,0migration/128143-21kthreadcore20:44:251
179940,0migration/125711-21latency_hist19:34:061
179940,0migration/122907-21taskset22:38:421
179940,0migration/115438-21kthreadcore20:59:241
169940,0watchdog/18772-21munin-node20:34:221
169940,0watchdog/18380-21sed18:54:261
169940,0watchdog/131405-21ssh23:29:101
169940,0watchdog/127400-21ssh23:04:141
169940,0watchdog/116905-21ssh23:41:421
169940,0watchdog/1118402sleep100:02:301
169940,0watchdog/110856-21sh19:19:181
169940,0watchdog/10-21swapper/119:54:141
149940,0migration/022998-21kthreadcore20:39:250
149940,0migration/013634-21sh22:31:410
103909944,0cyclictest0-21swapper/223:29:222
103909943,0cyclictest7629-21sadc19:44:052
103909943,0cyclictest6040-21awk20:09:222
103909943,0cyclictest3599-21lldpd22:34:142
103909943,0cyclictest3599-21lldpd22:10:102
103909943,0cyclictest3599-21lldpd19:37:512
103909943,0cyclictest3599-21lldpd18:51:512
103909943,0cyclictest24081-21awk19:59:222
103909943,0cyclictest23305-21awk21:04:212
103909943,0cyclictest18522-21awk20:39:222
103909943,0cyclictest16801-21awk19:54:172
103909943,0cyclictest15455-21awk18:59:232
103909943,0cyclictest11762-21awk18:34:232
103909940,0cyclictest0-21swapper/222:59:212
103899944,0cyclictest0-21swapper/122:54:271
103899944,0cyclictest0-21swapper/121:18:261
103899944,0cyclictest0-21swapper/119:39:161
103899943,0cyclictest3599-21lldpd23:20:471
103899943,0cyclictest3599-21lldpd23:18:471
103899943,0cyclictest3599-21lldpd23:18:471
103899943,0cyclictest3599-21lldpd22:49:141
103899943,0cyclictest3599-21lldpd22:05:391
103899943,0cyclictest3599-21lldpd21:50:091
103899943,0cyclictest3599-21lldpd20:52:271
103899943,0cyclictest3599-21lldpd20:14:231
103899943,0cyclictest3599-21lldpd19:44:211
103899943,0cyclictest3262-21irqbalance22:30:201
103899943,0cyclictest3262-21irqbalance20:29:201
103899943,0cyclictest3262-21irqbalance20:25:201
103899943,0cyclictest3262-21irqbalance20:19:301
103899943,0cyclictest3262-21irqbalance19:29:101
103899943,0cyclictest3262-21irqbalance19:10:001
103899943,0cyclictest3262-21irqbalance18:49:101
103899943,0cyclictest3262-21irqbalance18:39:301
103899943,0cyclictest29735-21sed21:34:191
103899943,0cyclictest28051-21sadc21:34:061
103899943,0cyclictest24009-21awk19:59:221
103899943,0cyclictest19062-21awk19:24:241
103899943,0cyclictest15328-21awk18:59:221
103899943,0cyclictest1104-21awk21:59:211
103889944,0cyclictest0-21swapper/023:32:580
103889943,0cyclictest23871-21snmpd23:34:400
103889943,0cyclictest23871-21snmpd19:08:240
103889943,0cyclictest23871-21snmpd18:59:240
103889943,0cyclictest19968-21awk22:59:240
179930,0migration/125865-21kthreadcore21:54:201
169930,0watchdog/18497-21diskmemload20:55:261
169930,0watchdog/10-21swapper/123:10:341
169930,0watchdog/10-21swapper/121:40:301
103909933,0cyclictest3599-21lldpd19:47:522
103909932,0cyclictest3599-21lldpd18:39:212
103909930,0cyclictest25764-21perl18:44:212
103909930,0cyclictest0-21swapper/219:04:302
103899932,0cyclictest3599-21lldpd21:09:321
103899932,0cyclictest31159-21awk20:04:221
103899930,0cyclictest0-21swapper/119:50:181
103889933,0cyclictest32583-21kworker/0:222:39:160
103889933,0cyclictest32583-21kworker/0:222:09:080
103889933,0cyclictest32583-21kworker/0:221:34:380
103889933,0cyclictest32583-21kworker/0:221:09:240
103889933,0cyclictest32583-21kworker/0:220:59:060
103889933,0cyclictest32583-21kworker/0:220:49:460
103889933,0cyclictest27674-21kworker/0:120:14:100
103889933,0cyclictest27674-21kworker/0:120:09:140
103889933,0cyclictest27674-21kworker/0:119:59:110
103889933,0cyclictest27674-21kworker/0:119:49:090
103889933,0cyclictest27674-21kworker/0:119:34:090
103889933,0cyclictest27674-21kworker/0:119:19:250
103889933,0cyclictest15821-21kworker/0:018:44:170
103889933,0cyclictest15821-21kworker/0:018:34:130
103889932,0cyclictest20468-21awk22:14:180
103889932,0cyclictest13230-21munin-node21:44:230
249920,0watchdog/20-21swapper/219:29:062
249920,0watchdog/20-21swapper/219:24:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional