You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-12-26 - 13:31
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Dec 26, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
371924937,0sleep20-21swapper/206:38:292
380124637,0sleep10-21swapper/106:39:201
365624434,0sleep30-21swapper/306:37:423
355323826,0sleep00-21swapper/006:36:310
3917991414,0cyclictest23871-21snmpd11:43:350
329980,0watchdog/331026-21tr06:55:153
329980,0watchdog/30-21swapper/309:05:073
329970,0watchdog/30-21swapper/310:40:473
329970,0watchdog/30-21swapper/309:00:593
39229966,0cyclictest0-21swapper/210:23:442
39229960,0cyclictest0-21swapper/211:40:312
39229960,0cyclictest0-21swapper/207:04:562
39189966,0cyclictest15374-21diskmemload10:20:001
39179963,0cyclictest27931-21munin-run09:09:500
339960,0migration/31307-21sh11:55:053
329960,0watchdog/36456-21df_abs09:19:593
329960,0watchdog/3299250irq/32-eth0-rx-11:00:063
329960,0watchdog/30-21swapper/312:06:583
329960,0watchdog/30-21swapper/311:53:463
329960,0watchdog/30-21swapper/311:49:143
329960,0watchdog/30-21swapper/311:40:223
329960,0watchdog/30-21swapper/311:26:343
329960,0watchdog/30-21swapper/311:20:423
329960,0watchdog/30-21swapper/311:14:503
329960,0watchdog/30-21swapper/310:55:023
329960,0watchdog/30-21swapper/310:25:473
329960,0watchdog/30-21swapper/310:22:033
329960,0watchdog/30-21swapper/310:16:313
329960,0watchdog/30-21swapper/310:05:513
329960,0watchdog/30-21swapper/310:00:153
329960,0watchdog/30-21swapper/309:55:313
329960,0watchdog/30-21swapper/309:50:473
329960,0watchdog/30-21swapper/309:37:353
329960,0watchdog/30-21swapper/308:12:313
329960,0watchdog/30-21swapper/307:59:513
329960,0watchdog/30-21swapper/307:52:313
329960,0watchdog/30-21swapper/307:35:033
329960,0watchdog/30-21swapper/307:30:153
329960,0watchdog/30-21swapper/307:30:153
29935060,0irq/33-eth0-tx-822-21munin-node07:45:030
29935060,0irq/33-eth0-tx-22281-21yum08:40:170
29925060,0irq/32-eth0-rx-11739-21cut09:45:083
29925060,0irq/32-eth0-rx-0-21swapper/312:00:303
29925060,0irq/32-eth0-rx-0-21swapper/311:30:233
29925060,0irq/32-eth0-rx-0-21swapper/311:14:073
29925060,0irq/32-eth0-rx-0-21swapper/310:39:473
29925060,0irq/32-eth0-rx-0-21swapper/309:25:273
29925060,0irq/32-eth0-rx-0-21swapper/307:10:023
149960,0migration/08149-21sh09:41:450
149960,0migration/032490-21sh09:38:450
149960,0migration/031985-21smart_sda06:55:200
149960,0migration/027814-21sh11:27:210
149960,0migration/019374-21latency10:10:090
149960,0migration/017499-21mdstat11:20:070
4150,0ktimersoftd/00-21swapper/011:07:170
39229955,0cyclictest0-21swapper/211:57:422
39229955,0cyclictest0-21swapper/210:50:322
39229955,0cyclictest0-21swapper/210:36:502
39229955,0cyclictest0-21swapper/209:34:142
39229955,0cyclictest0-21swapper/208:20:182
39229955,0cyclictest0-21swapper/207:50:202
39229954,0cyclictest1898-21kthreadcore08:50:092
39229950,0cyclictest0-21swapper/211:02:222
39229950,0cyclictest0-21swapper/209:34:522
39189955,0cyclictest0-21swapper/110:30:051
39179950,0cyclictest0-21swapper/010:57:210
339950,0migration/33217-21munin-node07:20:113
339950,0migration/331510-21kthreadcore10:45:113
339950,0migration/326062-21latency_hist07:39:513
339950,0migration/323202-21fschecks_time07:15:043
339950,0migration/313406-21latency_hist10:09:503
329950,0watchdog/3299250irq/32-eth0-rx-08:54:513
329950,0watchdog/317851-21netstat08:15:113
329950,0watchdog/312443-21if_eth106:45:073
329950,0watchdog/30-21swapper/310:50:543
329950,0watchdog/30-21swapper/309:41:073
329950,0watchdog/30-21swapper/309:30:393
329950,0watchdog/30-21swapper/309:14:513
329950,0watchdog/30-21swapper/309:09:553
329950,0watchdog/30-21swapper/308:44:513
329950,0watchdog/30-21swapper/308:40:073
329950,0watchdog/30-21swapper/308:35:073
329950,0watchdog/30-21swapper/308:29:553
329950,0watchdog/30-21swapper/308:25:153
329950,0watchdog/30-21swapper/308:19:513
329950,0watchdog/30-21swapper/308:05:033
329950,0watchdog/30-21swapper/307:56:353
329950,0watchdog/30-21swapper/307:26:033
329950,0watchdog/30-21swapper/307:06:233
329950,0watchdog/30-21swapper/307:00:593
329950,0watchdog/30-21swapper/306:43:353
29935050,0irq/33-eth0-tx-26885-21kthreadcore08:45:112
29935050,0irq/33-eth0-tx-0-21swapper/211:50:022
29935050,0irq/33-eth0-tx-0-21swapper/211:30:592
29935050,0irq/33-eth0-tx-0-21swapper/210:28:242
29935050,0irq/33-eth0-tx-0-21swapper/208:54:502
29935050,0irq/33-eth0-tx-0-21swapper/007:57:520
29935050,0irq/33-eth0-tx-0-21swapper/007:12:470
29935050,0irq/33-eth0-tx-0-21swapper/006:50:060
29925050,0irq/32-eth0-rx-0-21swapper/311:35:233
29925050,0irq/32-eth0-rx-0-21swapper/311:04:593
29925050,0irq/32-eth0-rx-0-21swapper/310:30:033
29925050,0irq/32-eth0-rx-0-21swapper/308:49:563
29925050,0irq/32-eth0-rx-0-21swapper/307:45:023
29925050,0irq/32-eth0-rx-0-21swapper/306:50:113
27150,0ktimersoftd/20-21swapper/208:06:062
259950,0migration/27094-21kthreadcore08:10:082
259950,0migration/26106-21taskset11:36:502
259950,0migration/232031-21munin-node07:40:112
259950,0migration/231522-21kthreadcore07:20:062
259950,0migration/227531-21sh09:08:372
259950,0migration/220912-21latency07:10:102
259950,0migration/213939-21kthreadcore08:15:092
259950,0migration/211965-21kthreadcore09:45:082
249950,0watchdog/232099-21mdstat08:25:112
249950,0watchdog/214960-21ssh10:58:582
179950,0migration/18433-21sh11:12:521
179950,0migration/131332-21sh09:35:131
179950,0migration/128735-21unixbench-2d07:15:181
179950,0migration/123304-21munin-node09:30:021
179950,0migration/122720-21idleruntime-cro09:29:501
179950,0migration/111332-21taskset07:29:011
169950,0watchdog/116180-21cut07:30:111
169950,0watchdog/116180-21cut07:30:111
149950,0migration/06419-21kthreadcore08:30:070
149950,0migration/06231-21sh11:37:100
149950,0migration/04037-21kthreadcore07:00:070
149950,0migration/032749-21sh11:53:040
149950,0migration/032096-21sh10:24:200
149950,0migration/032050-21kthreadcore08:05:100
149950,0migration/03182-21sh10:00:550
149950,0migration/030920-21kthreadcore09:10:090
149950,0migration/030900-21irqrtprio07:20:060
149950,0migration/028394-21kthreadcore07:40:090
149950,0migration/027892-21kthreadcore07:15:100
149950,0migration/024721-21munin-node07:35:090
149950,0migration/022397-21kthreadcore10:40:050
149950,0migration/021280-21kthreadcore11:00:050
149950,0migration/021111-21kthreadcore11:45:080
149950,0migration/014887-21turbostat.cron07:09:460
149950,0migration/014750-21timerandwakeup08:35:160
149950,0migration/014137-21sh09:24:250
149950,0migration/014015-21kthreadcore09:00:040
149950,0migration/013531-21kthreadcore07:50:080
149950,0migration/012383-21taskset10:52:380
149950,0migration/012172-21cpuspeed_turbos08:15:040
149950,0migration/011049-21taskset06:44:210
149950,0migration/010658-21sh11:18:120
149950,0migration/01004-21kthreadcore11:30:100
8315040,0irq/26-0000:00:0-21swapper/208:35:052
39229943,0cyclictest23871-21snmpd10:00:472
39229943,0cyclictest23871-21snmpd06:57:162
39229943,0cyclictest23871-21snmpd06:41:462
39229943,0cyclictest19912-21awk07:35:052
39229943,0cyclictest19547-21awk06:50:052
39229943,0cyclictest17640-21grep07:30:132
39229943,0cyclictest17640-21grep07:30:132
39189944,0cyclictest0-21swapper/110:17:281
39189944,0cyclictest0-21swapper/109:12:311
39189944,0cyclictest0-21swapper/108:52:011
39189943,0cyclictest9587-21awk08:35:071
39189943,0cyclictest9318-21awk07:50:061
39189943,0cyclictest8952-21awk07:05:051
39189943,0cyclictest5235-21awk06:40:061
39189943,0cyclictest28362-21sadc09:59:501
39189943,0cyclictest26080-21awk11:50:061
39189943,0cyclictest23871-21snmpd11:25:471
39189943,0cyclictest23871-21snmpd08:18:321
39189943,0cyclictest20681-21awk09:05:041
39189943,0cyclictest17340-21awk11:45:051
39189943,0cyclictest14619-21awk10:10:061
39189940,0cyclictest0-21swapper/111:55:131
39179944,0cyclictest0-21swapper/009:15:490
39179943,0cyclictest7112-21kworker/0:011:56:410
39179943,0cyclictest25298-21awk08:45:090
39179943,0cyclictest23871-21snmpd08:26:320
39179943,0cyclictest13394-21awk12:05:050
39179943,0cyclictest12466-21awk06:45:070
29935040,0irq/33-eth0-tx-24257-21ssh11:04:562
29935040,0irq/33-eth0-tx-22921-21ssh10:17:322
29935040,0irq/33-eth0-tx-15374-21diskmemload09:55:142
29935040,0irq/33-eth0-tx-15374-21diskmemload09:00:202
29935040,0irq/33-eth0-tx-14538-21munin-node09:25:082
29935040,0irq/33-eth0-tx-0-21swapper/212:05:032
29935040,0irq/33-eth0-tx-0-21swapper/212:01:542
29935040,0irq/33-eth0-tx-0-21swapper/211:46:012
29935040,0irq/33-eth0-tx-0-21swapper/211:25:142
29935040,0irq/33-eth0-tx-0-21swapper/211:17:052
29935040,0irq/33-eth0-tx-0-21swapper/211:09:562
29935040,0irq/33-eth0-tx-0-21swapper/210:29:592
29935040,0irq/33-eth0-tx-0-21swapper/210:10:132
29935040,0irq/33-eth0-tx-0-21swapper/209:39:562
29935040,0irq/33-eth0-tx-0-21swapper/209:14:532
29935040,0irq/33-eth0-tx-0-21swapper/209:10:092
29935040,0irq/33-eth0-tx-0-21swapper/008:22:000
29935040,0irq/33-eth0-tx-0-21swapper/007:30:090
29935040,0irq/33-eth0-tx-0-21swapper/007:30:090
259940,0migration/26950-21munin-node07:45:112
259940,0migration/24992-21munin-node10:05:022
259940,0migration/23208-21kthreadcore07:00:072
259940,0migration/228359-21pmu-power07:15:132
259940,0migration/216743-21if_eth008:40:052
259940,0migration/213853-21kthreadcore11:20:052
259940,0migration/211648-21cpuspeed_turbos06:45:042
249940,0watchdog/26457-21df_abs09:19:592
249940,0watchdog/25593-21iostat_ios07:25:072
249940,0watchdog/24930-21grep08:30:072
249940,0watchdog/216768-21irqcore07:55:072
249940,0watchdog/20-21swapper/210:45:072
249940,0watchdog/20-21swapper/210:40:112
249940,0watchdog/20-21swapper/209:50:112
20633240,0sleep123933-21kthreadcore07:35:081
179940,0migration/17540-21kthreadcore10:50:091
179940,0migration/14410-21taskset10:04:191
179940,0migration/13783-21taskset11:31:171
179940,0migration/13260-21mdstat08:05:111
179940,0migration/127399-21kthreadcore08:45:111
179940,0migration/127212-21if_eth107:40:071
179940,0migration/125709-21taskset08:20:521
179940,0migration/125475-21kthreadcore08:00:071
179940,0migration/123915-21kthreadcore06:50:091
179940,0migration/123226-21runrttasks08:43:191
179940,0migration/122481-21taskset11:00:451
179940,0migration/112634-21taskset10:08:021
179940,0migration/112124-21kthreadcore11:40:051
179940,0migration/110422-21fschecks_time09:45:041
169940,0watchdog/16458-21df_abs09:19:591
169940,0watchdog/14659-21munin-node08:10:111
169940,0watchdog/1444-21df10:25:031
169940,0watchdog/14412-21cut07:00:071
169940,0watchdog/129221-21cut10:45:071
169940,0watchdog/117955-21ssh09:00:591
169940,0watchdog/114264-21grep12:05:061
169940,0watchdog/114114-21ssh10:56:421
169940,0watchdog/10-21swapper/111:20:061
169940,0watchdog/10-21swapper/108:30:071
169940,0watchdog/10-21swapper/107:55:071
169940,0watchdog/10-21swapper/107:45:031
149940,0migration/06241-21missed_timers10:25:090
149940,0migration/05723-21kthreadcore12:00:040
149940,0migration/04154-21kthreadcore08:50:100
149940,0migration/025400-21sh09:51:540
149940,0migration/014996-21forks09:25:020
149940,0migration/013239-21kthreadcore09:45:090
149940,0migration/012612-21runrttasks10:07:590
39229932,0cyclictest23713-21awk08:00:052
39189932,0cyclictest3532-21basename11:59:581
39189932,0cyclictest27396-21awk08:25:081
39189932,0cyclictest26905-21awk06:55:131
39189932,0cyclictest24318-21basename11:05:021
39189932,0cyclictest19289-21awk09:50:071
39189930,0cyclictest30349-21perl07:20:021
39189930,0cyclictest19990-21df10:40:001
39179933,0cyclictest21087-21kworker/0:010:44:530
39179933,0cyclictest21087-21kworker/0:010:34:550
39179933,0cyclictest21087-21kworker/0:010:29:530
39179933,0cyclictest21087-21kworker/0:010:14:510
39179933,0cyclictest21087-21kworker/0:009:55:010
39179933,0cyclictest21087-21kworker/0:009:29:530
39179933,0cyclictest21087-21kworker/0:008:09:550
39179933,0cyclictest21087-21kworker/0:008:00:070
39179933,0cyclictest21087-21kworker/0:007:24:550
39179933,0cyclictest16548-21kworker/0:111:09:510
39179933,0cyclictest0-21swapper/008:54:510
179930,0migration/119157-21kthreadcore07:10:091
179930,0migration/117703-21sh10:35:201
169930,0watchdog/115374-21diskmemload11:15:261
169930,0watchdog/10-21swapper/111:35:181
169930,0watchdog/10-21swapper/109:40:031
169930,0watchdog/10-21swapper/109:17:271
169930,0watchdog/10-21swapper/108:55:271
169930,0watchdog/10-21swapper/106:45:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional