You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 14:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Fri Nov 22, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2147723930,0sleep20-21swapper/206:38:002
2159023829,0sleep30-21swapper/306:39:183
2148623727,0sleep10-21swapper/106:38:081
2149022824,0sleep00-21swapper/006:38:110
329970,0watchdog/30-21swapper/311:57:123
329970,0watchdog/30-21swapper/310:07:003
329970,0watchdog/30-21swapper/306:57:083
29925070,0irq/32-eth0-rx-0-21swapper/311:52:283
8315060,0irq/26-0000:00:26575-21awk12:11:512
8315060,0irq/26-0000:00:106202chrt09:51:512
339960,0migration/317281-21sh12:05:213
329960,0watchdog/323135-21ssh11:49:283
329960,0watchdog/30-21swapper/311:32:083
329960,0watchdog/30-21swapper/311:27:243
329960,0watchdog/30-21swapper/311:17:483
329960,0watchdog/30-21swapper/311:13:243
329960,0watchdog/30-21swapper/311:08:043
329960,0watchdog/30-21swapper/311:05:123
329960,0watchdog/30-21swapper/310:54:283
329960,0watchdog/30-21swapper/310:42:163
329960,0watchdog/30-21swapper/310:36:563
329960,0watchdog/30-21swapper/310:33:123
329960,0watchdog/30-21swapper/310:23:163
329960,0watchdog/30-21swapper/310:12:003
329960,0watchdog/30-21swapper/310:02:003
329960,0watchdog/30-21swapper/310:01:083
329960,0watchdog/30-21swapper/309:52:283
329960,0watchdog/30-21swapper/309:47:083
329960,0watchdog/30-21swapper/309:42:283
329960,0watchdog/30-21swapper/309:36:323
329960,0watchdog/30-21swapper/309:12:523
329960,0watchdog/30-21swapper/309:10:363
329960,0watchdog/30-21swapper/307:27:203
329960,0watchdog/30-21swapper/307:07:043
329960,0watchdog/30-21swapper/307:03:523
329960,0watchdog/30-21swapper/306:42:083
29935060,0irq/33-eth0-tx-12825-21munin-node08:32:170
29925060,0irq/32-eth0-rx-27162-21munin-node10:17:173
29925060,0irq/32-eth0-rx-21352-21ssh08:57:063
29925060,0irq/32-eth0-rx-11274-21cut10:47:103
29925060,0irq/32-eth0-rx-0-21swapper/311:42:193
29925060,0irq/32-eth0-rx-0-21swapper/311:37:053
29925060,0irq/32-eth0-rx-0-21swapper/310:59:283
29925060,0irq/32-eth0-rx-0-21swapper/310:27:103
29925060,0irq/32-eth0-rx-0-21swapper/309:38:333
29925060,0irq/32-eth0-rx-0-21swapper/308:48:033
218829960,0cyclictest0-21swapper/210:33:142
218829960,0cyclictest0-21swapper/210:23:182
218809960,0cyclictest0-21swapper/010:23:430
149960,0migration/029921-21kthreadcore11:17:090
149960,0migration/025699-21sh12:09:340
149960,0migration/010778-21sh08:48:570
4150,0ktimersoftd/00-21swapper/011:28:410
339950,0migration/331760-21munin-node08:22:083
339950,0migration/329527-21cpuspeed_turbos06:47:053
339950,0migration/312850-21cpuspeed_turbos07:47:053
329950,0watchdog/39194-21fschecks_count07:22:123
329950,0watchdog/30-21swapper/309:27:043
329950,0watchdog/30-21swapper/309:22:123
329950,0watchdog/30-21swapper/309:17:283
329950,0watchdog/30-21swapper/308:44:203
329950,0watchdog/30-21swapper/308:37:123
329950,0watchdog/30-21swapper/308:34:243
329950,0watchdog/30-21swapper/308:27:403
329950,0watchdog/30-21swapper/308:19:563
329950,0watchdog/30-21swapper/308:11:563
329950,0watchdog/30-21swapper/308:06:523
329950,0watchdog/30-21swapper/308:03:243
329950,0watchdog/30-21swapper/307:56:563
329950,0watchdog/30-21swapper/307:52:483
329950,0watchdog/30-21swapper/307:42:003
329950,0watchdog/30-21swapper/307:41:243
329950,0watchdog/30-21swapper/307:32:003
329950,0watchdog/30-21swapper/307:17:483
329950,0watchdog/30-21swapper/307:13:123
329950,0watchdog/30-21swapper/306:55:083
329950,0watchdog/30-21swapper/306:55:083
29935050,0irq/33-eth0-tx-9124-21munin-node08:07:170
29935050,0irq/33-eth0-tx-8811-21telnet07:22:110
29935050,0irq/33-eth0-tx-5598-21proc_pri10:42:182
29935050,0irq/33-eth0-tx-5255-21awk11:38:152
29935050,0irq/33-eth0-tx-31568-21ssh09:23:092
29935050,0irq/33-eth0-tx-30562-21ssh10:39:342
29935050,0irq/33-eth0-tx-29290-21munin-node06:47:100
29935050,0irq/33-eth0-tx-22435-21grep11:12:102
29935050,0irq/33-eth0-tx-20681-21munin-node10:17:132
29935050,0irq/33-eth0-tx-19995-21ssh08:55:322
29935050,0irq/33-eth0-tx-0-21swapper/211:03:302
29935050,0irq/33-eth0-tx-0-21swapper/210:58:492
29935050,0irq/33-eth0-tx-0-21swapper/210:48:592
29935050,0irq/33-eth0-tx-0-21swapper/210:09:212
29935050,0irq/33-eth0-tx-0-21swapper/209:42:172
29935050,0irq/33-eth0-tx-0-21swapper/008:42:140
29935050,0irq/33-eth0-tx-0-21swapper/008:23:240
29935050,0irq/33-eth0-tx-0-21swapper/008:12:030
29935050,0irq/33-eth0-tx-0-21swapper/007:07:120
29925050,0irq/32-eth0-rx-13202-21if_eth108:52:093
29925050,0irq/32-eth0-rx-0-21swapper/312:07:013
29925050,0irq/32-eth0-rx-0-21swapper/311:22:213
29925050,0irq/32-eth0-rx-0-21swapper/309:02:123
27150,0ktimersoftd/20-21swapper/208:49:342
259950,0migration/25413-21latency_hist10:26:512
259950,0migration/231175-21kthreadcore06:47:092
259950,0migration/227369-21kthreadcore11:32:092
259950,0migration/21786-21sh09:27:062
259950,0migration/217467-21kernelversion07:27:152
259950,0migration/214758-21kthreadcore11:27:072
259950,0migration/210798-21switchtime11:22:192
259950,0migration/210253-21kthreadcore11:42:142
249950,0watchdog/223186-21ssh11:49:362
218829955,0cyclictest0-21swapper/211:54:542
218829955,0cyclictest0-21swapper/209:17:522
218829955,0cyclictest0-21swapper/208:26:472
218829950,0cyclictest0-21swapper/209:34:442
218829950,0cyclictest0-21swapper/208:56:542
218819955,0cyclictest0-21swapper/108:37:161
218819955,0cyclictest0-21swapper/108:26:521
218819950,0cyclictest0-21swapper/111:41:251
218819950,0cyclictest0-21swapper/111:23:171
218819950,0cyclictest0-21swapper/111:09:341
218819950,0cyclictest0-21swapper/108:32:041
218809955,0cyclictest0-21swapper/011:58:000
218809955,0cyclictest0-21swapper/011:23:470
218809950,0cyclictest0-21swapper/007:57:060
19150,0ktimersoftd/10-21swapper/111:32:441
19150,0ktimersoftd/10-21swapper/111:28:411
179950,0migration/112580-21aten2.4-expect07:47:041
169950,0watchdog/10-21swapper/107:22:161
149950,0migration/07408-21kthreadcore11:02:120
149950,0migration/05589-21latency_hist08:26:510
149950,0migration/031985-21kthreadcore07:37:170
149950,0migration/031294-21sh11:52:550
149950,0migration/031156-21fschecks_count11:37:040
149950,0migration/030874-21kthreadcore10:57:110
149950,0migration/030780-21irqcore09:02:110
149950,0migration/029305-21kthreadcore10:02:140
149950,0migration/02600-21kthreadcore07:12:090
149950,0migration/025542-21kthreadcore10:17:090
149950,0migration/025110-21kernelversion08:17:100
149950,0migration/024743-21kthreadcore10:37:090
149950,0migration/023627-21munin-node06:42:090
149950,0migration/022533-21fschecks_time11:32:030
149950,0migration/020611-21kthreadcore07:02:100
149950,0migration/019516-21munin-node09:57:050
149950,0migration/018316-21fschecks_time09:37:040
149950,0migration/016960-21kthreadcore10:12:170
149950,0migration/016459-21kthreadcore08:52:110
149950,0migration/016308-21kthreadcore11:07:090
149950,0migration/016138-21cpuspeed_turbos07:27:100
149950,0migration/014114-21sh09:11:530
149950,0migration/011265-21needreboot07:42:110
149950,0migration/011117-21kthreadcore10:27:110
149950,0migration/010414-21irqcore09:32:090
8315040,0irq/26-0000:00:0-21swapper/206:56:522
29935040,0irq/33-eth0-tx-23912-21sensors09:37:132
29935040,0irq/33-eth0-tx-1785-21munin-node08:02:120
29935040,0irq/33-eth0-tx-0-21swapper/211:56:582
29935040,0irq/33-eth0-tx-0-21swapper/211:17:062
29935040,0irq/33-eth0-tx-0-21swapper/209:57:052
29935040,0irq/33-eth0-tx-0-21swapper/209:47:502
29935040,0irq/33-eth0-tx-0-21swapper/209:12:082
29935040,0irq/33-eth0-tx-0-21swapper/209:02:162
259940,0migration/29603-21aten_r1power_po12:02:072
259940,0migration/27231-21kthreadcore07:42:092
259940,0migration/27111-21irqrtprio09:07:092
259940,0migration/25777-21kthreadcore06:52:082
259940,0migration/25777-21kthreadcore06:52:082
259940,0migration/25552-21kthreadcore07:12:102
259940,0migration/23627-21kthreadcore08:02:102
259940,0migration/229922-21kthreadcore08:42:112
259940,0migration/227015-21kthreadcore06:42:102
259940,0migration/223591-21aten_r1power_en08:17:042
259940,0migration/223116-21runrttasks07:32:012
259940,0migration/220480-21munin-node08:37:072
259940,0migration/212758-21taskset10:12:112
249940,0watchdog/29095-21fschecks_count07:22:122
249940,0watchdog/2135842sleep207:47:082
249940,0watchdog/210996-21entropy11:07:042
249940,0watchdog/20-21swapper/208:15:322
218829943,0cyclictest8412-21sadc07:21:522
218829943,0cyclictest6697-21awk08:27:072
218829943,0cyclictest5035-21sadc07:41:512
218829943,0cyclictest28179-21awk07:57:092
218829943,0cyclictest23871-21snmpd10:52:182
218829943,0cyclictest23871-21snmpd10:02:482
218829943,0cyclictest13889-21awk08:32:072
218829940,0cyclictest0-21swapper/207:10:552
218819944,0cyclictest0-21swapper/109:18:531
218819944,0cyclictest0-21swapper/107:42:051
218819943,0cyclictest3599-21lldpd11:50:051
218819943,0cyclictest3599-21lldpd11:45:331
218819943,0cyclictest3599-21lldpd10:52:031
218819943,0cyclictest3599-21lldpd10:42:331
218819943,0cyclictest3599-21lldpd10:28:331
218819943,0cyclictest3599-21lldpd09:33:411
218819943,0cyclictest3599-21lldpd09:15:001
218819943,0cyclictest3599-21lldpd08:57:531
218819943,0cyclictest3599-21lldpd07:27:001
218819943,0cyclictest31806-21awk08:22:081
218819943,0cyclictest28986-21grep10:02:131
218819943,0cyclictest26919-21awk07:07:071
218819943,0cyclictest25465-21awk11:52:081
218819943,0cyclictest24155-21awk07:32:071
218819943,0cyclictest23871-21snmpd10:13:181
218819943,0cyclictest23871-21snmpd10:00:331
218819943,0cyclictest23871-21snmpd07:19:171
218819943,0cyclictest20924-21awk07:52:091
218819943,0cyclictest15279-21awk10:32:061
218819940,0cyclictest0-21swapper/108:10:551
218809944,0cyclictest0-21swapper/010:47:510
218809944,0cyclictest0-21swapper/007:17:030
218809943,0cyclictest8119-21awk11:42:120
218809943,0cyclictest24025-21sadc11:51:510
218809943,0cyclictest20337-21awk11:12:070
179940,0migration/14507-21gltestperf10:07:031
179940,0migration/13778-21taskset06:50:051
179940,0migration/132316-21kthreadcore10:22:141
179940,0migration/130930-21smart_sdb09:22:151
179940,0migration/13039-21kthreadcore07:12:101
179940,0migration/130245-21taskset08:19:071
179940,0migration/128538-21kthreadcore10:37:101
179940,0migration/123216-21if_eth106:42:081
179940,0migration/12185-21irqrtprio11:57:121
179940,0migration/12068-21taskset09:47:001
179940,0migration/119693-21munin-node07:02:091
179940,0migration/113298-21munin-node08:52:101
179940,0migration/112231-21unixbench_singl09:07:191
169940,0watchdog/14188-21fschecks_time08:47:041
169940,0watchdog/127328-21iostat09:42:161
169940,0watchdog/121985-21irqcore10:17:081
169940,0watchdog/10-21swapper/112:07:361
169940,0watchdog/10-21swapper/111:02:121
169940,0watchdog/10-21swapper/110:57:041
169940,0watchdog/10-21swapper/110:50:361
169940,0watchdog/10-21swapper/109:29:201
169940,0watchdog/10-21swapper/109:02:121
149940,0migration/07977-21kthreadcore09:47:110
149940,0migration/07275-21kthreadcore10:07:070
149940,0migration/05997-21kthreadcore06:52:080
149940,0migration/05997-21kthreadcore06:52:080
149940,0migration/032180-21sshd09:24:590
149940,0migration/027941-21kthreadcore09:42:170
149940,0migration/016553-21latency09:52:110
8315030,0irq/26-0000:00:0-21swapper/207:51:512
29935030,0irq/33-eth0-tx-0-21swapper/008:37:050
29935030,0irq/33-eth0-tx-0-21swapper/007:47:060
29935030,0irq/33-eth0-tx-0-21swapper/006:57:030
259930,0migration/222119-21kthreadcore07:02:112
249930,0watchdog/20-21swapper/208:07:042
218819932,0cyclictest6438-21kthreadcore08:02:111
218819932,0cyclictest3599-21lldpd11:17:031
218819932,0cyclictest3599-21lldpd07:37:091
218819932,0cyclictest29961-21kthreadcore07:57:111
218819932,0cyclictest28370-21awk08:42:091
218819932,0cyclictest18657-21awk09:37:061
218819932,0cyclictest17375-21awk08:12:071
218819932,0cyclictest11514-21basename06:57:031
218819932,0cyclictest10673-21awk12:02:121
218819930,0cyclictest24799-21kthreadcore11:12:111
218809933,0cyclictest30729-21kworker/0:012:02:140
218809933,0cyclictest27057-21kworker/0:210:52:140
218809933,0cyclictest13893-21kworker/0:109:27:080
218809933,0cyclictest13893-21kworker/0:108:57:240
218809932,0cyclictest31726-21munin-node10:42:220
218809932,0cyclictest23119-21munin-node07:32:040
218809932,0cyclictest19716-21cut07:51:510
218809932,0cyclictest15351-21awk10:32:060
218809932,0cyclictest13893-21kworker/0:109:17:180
218809930,0cyclictest9050-21grep09:07:100
169930,0watchdog/126992-21diskmemload09:52:121
169930,0watchdog/10-21swapper/106:52:081
169930,0watchdog/10-21swapper/106:52:081
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional