You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-23 - 17:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Sat Nov 23, 2024 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2457724434,0sleep30-21swapper/306:41:243
2439023826,0sleep00-21swapper/006:39:130
2427123727,0sleep10-21swapper/106:37:461
2429223626,0sleep20-21swapper/206:38:032
246849999,0cyclictest22364-21sensors09:47:131
8315070,0irq/26-0000:00:0-21swapper/307:01:483
329970,0watchdog/322931-21grep07:22:053
29925070,0irq/32-eth0-rx-0-21swapper/311:08:433
29925070,0irq/32-eth0-rx-0-21swapper/310:55:163
339960,0migration/38712-21munin-node09:42:073
339960,0migration/35580-21sh11:56:003
339960,0migration/324013-21sh10:07:263
339960,0migration/313574-21kthreadcore10:42:073
339960,0migration/312026-21irqcore08:02:063
339960,0migration/311418-21fschecks_time07:17:043
329960,0watchdog/39982-21sed07:57:133
329960,0watchdog/330203-21irqrtprio07:52:053
329960,0watchdog/327005-21sed11:47:083
329960,0watchdog/30-21swapper/312:08:043
329960,0watchdog/30-21swapper/311:58:163
329960,0watchdog/30-21swapper/311:44:483
329960,0watchdog/30-21swapper/311:14:123
329960,0watchdog/30-21swapper/310:34:043
329960,0watchdog/30-21swapper/310:22:283
329960,0watchdog/30-21swapper/310:20:043
329960,0watchdog/30-21swapper/310:03:003
329960,0watchdog/30-21swapper/309:31:483
329960,0watchdog/30-21swapper/309:30:283
329960,0watchdog/30-21swapper/309:21:483
329960,0watchdog/30-21swapper/309:17:003
329960,0watchdog/30-21swapper/309:12:003
329960,0watchdog/30-21swapper/308:58:203
329960,0watchdog/30-21swapper/308:46:483
329960,0watchdog/30-21swapper/308:45:083
329960,0watchdog/30-21swapper/308:13:013
329960,0watchdog/30-21swapper/307:27:133
329960,0watchdog/30-21swapper/306:52:093
29935060,0irq/33-eth0-tx-6962-21munin-node06:52:090
29925060,0irq/32-eth0-rx-0-21swapper/311:32:573
29925060,0irq/32-eth0-rx-0-21swapper/311:24:293
29925060,0irq/32-eth0-rx-0-21swapper/311:20:383
29925060,0irq/32-eth0-rx-0-21swapper/310:57:303
29925060,0irq/32-eth0-rx-0-21swapper/310:39:463
29925060,0irq/32-eth0-rx-0-21swapper/309:57:173
29925060,0irq/32-eth0-rx-0-21swapper/309:46:503
29925060,0irq/32-eth0-rx-0-21swapper/309:37:073
29925060,0irq/32-eth0-rx-0-21swapper/308:53:523
246839960,0cyclictest0-21swapper/011:24:160
246839960,0cyclictest0-21swapper/008:17:070
246839960,0cyclictest0-21swapper/007:12:010
149960,0migration/025699-21sh11:47:080
149960,0migration/021820-21sh09:11:290
8315050,0irq/26-0000:00:9411-21cat11:02:022
8315050,0irq/26-0000:00:5445-21open_files09:37:102
8315050,0irq/26-0000:00:0-21swapper/308:27:023
339950,0migration/36121-21munin-node07:32:123
339950,0migration/36042-21switchtime06:47:153
339950,0migration/331785-21kthreadcore07:07:073
339950,0migration/321808-21cpuspeed_turbos07:47:023
329950,0watchdog/39456-21ntp_states07:12:093
329950,0watchdog/330102-21cut09:52:083
329950,0watchdog/30-21swapper/311:36:563
329950,0watchdog/30-21swapper/311:03:363
329950,0watchdog/30-21swapper/310:11:523
329950,0watchdog/30-21swapper/309:02:083
329950,0watchdog/30-21swapper/308:38:483
329950,0watchdog/30-21swapper/308:32:043
329950,0watchdog/30-21swapper/308:22:213
329950,0watchdog/30-21swapper/308:17:053
329950,0watchdog/30-21swapper/308:06:493
329950,0watchdog/30-21swapper/307:43:133
329950,0watchdog/30-21swapper/307:38:173
329950,0watchdog/30-21swapper/306:57:013
329950,0watchdog/30-21swapper/306:42:053
29935050,0irq/33-eth0-tx-7298-21munin-node08:22:110
29935050,0irq/33-eth0-tx-5453-21chrt10:57:202
29935050,0irq/33-eth0-tx-30928-21ssh12:08:542
29935050,0irq/33-eth0-tx-30538-21munin-node09:17:032
29935050,0irq/33-eth0-tx-26392-21grep10:12:022
29935050,0irq/33-eth0-tx-14498-21munin-node08:27:040
29935050,0irq/33-eth0-tx-12380-21munin-node11:42:042
29935050,0irq/33-eth0-tx-0-21swapper/211:37:042
29935050,0irq/33-eth0-tx-0-21swapper/211:10:132
29935050,0irq/33-eth0-tx-0-21swapper/210:23:592
29935050,0irq/33-eth0-tx-0-21swapper/210:04:032
29935050,0irq/33-eth0-tx-0-21swapper/208:50:192
29935050,0irq/33-eth0-tx-0-21swapper/006:49:290
29925050,0irq/32-eth0-rx-0-21swapper/312:02:363
29925050,0irq/32-eth0-rx-0-21swapper/311:27:003
29925050,0irq/32-eth0-rx-0-21swapper/310:47:023
29925050,0irq/32-eth0-rx-0-21swapper/310:26:593
29925050,0irq/32-eth0-rx-0-21swapper/309:07:383
27150,0ktimersoftd/20-21swapper/209:32:202
259950,0migration/27168-21hddtemp_smartct11:57:022
259950,0migration/24968-21kthreadcore10:17:062
259950,0migration/231005-21munin-node11:12:072
259950,0migration/230140-21sh09:16:202
259950,0migration/229861-21kthreadcore06:42:072
259950,0migration/226027-21hwlatdetect07:27:022
259950,0migration/221588-21gltestperf08:52:042
259950,0migration/216132-21kthreadcore07:17:092
259950,0migration/214617-21fschecks_count06:57:012
259950,0migration/214376-21latency_hist08:26:482
246859954,0cyclictest23871-21snmpd08:41:272
246859950,0cyclictest0-21swapper/211:35:282
246849955,0cyclictest0-21swapper/110:02:191
246849955,0cyclictest0-21swapper/108:54:321
246849950,0cyclictest0-21swapper/111:50:381
246849950,0cyclictest0-21swapper/110:15:151
246849950,0cyclictest0-21swapper/110:08:281
246849950,0cyclictest0-21swapper/109:26:411
246849950,0cyclictest0-21swapper/109:10:461
246839955,0cyclictest0-21swapper/011:37:210
246839955,0cyclictest0-21swapper/011:13:040
246839955,0cyclictest0-21swapper/011:03:460
246839954,0cyclictest11125-21awk10:22:070
246839950,0cyclictest0-21swapper/010:15:220
246839950,0cyclictest0-21swapper/008:41:520
179950,0migration/18975-21kthreadcore09:02:131
179950,0migration/18477-21sh10:18:041
179950,0migration/16910-21switchtime09:57:141
179950,0migration/14106-21kthreadcore11:17:071
179950,0migration/131617-21kthreadcore10:52:061
179950,0migration/126117-21sh11:29:221
179950,0migration/118141-21kthreadcore11:07:051
179950,0migration/112255-21kthreadcore11:02:031
179950,0migration/111970-21latency_hist08:46:481
169950,0watchdog/128894-21vmstat09:12:161
169950,0watchdog/126232-21irqcore06:42:051
149950,0migration/07722-21aten_r1power_en09:42:030
149950,0migration/04764-21unixbench-2d09:17:140
149950,0migration/04323-21kthreadcore07:32:070
149950,0migration/03370-21kthreadcore11:17:060
149950,0migration/032198-21fschecks_count09:37:020
149950,0migration/030820-21switchtime09:52:150
149950,0migration/030571-21fschecks_time11:52:020
149950,0migration/029745-21fschecks_time08:37:010
149950,0migration/02954-21taskset07:52:350
149950,0migration/027745-21timerwakeupswit07:02:180
149950,0migration/021168-21latency_hist07:01:480
149950,0migration/020737-21kthreadcore11:27:070
149950,0migration/020319-21kthreadcore11:07:060
149950,0migration/018159-21aten_r1power_vo08:06:590
149950,0migration/017996-21kthreadcore09:47:080
149950,0migration/017771-21latency_hist07:21:480
149950,0migration/016921-21kthreadcore09:27:060
149950,0migration/016441-21kthreadcore10:42:080
149950,0migration/015350-21munin-node12:02:040
149950,0migration/012185-21missed_timers11:57:070
149950,0migration/011184-21needreboot09:02:150
29935040,0irq/33-eth0-tx-6052-21kthreadcore11:17:072
29935040,0irq/33-eth0-tx-3544-21diskmemload10:31:572
29935040,0irq/33-eth0-tx-3238-21ssh10:36:502
29935040,0irq/33-eth0-tx-27949-21cat09:52:082
29935040,0irq/33-eth0-tx-18509-21ssh10:26:532
29935040,0irq/33-eth0-tx-11764-21ssh08:46:232
29935040,0irq/33-eth0-tx-0-21swapper/212:01:582
29935040,0irq/33-eth0-tx-0-21swapper/211:51:522
29935040,0irq/33-eth0-tx-0-21swapper/211:27:382
29935040,0irq/33-eth0-tx-0-21swapper/210:42:092
29935040,0irq/33-eth0-tx-0-21swapper/210:07:012
29935040,0irq/33-eth0-tx-0-21swapper/209:57:012
29935040,0irq/33-eth0-tx-0-21swapper/209:47:072
29935040,0irq/33-eth0-tx-0-21swapper/209:27:042
29935040,0irq/33-eth0-tx-0-21swapper/209:21:542
29935040,0irq/33-eth0-tx-0-21swapper/208:56:582
29935040,0irq/33-eth0-tx-0-21swapper/008:02:090
29935040,0irq/33-eth0-tx-0-21swapper/007:44:380
29935040,0irq/33-eth0-tx-0-21swapper/007:29:360
259940,0migration/26102-21kthreadcore07:57:052
259940,0migration/232532-21fschecks_count06:47:032
259940,0migration/226445-21kthreadcore08:32:082
259940,0migration/225462-21kthreadcore07:47:082
259940,0migration/222815-21kthreadcore10:47:062
259940,0migration/215024-21fschecks_time07:42:042
259940,0migration/213786-21aten_r1power_vo09:06:592
246859944,0cyclictest0-21swapper/208:12:052
246859943,0cyclictest29932-21awk07:52:042
246859943,0cyclictest29923-21grep10:52:022
246859943,0cyclictest23871-21snmpd11:46:582
246859943,0cyclictest23871-21snmpd08:29:422
246859943,0cyclictest23871-21snmpd08:04:422
246859943,0cyclictest23871-21snmpd07:41:412
246859943,0cyclictest19164-21awk08:07:032
246859943,0cyclictest18933-21awk07:22:022
246849943,0cyclictest8428-21awk08:22:061
246849943,0cyclictest3599-21lldpd08:12:271
246849943,0cyclictest3599-21lldpd06:55:421
246849943,0cyclictest23871-21snmpd07:41:111
246849943,0cyclictest11787-21awk07:17:061
246839944,0cyclictest0-21swapper/010:56:590
246839943,0cyclictest3599-21lldpd10:09:040
246839943,0cyclictest23871-21snmpd10:30:580
246839943,0cyclictest21799-21awk08:52:060
246839943,0cyclictest21627-21awk10:47:040
246839943,0cyclictest11054-21df07:17:030
246839940,0cyclictest0-21swapper/008:16:300
249940,0watchdog/211174-21if_eth1.311:22:042
249940,0watchdog/20-21swapper/209:42:082
249940,0watchdog/20-21swapper/209:02:202
179940,0migration/16418-21cpuspeed_turbos11:57:001
179940,0migration/15169-21kthreadcore07:32:081
179940,0migration/14596-21kthreadcore08:17:071
179940,0migration/13244-21sh11:36:331
179940,0migration/130921-21kthreadcore07:07:061
179940,0migration/130853-21kthreadcore08:37:061
179940,0migration/127321-21kthreadcore09:52:071
179940,0migration/126989-21kthreadcore08:32:081
179940,0migration/124909-21aten_r1power_po11:11:581
179940,0migration/119831-21kthreadcore09:27:071
179940,0migration/117696-21kthreadcore07:42:081
179940,0migration/11558-21taskset10:32:471
179940,0migration/115061-21fschecks_count08:27:011
179940,0migration/113883-21kthreadcore08:47:061
179940,0migration/112859-21gltestperf10:42:041
179940,0migration/110748-21fschecks_time10:22:051
169940,0watchdog/16433-21ssh09:38:481
169940,0watchdog/14201-21ssh11:52:161
169940,0watchdog/13745-21diskstats10:37:001
169940,0watchdog/126722-21cut12:07:081
169940,0watchdog/121707-21iwlist10:47:041
169940,0watchdog/121074-21open_inodes12:02:121
169940,0watchdog/114772-21tune2fs06:57:011
169940,0watchdog/112330-21ssh11:41:521
169940,0watchdog/111175-21if_eth1.311:22:041
169940,0watchdog/110790-21rm11:38:201
169940,0watchdog/10-21swapper/110:59:241
169940,0watchdog/10-21swapper/108:07:131
169940,0watchdog/10-21swapper/107:57:131
169940,0watchdog/10-21swapper/107:47:091
169940,0watchdog/10-21swapper/107:27:051
149940,0migration/08790-21latency_hist10:01:480
149940,0migration/031917-21kthreadcore10:52:060
149940,0migration/030054-21runrttasks09:16:020
149940,0migration/024145-21fschecks_time09:32:030
149940,0migration/019978-21runrttasks08:50:420
149940,0migration/019746-21sh11:43:530
29935030,0irq/33-eth0-tx-0-21swapper/006:42:000
246859933,0cyclictest4475-21awk07:12:042
246859933,0cyclictest22430-21awk07:02:072
246859933,0cyclictest0-21swapper/207:06:492
246859932,0cyclictest7063-21sadc08:21:482
246859930,0cyclictest8546-21kthreadcore06:52:062
246859930,0cyclictest0-21swapper/207:32:022
246849932,0cyclictest7921-21cut07:12:061
246849932,0cyclictest7740-21aten2.4-expect09:42:031
246849932,0cyclictest675-21awk06:47:051
246849932,0cyclictest3599-21lldpd07:06:291
246849932,0cyclictest30159-21awk08:57:031
246849932,0cyclictest19006-21awk07:22:021
246839933,0cyclictest2366-21kworker/0:107:36:590
246839933,0cyclictest13523-21kworker/0:110:32:060
246839933,0cyclictest13523-21kworker/0:110:16:480
246839933,0cyclictest13523-21kworker/0:109:22:290
246839933,0cyclictest13523-21kworker/0:108:56:510
246839933,0cyclictest13523-21kworker/0:108:32:450
246839933,0cyclictest13348-21kworker/0:007:48:450
246839933,0cyclictest12937-21kworker/0:211:32:000
246839932,0cyclictest4754-21awk07:57:030
246839932,0cyclictest4382-21awk10:37:020
246839932,0cyclictest28998-21tr12:07:090
246839930,0cyclictest6720-21port_sshpriv09:57:110
246839930,0cyclictest0-21swapper/007:07:140
179930,0migration/125445-21diskmemload10:27:491
179930,0migration/117732-21runrttasks08:06:221
169930,0watchdog/13544-21diskmemload09:31:481
169930,0watchdog/10-21swapper/109:16:481
169930,0watchdog/10-21swapper/107:52:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional