You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Sat Jul 27, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2886923927,0sleep30-21swapper/306:44:273
2849923827,0sleep20-21swapper/206:44:222
2941723525,0sleep10-21swapper/106:46:491
2927123026,0sleep00-21swapper/006:45:040
29678991212,0cyclictest20064-21sensors11:04:250
296799986,0cyclictest8598-21diskmemload11:59:091
329970,0watchdog/30-21swapper/311:44:213
329970,0watchdog/30-21swapper/309:59:573
329970,0watchdog/30-21swapper/309:47:133
29925070,0irq/32-eth0-rx-0-21swapper/309:14:203
8315060,0irq/26-0000:00:0-21swapper/308:49:213
339960,0migration/3328-21sh09:00:263
329960,0watchdog/312892-21missed_timers12:14:213
329960,0watchdog/30-21swapper/311:56:013
329960,0watchdog/30-21swapper/311:49:373
329960,0watchdog/30-21swapper/311:41:413
329960,0watchdog/30-21swapper/311:38:013
329960,0watchdog/30-21swapper/311:21:413
329960,0watchdog/30-21swapper/311:09:053
329960,0watchdog/30-21swapper/310:56:053
329960,0watchdog/30-21swapper/310:49:213
329960,0watchdog/30-21swapper/310:46:053
329960,0watchdog/30-21swapper/310:39:373
329960,0watchdog/30-21swapper/310:32:093
329960,0watchdog/30-21swapper/310:24:133
329960,0watchdog/30-21swapper/310:12:493
329960,0watchdog/30-21swapper/309:19:453
329960,0watchdog/30-21swapper/309:09:373
329960,0watchdog/30-21swapper/308:39:373
329960,0watchdog/30-21swapper/308:29:013
329960,0watchdog/30-21swapper/307:54:013
29935060,0irq/33-eth0-tx-24631-21ssh11:28:202
29925060,0irq/32-eth0-rx-6386-21runrttasks10:21:043
29925060,0irq/32-eth0-rx-16212-21munin-node08:09:143
29925060,0irq/32-eth0-rx-14996-21iostat_ios11:04:173
29925060,0irq/32-eth0-rx-0-21swapper/312:09:263
29925060,0irq/32-eth0-rx-0-21swapper/311:32:293
29925060,0irq/32-eth0-rx-0-21swapper/311:16:333
29925060,0irq/32-eth0-rx-0-21swapper/311:01:143
29925060,0irq/32-eth0-rx-0-21swapper/310:15:493
29925060,0irq/32-eth0-rx-0-21swapper/309:49:493
29925060,0irq/32-eth0-rx-0-21swapper/309:42:533
29925060,0irq/32-eth0-rx-0-21swapper/309:25:153
29925060,0irq/32-eth0-rx-0-21swapper/308:55:143
296809960,0cyclictest0-21swapper/211:43:382
296809960,0cyclictest0-21swapper/210:33:472
296789960,0cyclictest0-21swapper/010:43:270
149960,0migration/028734-21sh09:37:150
149960,0migration/025327-21sh08:58:370
4150,0ktimersoftd/00-21swapper/011:29:570
339950,0migration/38352-21kthreadcore08:24:223
339950,0migration/331815-21kthreadcore07:34:193
339950,0migration/32902-21kthreadcore06:49:233
339950,0migration/321796-21kthreadcore08:34:273
339950,0migration/316158-21kthreadcore09:29:203
339950,0migration/315087-21kthreadcore11:59:193
329950,0watchdog/383150irq/26-0000:00:07:14:013
329950,0watchdog/31141-21ssh10:35:053
329950,0watchdog/30-21swapper/311:24:173
329950,0watchdog/30-21swapper/310:05:413
329950,0watchdog/30-21swapper/309:54:213
329950,0watchdog/30-21swapper/309:34:173
329950,0watchdog/30-21swapper/308:45:333
329950,0watchdog/30-21swapper/308:19:013
329950,0watchdog/30-21swapper/308:14:333
329950,0watchdog/30-21swapper/308:04:173
329950,0watchdog/30-21swapper/307:59:053
329950,0watchdog/30-21swapper/307:48:373
329950,0watchdog/30-21swapper/307:29:013
329950,0watchdog/30-21swapper/307:24:013
329950,0watchdog/30-21swapper/307:19:093
329950,0watchdog/30-21swapper/307:04:293
329950,0watchdog/30-21swapper/307:04:293
329950,0watchdog/30-21swapper/306:59:013
329950,0watchdog/30-21swapper/306:54:133
29935050,0irq/33-eth0-tx-8999-21ntp_states10:39:232
29935050,0irq/33-eth0-tx-6315-21iceccd08:39:390
29935050,0irq/33-eth0-tx-6315-21iceccd08:31:430
29935050,0irq/33-eth0-tx-29752-21ssh12:07:462
29935050,0irq/33-eth0-tx-15686-21munin-node07:24:180
29935050,0irq/33-eth0-tx-0-21swapper/212:17:042
29935050,0irq/33-eth0-tx-0-21swapper/211:50:522
29935050,0irq/33-eth0-tx-0-21swapper/210:48:222
29935050,0irq/33-eth0-tx-0-21swapper/007:54:320
29935050,0irq/33-eth0-tx-0-21swapper/007:05:320
29935050,0irq/33-eth0-tx-0-21swapper/007:05:320
29935050,0irq/33-eth0-tx-0-21swapper/006:54:240
29925050,0irq/32-eth0-rx-8498-21timerandwakeup09:04:323
29925050,0irq/32-eth0-rx-0-21swapper/312:04:383
29925050,0irq/32-eth0-rx-0-21swapper/307:49:113
29925050,0irq/32-eth0-rx-0-21swapper/307:09:113
296809955,0cyclictest0-21swapper/211:13:312
296809955,0cyclictest0-21swapper/211:02:572
296809955,0cyclictest0-21swapper/210:06:212
296809955,0cyclictest0-21swapper/206:54:012
296809954,0cyclictest21869-21chrt09:51:322
296809950,0cyclictest0-21swapper/209:15:172
296809950,0cyclictest0-21swapper/208:49:062
296799955,0cyclictest0-21swapper/109:15:591
296799954,0cyclictest6620-21rm11:17:041
296799950,0cyclictest0-21swapper/110:59:431
296799950,0cyclictest0-21swapper/110:44:071
296789955,0cyclictest0-21swapper/010:18:510
296789950,0cyclictest0-21swapper/010:27:130
27150,0ktimersoftd/20-21swapper/210:38:232
259950,0migration/28773-21kthreadcore09:44:172
259950,0migration/27486-21switchtime10:19:272
259950,0migration/225160-21kthreadcore08:14:202
259950,0migration/221587-21kthreadcore11:44:212
259950,0migration/217316-21aten_r1power_en08:54:122
259950,0migration/216420-21latency09:09:202
259950,0migration/211845-21runrttasks08:26:342
259950,0migration/210756-21munin-node06:54:252
249950,0watchdog/227178-21fschecks_time07:09:132
149950,0migration/04741-21latency_hist10:59:000
149950,0migration/04621-21kthreadcore07:59:200
149950,0migration/04571-21switchtime12:09:290
149950,0migration/03640-21munin-node09:39:290
149950,0migration/03469-21kthreadcore07:14:190
149950,0migration/032596-21irqcore11:14:160
149950,0migration/031434-21kernelversion06:49:220
149950,0migration/029562-21timerwakeupswit09:54:290
149950,0migration/028444-21latency11:09:220
149950,0migration/026831-21kthreadcore08:14:210
149950,0migration/026003-21kthreadcore07:29:210
149950,0migration/025721-21munin-node09:14:290
149950,0migration/02407-21taskset11:50:040
149950,0migration/021355-21kthreadcore11:44:210
149950,0migration/020438-21sh12:01:200
149950,0migration/01850-21kthreadcore09:59:190
149950,0migration/015997-21kthreadcore09:29:200
149950,0migration/015794-21taskset11:23:140
149950,0migration/012763-21fschecks_count07:44:140
149950,0migration/012609-21fschecks_time06:59:130
149950,0migration/012461-21kthreadcore08:49:200
149950,0migration/011885-21kthreadcore07:19:220
149950,0migration/010480-21munin-node12:14:190
11778250,0sleep013193-21cut09:09:190
329940,0watchdog/38343-21kthreadcore07:39:213
29935040,0irq/33-eth0-tx-31556-21ssh09:59:082
29935040,0irq/33-eth0-tx-2002-21ssh09:19:342
29935040,0irq/33-eth0-tx-15969-21ssh10:24:292
29935040,0irq/33-eth0-tx-10885-21ssh09:25:112
29935040,0irq/33-eth0-tx-0-21swapper/211:54:032
29935040,0irq/33-eth0-tx-0-21swapper/211:19:082
29935040,0irq/33-eth0-tx-0-21swapper/210:49:092
29935040,0irq/33-eth0-tx-0-21swapper/209:39:172
29935040,0irq/33-eth0-tx-0-21swapper/209:04:162
29935040,0irq/33-eth0-tx-0-21swapper/208:59:212
29935040,0irq/33-eth0-tx-0-21swapper/008:06:470
29935040,0irq/33-eth0-tx-0-21swapper/007:49:260
29935040,0irq/33-eth0-tx-0-21swapper/007:42:000
296809944,0cyclictest0-21swapper/208:44:322
296809943,0cyclictest3599-21lldpd10:14:022
296809943,0cyclictest3599-21lldpd07:40:392
296809943,0cyclictest3599-21lldpd06:59:062
296809943,0cyclictest3262-21irqbalance11:59:052
296809943,0cyclictest3262-21irqbalance11:15:152
296809943,0cyclictest27742-21awk08:39:182
296809943,0cyclictest2695-21awk11:34:192
296809943,0cyclictest2420-21awk07:59:172
296809943,0cyclictest23913-21awk07:29:172
296809943,0cyclictest13449-21awk08:29:162
296799944,0cyclictest0-21swapper/111:40:331
296799944,0cyclictest0-21swapper/109:55:521
296799944,0cyclictest0-21swapper/109:36:571
296799943,0cyclictest9669-21awk08:04:191
296799943,0cyclictest8861-21awk10:04:191
296799943,0cyclictest5459-21awk11:54:181
296799943,0cyclictest3599-21lldpd12:07:411
296799943,0cyclictest3599-21lldpd09:52:571
296799943,0cyclictest3599-21lldpd08:56:501
296799943,0cyclictest27697-21awk07:54:201
296799943,0cyclictest25908-21awk10:14:171
296799943,0cyclictest23871-21snmpd11:33:381
296799943,0cyclictest23871-21snmpd11:23:381
296799943,0cyclictest23871-21snmpd09:20:081
296799943,0cyclictest23871-21snmpd09:09:531
296799943,0cyclictest23871-21snmpd08:51:231
296799943,0cyclictest23871-21snmpd08:46:231
296799943,0cyclictest23871-21snmpd08:39:231
296799943,0cyclictest23871-21snmpd08:26:381
296799943,0cyclictest23871-21snmpd07:48:531
296799943,0cyclictest23871-21snmpd07:38:221
296799943,0cyclictest23871-21snmpd06:53:521
296799943,0cyclictest2244-21awk07:14:161
296799943,0cyclictest20981-21awk10:49:161
296799943,0cyclictest2097-21awk10:19:161
296799943,0cyclictest20425-21awk07:49:171
296799943,0cyclictest19187-21awk10:29:181
296799943,0cyclictest17585-21awk11:24:171
296799943,0cyclictest16902-21awk08:09:181
296799943,0cyclictest16816-21awk07:24:201
296799943,0cyclictest13661-21awk09:29:181
296799943,0cyclictest10725-21awk10:24:171
296789944,0cyclictest0-21swapper/011:27:360
296789943,0cyclictest3599-21lldpd11:34:100
296789943,0cyclictest3599-21lldpd09:19:220
296789943,0cyclictest27512-21awk07:09:160
296789943,0cyclictest23871-21snmpd08:13:530
296789943,0cyclictest22770-21hwlatdetect12:04:180
296789943,0cyclictest21047-21awk10:49:180
296789943,0cyclictest1516-21fschecks_count10:19:130
296789942,0cyclictest3676-21kworker/0:011:00:270
296789940,0cyclictest0-21swapper/009:05:040
259940,0migration/27307-21pmu-power07:14:232
259940,0migration/2356-21kthreadcore08:19:252
259940,0migration/232460-21munin-node07:54:242
259940,0migration/230069-21irqrtprio10:54:172
259940,0migration/22653-21kthreadcore07:34:202
259940,0migration/222242-21if_eth1.309:34:172
259940,0migration/220138-21fschecks_time08:34:212
259940,0migration/219943-21fschecks_count07:49:142
259940,0migration/214377-21kthreadcore07:44:192
259940,0migration/212562-21aten_r1power_po09:29:122
249940,0watchdog/230172-21ssh09:55:412
249940,0watchdog/226417-21iostat11:29:212
249940,0watchdog/220218-21munin-node07:04:172
249940,0watchdog/220218-21munin-node07:04:172
249940,0watchdog/216840-21basename08:09:172
249940,0watchdog/20-21swapper/212:09:212
249940,0watchdog/20-21swapper/211:04:172
249940,0watchdog/20-21swapper/210:09:172
169940,0watchdog/19460-21unixbench-2d10:39:291
169940,0watchdog/18598-21diskmemload11:06:411
169940,0watchdog/18278-21kthreadcore09:44:171
169940,0watchdog/16218-21cut06:54:211
169940,0watchdog/125240-21munin-node07:04:251
169940,0watchdog/125240-21munin-node07:04:251
169940,0watchdog/115457-21kthreadcore06:59:171
169940,0watchdog/113743-21irqrtprio08:29:171
169940,0watchdog/110973-21kthreadcore07:19:211
169940,0watchdog/10-21swapper/112:14:211
169940,0watchdog/10-21swapper/111:49:211
169940,0watchdog/10-21swapper/111:47:531
169940,0watchdog/10-21swapper/111:09:211
169940,0watchdog/10-21swapper/108:14:211
169940,0watchdog/10-21swapper/107:29:211
169940,0watchdog/10-21swapper/107:09:131
149940,0migration/06216-21cpuspeed_turbos09:44:120
149940,0migration/04700-21df_inode11:54:140
149940,0migration/03270-21kthreadcore08:44:210
149940,0migration/030211-21kthreadcore10:34:240
149940,0migration/026434-21fschecks_time08:59:170
149940,0migration/025359-21sh10:30:590
149940,0migration/020548-21nfs4_client09:49:260
149940,0migration/018128-21kthreadcore10:09:200
149940,0migration/010936-21kthreadcore10:04:230
296809932,0cyclictest9757-21awk08:04:202
296809932,0cyclictest15686-21munin-node07:24:212
296799932,0cyclictest7068-21kthreadcore09:04:221
296799932,0cyclictest6386-21runrttasks10:54:531
296799932,0cyclictest3599-21lldpd08:34:091
296799932,0cyclictest32659-21awk09:59:171
296799932,0cyclictest32542-21grep12:09:201
296799932,0cyclictest30813-21awk09:39:191
296799932,0cyclictest2494-21awk07:59:181
296799930,0cyclictest27121-21diskstats10:34:171
296789933,0cyclictest3676-21kworker/0:011:39:170
296789933,0cyclictest3676-21kworker/0:010:44:150
296789933,0cyclictest14533-21kworker/0:209:24:130
296789933,0cyclictest14533-21kworker/0:208:24:050
296789933,0cyclictest14533-21kworker/0:208:19:050
296789933,0cyclictest14533-21kworker/0:207:34:040
296789930,0cyclictest22521-21cat08:34:270
249930,0watchdog/20-21swapper/207:19:212
169930,0watchdog/132565-21kthreadcore08:19:251
169930,0watchdog/10-21swapper/111:34:011
169930,0watchdog/10-21swapper/110:09:171
169930,0watchdog/10-21swapper/109:24:051
169930,0watchdog/10-21swapper/108:59:251
169930,0watchdog/10-21swapper/107:39:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional