You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-03 - 01:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Jan 02, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
410825041,0sleep20-21swapper/206:37:322
2968423726,0sleep30-21swapper/306:34:323
421823627,0sleep10-21swapper/106:38:511
396623524,0sleep00-21swapper/006:35:530
329970,0watchdog/321846-21kthreadcore06:49:423
29925070,0irq/32-eth0-rx-0-21swapper/311:54:103
29925070,0irq/32-eth0-rx-0-21swapper/310:57:063
43469960,0cyclictest28665-21diskmemload11:50:162
43469960,0cyclictest0-21swapper/212:01:082
43469960,0cyclictest0-21swapper/210:59:012
43469960,0cyclictest0-21swapper/209:58:382
43419960,0cyclictest0-21swapper/011:26:000
43419960,0cyclictest0-21swapper/011:26:000
339960,0migration/328261-21sh10:34:543
339960,0migration/319237-21sh10:29:033
329960,0watchdog/322925-21cut11:39:413
329960,0watchdog/30-21swapper/312:06:333
329960,0watchdog/30-21swapper/311:55:013
329960,0watchdog/30-21swapper/311:34:373
329960,0watchdog/30-21swapper/311:29:413
329960,0watchdog/30-21swapper/311:03:543
329960,0watchdog/30-21swapper/310:42:423
329960,0watchdog/30-21swapper/310:31:423
329960,0watchdog/30-21swapper/310:17:463
329960,0watchdog/30-21swapper/310:11:063
329960,0watchdog/30-21swapper/309:59:303
329960,0watchdog/30-21swapper/309:49:423
329960,0watchdog/30-21swapper/309:44:383
329960,0watchdog/30-21swapper/309:25:343
329960,0watchdog/30-21swapper/309:15:303
329960,0watchdog/30-21swapper/309:00:303
329960,0watchdog/30-21swapper/308:55:063
329960,0watchdog/30-21swapper/308:47:063
329960,0watchdog/30-21swapper/308:34:263
329960,0watchdog/30-21swapper/308:12:503
329960,0watchdog/30-21swapper/308:08:303
329960,0watchdog/30-21swapper/307:29:223
329960,0watchdog/30-21swapper/307:19:383
329960,0watchdog/30-21swapper/307:14:263
29935060,0irq/33-eth0-tx-24818-21munin-node07:29:500
29925060,0irq/32-eth0-rx-6384-21cat10:44:393
29925060,0irq/32-eth0-rx-4885-21kthreadcore11:04:443
29925060,0irq/32-eth0-rx-3968-21if_eth1.309:29:423
29925060,0irq/32-eth0-rx-21524-21ssh10:49:543
29925060,0irq/32-eth0-rx-0-21swapper/311:59:343
29925060,0irq/32-eth0-rx-0-21swapper/311:44:573
29925060,0irq/32-eth0-rx-0-21swapper/311:25:343
29925060,0irq/32-eth0-rx-0-21swapper/311:25:343
29925060,0irq/32-eth0-rx-0-21swapper/311:15:443
29925060,0irq/32-eth0-rx-0-21swapper/309:57:443
29925060,0irq/32-eth0-rx-0-21swapper/309:09:443
29925060,0irq/32-eth0-rx-0-21swapper/309:05:323
149960,0migration/04843-21sh10:41:160
149960,0migration/027168-21sh10:32:140
149960,0migration/022875-21sh10:53:290
149960,0migration/021961-21sh12:03:470
149960,0migration/01776-21sh10:17:540
149960,0migration/011829-21sh08:45:260
8315050,0irq/26-0000:00:405-21ls10:14:252
43469955,0cyclictest0-21swapper/211:12:162
43469955,0cyclictest0-21swapper/210:25:172
43469955,0cyclictest0-21swapper/209:23:102
43469950,0cyclictest0-21swapper/212:08:232
43469950,0cyclictest0-21swapper/211:57:312
43469950,0cyclictest0-21swapper/211:29:082
43469950,0cyclictest0-21swapper/211:29:082
43469950,0cyclictest0-21swapper/211:05:392
43469950,0cyclictest0-21swapper/209:10:522
43429955,0cyclictest23871-21snmpd11:47:561
43429950,0cyclictest0-21swapper/111:59:361
43419950,0cyclictest0-21swapper/009:49:210
339950,0migration/38409-21kthreadcore06:39:463
339950,0migration/33193-21kthreadcore08:19:443
339950,0migration/32582-21timerwakeupswit07:54:543
339950,0migration/32580-21aten_r1power_cu10:19:323
339950,0migration/31910-21kthreadcore07:34:433
339950,0migration/31586-21latency_hist07:04:253
329950,0watchdog/3299250irq/32-eth0-rx-10:05:543
329950,0watchdog/30-21swapper/309:39:303
329950,0watchdog/30-21swapper/309:34:463
329950,0watchdog/30-21swapper/308:29:383
329950,0watchdog/30-21swapper/308:25:103
329950,0watchdog/30-21swapper/308:14:463
329950,0watchdog/30-21swapper/307:59:423
329950,0watchdog/30-21swapper/307:53:343
329950,0watchdog/30-21swapper/307:44:303
329950,0watchdog/30-21swapper/307:39:463
329950,0watchdog/30-21swapper/307:29:343
329950,0watchdog/30-21swapper/307:12:023
329950,0watchdog/30-21swapper/307:00:503
329950,0watchdog/30-21swapper/306:57:223
329950,0watchdog/30-21swapper/306:44:343
29935050,0irq/33-eth0-tx-28896-21ssh08:39:290
29935050,0irq/33-eth0-tx-22893-21munin-node11:19:382
29935050,0irq/33-eth0-tx-19359-21kthreadcore10:49:452
29935050,0irq/33-eth0-tx-0-21swapper/210:45:032
29935050,0irq/33-eth0-tx-0-21swapper/209:41:422
29935050,0irq/33-eth0-tx-0-21swapper/209:04:442
29935050,0irq/33-eth0-tx-0-21swapper/008:34:400
29935050,0irq/33-eth0-tx-0-21swapper/008:11:270
29925050,0irq/32-eth0-rx-0-21swapper/311:19:403
29925050,0irq/32-eth0-rx-0-21swapper/311:09:483
29925050,0irq/32-eth0-rx-0-21swapper/309:19:383
29925050,0irq/32-eth0-rx-0-21swapper/308:49:333
29925050,0irq/32-eth0-rx-0-21swapper/308:39:363
259950,0migration/28450-21fschecks_time09:59:362
259950,0migration/24830-21sh09:52:162
259950,0migration/226595-21aten_r1power_vo11:44:372
259950,0migration/223024-21kthreadcore09:44:442
259950,0migration/219693-21fschecks_time06:49:382
259950,0migration/218849-21kthreadcore07:19:392
249950,0watchdog/227459-21kthreadcore10:59:382
249950,0watchdog/220878-21ssh11:18:052
249950,0watchdog/216961-21sh09:15:302
179950,0migration/17741-21ntp_offset07:04:451
179950,0migration/17355-21if_eth1.310:44:421
179950,0migration/15215-21kthreadcore11:04:441
179950,0migration/129303-21sh11:22:201
179950,0migration/128761-21kthreadcore07:29:441
179950,0migration/116879-21munin-node08:04:471
179950,0migration/113036-21munin-node07:39:471
179950,0migration/112087-21sh09:37:311
179950,0migration/111393-21timerandwakeup08:44:511
149950,0migration/0951-21unixbench_singl06:59:460
149950,0migration/09303-21kthreadcore11:29:430
149950,0migration/08538-21cpuspeed_turbos09:09:380
149950,0migration/07935-21kthreadcore06:39:460
149950,0migration/0747-21fschecks_time08:19:400
149950,0migration/05405-21kthreadcore11:04:440
149950,0migration/04598-21kthreadcore11:49:410
149950,0migration/04208-21fschecks_time07:59:380
149950,0migration/031069-21needreboot08:14:460
149950,0migration/029110-21sh11:21:490
149950,0migration/025213-21fschecks_count10:09:380
149950,0migration/024521-21sh11:40:030
149950,0migration/023824-21aten_r1power_fr08:59:340
149950,0migration/023483-21sshd10:54:300
149950,0migration/023137-21kthreadcore07:49:420
149950,0migration/023040-21switchtime07:19:460
149950,0migration/019516-21kthreadcore10:04:420
149950,0migration/015791-21kthreadcore11:09:440
149950,0migration/015748-21kthreadcore09:39:400
149950,0migration/015004-21runrttasks08:49:430
149950,0migration/01374-21kthreadcore07:54:460
149950,0migration/012033-21fschecks_time10:24:390
149950,0migration/010285-21sh10:22:410
8315040,0irq/26-0000:00:0-21swapper/211:29:252
43469943,0cyclictest3721-21awk10:19:372
43469943,0cyclictest3599-21lldpd10:14:472
43469943,0cyclictest3599-21lldpd07:36:132
43469943,0cyclictest3599-21lldpd07:15:032
43469943,0cyclictest3262-21irqbalance06:54:302
43469943,0cyclictest3262-21irqbalance06:44:402
43469943,0cyclictest29713-21awk07:54:442
43469943,0cyclictest26288-21awk08:14:432
43469943,0cyclictest22436-21awk07:49:402
43469943,0cyclictest10004-21awk07:09:402
43429944,0cyclictest0-21swapper/110:50:041
43429943,0cyclictest3599-21lldpd09:13:331
43429943,0cyclictest32117-21sadc08:19:251
43429943,0cyclictest30111-21awk08:39:411
43429943,0cyclictest23871-21snmpd10:55:431
43429943,0cyclictest23871-21snmpd09:54:571
43429943,0cyclictest23871-21snmpd08:32:261
43429943,0cyclictest23871-21snmpd07:00:561
43429943,0cyclictest23871-21snmpd06:55:561
43429943,0cyclictest23871-21snmpd06:44:561
43429943,0cyclictest18846-21awk11:39:391
43429943,0cyclictest17392-21sadc11:39:251
43429943,0cyclictest10106-21awk07:09:401
43419943,0cyclictest7976-21awk07:39:420
43419943,0cyclictest5903-21kworker/0:208:54:480
43419943,0cyclictest2699-21awk07:04:400
43419943,0cyclictest23871-21snmpd11:34:290
43419943,0cyclictest23871-21snmpd11:02:130
29935040,0irq/33-eth0-tx-6315-21iceccd10:39:372
29935040,0irq/33-eth0-tx-11671-21ssh09:36:082
29935040,0irq/33-eth0-tx-0-21swapper/211:39:372
29935040,0irq/33-eth0-tx-0-21swapper/210:38:282
29935040,0irq/33-eth0-tx-0-21swapper/209:24:382
29935040,0irq/33-eth0-tx-0-21swapper/208:55:152
29935040,0irq/33-eth0-tx-0-21swapper/208:49:322
29935040,0irq/33-eth0-tx-0-21swapper/008:29:390
29935040,0irq/33-eth0-tx-0-21swapper/008:24:480
29935040,0irq/33-eth0-tx-0-21swapper/008:07:150
29935040,0irq/33-eth0-tx-0-21swapper/007:27:080
259940,0migration/27697-21kthreadcore08:44:442
259940,0migration/27565-21munin-node08:24:392
259940,0migration/27093-21aten_r1power_vo07:39:382
259940,0migration/26351-21timerandwakeup08:19:512
259940,0migration/227227-21df_abs06:59:342
259940,0migration/226800-21kthreadcore07:29:432
259940,0migration/224030-21cpuspeed_turbos08:59:362
259940,0migration/221199-21latency_hist07:49:252
259940,0migration/219592-21taskset10:29:272
259940,0migration/218741-21kthreadcore08:29:442
259940,0migration/215819-21sh11:35:162
259940,0migration/215776-21kthreadcore08:04:432
249940,0watchdog/25456-21perl06:39:422
249940,0watchdog/24304-21perl07:59:382
249940,0watchdog/224495-40yum-updatesd-he08:10:062
249940,0watchdog/222071-21diskstats08:34:382
249940,0watchdog/216627-21diskstats10:04:382
249940,0watchdog/20-21swapper/209:29:382
249940,0watchdog/20-21swapper/208:39:462
249940,0watchdog/20-21swapper/207:04:422
179940,0migration/1958-21if_eth109:04:431
179940,0migration/17751-21kthreadcore06:39:461
179940,0migration/15859-21kthreadcore10:19:391
179940,0migration/15803-21proc_pri07:34:471
179940,0migration/14949-21kthreadcore11:49:411
179940,0migration/14111-21fschecks_time07:59:381
179940,0migration/12814-21sh09:29:371
179940,0migration/127326-21kthreadcore08:34:431
179940,0migration/125080-21unixbench_singl09:19:511
179940,0migration/124559-21latency_hist10:09:251
179940,0migration/123459-21sh10:06:401
179940,0migration/121229-21kernelversion10:29:441
179940,0migration/119995-21hwlatdetect06:49:401
179940,0migration/118972-21needreboot09:39:421
179940,0migration/117439-21fschecks_count07:19:361
179940,0migration/114182-21kthreadcore10:24:421
179940,0migration/112312-21kthreadcore08:24:451
179940,0migration/111683-21fschecks_time11:09:401
169940,0watchdog/1371-21ssh11:01:101
169940,0watchdog/132017-21kthreadcore11:24:451
169940,0watchdog/132017-21kthreadcore11:24:451
169940,0watchdog/130082-21ssh10:39:341
169940,0watchdog/129823-21sh12:06:291
169940,0watchdog/128665-21diskmemload11:29:371
169940,0watchdog/127679-21timerandwakeup07:49:501
169940,0watchdog/124735-21smartctl08:59:381
169940,0watchdog/121382-21munin-node09:44:381
169940,0watchdog/120851-21ssh11:18:011
169940,0watchdog/120736-21ssh08:51:381
169940,0watchdog/10-21swapper/109:59:381
169940,0watchdog/10-21swapper/109:17:261
149940,0migration/06400-21runrttasks07:36:120
149940,0migration/030719-21sh12:08:500
149940,0migration/029406-21kthreadcore11:44:420
149940,0migration/013787-21runrttasks09:59:420
149940,0migration/011355-21kthreadcore07:09:420
149940,0migration/010752-21kthreadcore10:44:440
43469932,0cyclictest3599-21lldpd07:24:332
43429933,0cyclictest3599-21lldpd11:57:391
43429933,0cyclictest0-21swapper/110:14:361
43429932,0cyclictest30769-21awk09:49:401
43429932,0cyclictest30119-21irqstats07:54:451
43429932,0cyclictest23871-21snmpd08:55:121
43429932,0cyclictest23871-21snmpd07:27:411
43429932,0cyclictest22733-21cut08:09:451
43419933,0cyclictest5903-21kworker/0:209:54:280
43419933,0cyclictest5903-21kworker/0:209:49:260
43419933,0cyclictest5903-21kworker/0:209:34:280
43419933,0cyclictest5903-21kworker/0:209:29:260
43419933,0cyclictest5903-21kworker/0:209:24:260
43419933,0cyclictest5903-21kworker/0:209:19:320
43419933,0cyclictest5903-21kworker/0:209:14:300
43419933,0cyclictest5903-21kworker/0:209:04:300
43419933,0cyclictest4534-21kworker/0:007:44:260
43419933,0cyclictest4534-21kworker/0:007:14:320
43419933,0cyclictest4534-21kworker/0:006:54:280
43419933,0cyclictest4534-21kworker/0:006:49:260
43419933,0cyclictest4534-21kworker/0:006:44:260
43419933,0cyclictest25828-21kworker/0:010:34:280
43419933,0cyclictest22639-21kworker/0:011:54:300
43419933,0cyclictest17186-21kworker/0:111:14:260
169930,0watchdog/10-21swapper/110:35:221
169930,0watchdog/10-21swapper/109:25:141
169930,0watchdog/10-21swapper/108:19:421
169920,0watchdog/10-21swapper/107:44:301
169920,0watchdog/10-21swapper/107:14:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional