You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-23 - 17:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot2.osadl.org (updated Sat Nov 23, 2024 12:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
374182524846,1sleep70-21swapper/712:35:367
366011124846,1sleep70-21swapper/711:54:187
362939024846,1sleep70-21swapper/711:38:407
341401124846,1sleep70-21swapper/709:48:567
336291424846,1sleep70-21swapper/709:23:407
317907924846,1sleep70-21swapper/707:55:117
314837824846,1sleep70-21swapper/707:39:097
312817924846,1sleep70-21swapper/707:32:047
343437023230,1sleep70-21swapper/709:58:497
348548622725,1sleep70-21swapper/710:25:017
3073441992522,2cyclictest3082577-21kworker/u16:6+events_unbound07:27:365
3073438992520,4cyclictest3383639-21latency_hist09:37:344
3073441992321,2cyclictest3280485-21kworker/u16:0+events_unbound10:13:025
307345399210,12cyclictest0-21swapper/708:36:357
3073441992117,2cyclictest3494750-21kworker/u16:2+events_unbound11:42:345
3073448992016,3cyclictest3082577-21kworker/u16:6+events_unbound07:37:576
3073441992018,2cyclictest3190203-21kworker/u16:1+events_unbound08:17:495
3073441992016,3cyclictest3670620-21latency_hist12:02:335
3073448991915,3cyclictest3650139-21awk11:52:336
3073441991917,2cyclictest3433716-21kworker/u16:3+events_unbound10:37:475
3073441991917,2cyclictest3433716-21kworker/u16:3+events_unbound10:37:475
3073441991816,2cyclictest3117231-21kworker/u16:2+flush-8:007:47:445
3073438991814,3cyclictest3363202-21latency_hist09:27:344
3073453991716,0cyclictest0-21swapper/712:17:347
307345399170,1cyclictest0-21swapper/711:46:227
3073441991715,2cyclictest3535755-21kworker/u16:4+events_unbound11:51:475
3073441991715,2cyclictest3535755-21kworker/u16:4+events_unbound11:38:475
3073441991715,2cyclictest3071790-21kworker/u16:3+flush-8:007:07:535
3073438991714,2cyclictest3137507-21kworker/u16:1+events_unbound07:47:344
3073438991713,3cyclictest3071790-21kworker/u16:3+events_unbound07:12:494
307345399160,1cyclictest0-21swapper/709:43:497
3073448991612,3cyclictest3071790-21kworker/u16:3+flush-8:007:12:506
3073441991614,2cyclictest3280485-21kworker/u16:0+events_unbound10:25:345
3073441991614,2cyclictest3117231-21kworker/u16:2+flush-8:007:42:345
3073441991614,2cyclictest3071790-21kworker/u16:3+flush-8:007:22:495
307343899168,1cyclictest0-21swapper/411:38:554
307345399150,8cyclictest0-21swapper/708:02:447
307345399150,7cyclictest0-21swapper/707:50:557
3073448991513,2cyclictest3517069-21kworker/u16:0+events_unbound11:37:476
3073448991513,2cyclictest3160064-21kworker/u16:3+flush-8:007:47:476
3073441991513,2cyclictest3517069-21kworker/u16:0+events_unbound12:07:195
3073441991513,2cyclictest3494750-21kworker/u16:2+events_unbound11:55:015
3073441991513,2cyclictest3425184-21kworker/u16:2+events_unbound10:01:125
3073441991513,2cyclictest3334507-21kworker/u16:3+events_unbound09:29:135
3073441991513,2cyclictest3334507-21kworker/u16:3+events_unbound09:22:485
3073441991513,2cyclictest3190203-21kworker/u16:1+flush-8:008:42:345
3073441991513,2cyclictest3117231-21kworker/u16:2+events_unbound08:02:585
307344899149,4cyclictest3587993-21latency_hist11:17:346
3073441991412,2cyclictest3535755-21kworker/u16:4+flush-8:012:11:145
3073441991412,2cyclictest3082577-21kworker/u16:6+flush-8:007:12:365
3073438991413,1cyclictest0-21swapper/408:02:464
307344899139,3cyclictest3577883-21latency_hist11:12:346
307344899139,3cyclictest3080611-21cut07:07:536
307344899136,7cyclictest0-21swapper/607:55:436
3073448991311,2cyclictest3474534-21kworker/u16:5+flush-8:010:27:346
3073448991311,2cyclictest3425184-21kworker/u16:2+flush-8:010:02:346
3073448991311,2cyclictest3362908-21kworker/u16:1+events_unbound09:28:036
307344899130,1cyclictest0-21swapper/609:32:556
3073441991311,2cyclictest3494750-21kworker/u16:2+events_unbound12:19:075
3073441991311,2cyclictest3362908-21kworker/u16:1+events_unbound09:35:135
3073441991311,2cyclictest3280485-21kworker/u16:0+events_unbound09:43:395
3073438991310,2cyclictest3071790-21kworker/u16:3+events_unbound07:07:484
307345399129,2cyclictest3353349-21turbostat09:22:347
307344899128,3cyclictest3252343-21cat08:32:476
3073448991210,2cyclictest3494750-21kworker/u16:2+events_unbound11:42:596
307344899120,1cyclictest0-21swapper/612:34:066
3073441991210,2cyclictest3732508-21kworker/u16:1+events_unbound12:34:175
3073441991210,2cyclictest3128127-21kworker/u16:0+flush-8:008:33:445
3073441991210,2cyclictest3082577-21kworker/u16:6+events_unbound07:53:575
307343899124,8cyclictest0-21swapper/407:50:044
3073438991211,1cyclictest0-21swapper/407:23:594
307343899120,1cyclictest0-21swapper/407:53:094
307345399119,1cyclictest0-21swapper/709:28:097
3073453991110,0cyclictest0-21swapper/708:21:507
3073453991110,0cyclictest0-21swapper/707:13:157
307345399110,9cyclictest0-21swapper/712:07:567
307345399110,4cyclictest0-21swapper/712:02:507
307345399110,1cyclictest0-21swapper/710:39:387
307345399110,1cyclictest0-21swapper/710:39:387
307344899119,2cyclictest3362908-21kworker/u16:1+events_unbound10:38:016
307344899119,2cyclictest3362908-21kworker/u16:1+events_unbound10:38:016
307344899118,2cyclictest1166-21snmpd09:26:186
307344899118,2cyclictest0-21swapper/612:17:536
307344899117,3cyclictest3651641-21cat11:52:496
3073448991110,1cyclictest0-21swapper/609:44:186
307344899110,11cyclictest0-21swapper/610:17:246
307344899110,11cyclictest0-21swapper/609:50:176
307344899110,11cyclictest0-21swapper/607:25:336
307344199118,2cyclictest0-21swapper/510:07:345
3073441991110,1cyclictest0-21swapper/509:47:545
307343899118,2cyclictest3395569-21irqcore09:42:494
307343899118,2cyclictest1166-21snmpd12:34:424
307343899118,2cyclictest1166-21snmpd07:28:214
3073438991110,1cyclictest0-21swapper/412:07:004
3073438991110,1cyclictest0-21swapper/410:42:294
3073438991110,1cyclictest0-21swapper/410:42:294
3073438991110,1cyclictest0-21swapper/410:24:024
3073438991110,1cyclictest0-21swapper/410:14:544
307343899110,1cyclictest0-21swapper/407:38:204
307345399109,0cyclictest0-21swapper/709:35:007
307345399109,0cyclictest0-21swapper/707:09:037
307344899109,1cyclictest0-21swapper/612:10:276
307344899105,3cyclictest3535755-21kworker/u16:4+flush-8:010:57:346
307343899109,1cyclictest862-21avahi-daemon12:10:094
307343899109,1cyclictest0-21swapper/411:50:454
307343899109,1cyclictest0-21swapper/409:30:354
307343899103,7cyclictest0-21swapper/408:18:404
30734539999,0cyclictest0-21swapper/712:18:497
30734539998,0cyclictest0-21swapper/707:23:217
30734489990,3cyclictest3195342-21kthreadcore08:02:526
30734489990,2cyclictest3557645-21date11:02:346
30734489990,2cyclictest3122562-21cat07:27:536
30734419997,1cyclictest0-21swapper/508:12:345
30734389998,1cyclictest0-21swapper/411:53:554
30734389998,1cyclictest0-21swapper/409:47:564
30734389995,3cyclictest3252341-21cat08:32:474
30734389994,4cyclictest3634428-21cut11:42:524
30734389994,4cyclictest3598335-21latency_hist11:22:344
30734389990,8cyclictest0-21swapper/409:59:494
30734489986,1cyclictest0-21swapper/612:27:346
30734489986,1cyclictest0-21swapper/607:22:346
30734489985,2cyclictest3292205-21latency_hist08:52:356
30734489983,4cyclictest3229226-21sed08:17:556
30734489980,1cyclictest0-21swapper/612:03:006
30734489980,1cyclictest0-21swapper/610:47:346
30734419986,2cyclictest3517069-21kworker/u16:0+events_unbound11:22:345
30734389986,1cyclictest0-21swapper/412:27:334
30734389986,1cyclictest0-21swapper/412:17:554
30734389984,2cyclictest3240966-21turbostat.cron08:27:344
30734539975,1cyclictest0-21swapper/711:47:587
30734539975,1cyclictest0-21swapper/711:12:347
30734489975,1cyclictest0-21swapper/611:27:346
30734489975,1cyclictest0-21swapper/608:17:346
30734489975,1cyclictest0-21swapper/608:17:346
30734489974,2cyclictest0-21swapper/608:27:346
30734489973,3cyclictest3475942-21cut10:22:346
30734489973,3cyclictest3445294-21cat10:07:346
30734419974,2cyclictest751ktimers/507:47:345
30734419974,2cyclictest3292010-21cut08:52:345
30734419973,3cyclictest3732458-21cut12:32:345
30734419973,3cyclictest3333011-21date09:12:355
30734419972,4cyclictest3496526-21date10:32:345
30734389975,1cyclictest0-21swapper/409:07:354
30734389975,1cyclictest0-21swapper/408:02:344
30734389973,3cyclictest3588245-21cut11:17:344
30734389973,3cyclictest3526865-21journal-offline10:47:344
30734389973,3cyclictest3271861-21cut08:42:344
30734539965,0cyclictest0-21swapper/711:07:347
30734539964,1cyclictest0-21swapper/710:32:347
30734539964,1cyclictest0-21swapper/710:07:347
30734539960,4cyclictest3456600-21taskset10:12:507
30734489964,1cyclictest3598335-21latency_hist11:22:346
30734489964,1cyclictest0-21swapper/611:32:346
30734489963,2cyclictest3670620-21latency_hist12:02:346
30734489962,3cyclictest3496350-21wc10:32:346
30734489962,3cyclictest3322710-21basename09:07:346
30734489962,3cyclictest3302507-21cut08:57:356
30734489962,3cyclictest3251425-21latency_hist08:32:346
30734489962,3cyclictest3210346-21sed08:12:346
30734489962,3cyclictest3189988-21date08:02:346
30734489962,3cyclictest3159201-21sort07:47:346
30734489961,4cyclictest3353348-21latency_hist09:22:356
30734419965,1cyclictest0-21swapper/511:02:345
30734419965,1cyclictest0-21swapper/510:22:345
30734419964,1cyclictest14250irq/27-ahci[0000:00:1f.2]11:17:345
30734419964,1cyclictest0-21swapper/507:37:345
30734419962,3cyclictest864-21dbus-daemon12:27:335
30734419962,3cyclictest3609902-21cut11:32:345
30734419962,3cyclictest3547534-21latency_hist10:57:345
30734419962,3cyclictest3241184-21date08:27:345
30734419962,3cyclictest3190018-21cut08:02:345
30734419962,3cyclictest3108405-21exim407:22:345
30734419961,4cyclictest3526904-21exim410:47:345
30734419960,5cyclictest3599486-21cat11:27:345
30734419960,5cyclictest3506831-21cat10:37:345
30734419960,5cyclictest3282098-21cat08:47:355
30734419960,5cyclictest3220690-21cat08:17:345
30734419960,5cyclictest3220690-21cat08:17:345
30734419960,3cyclictest3620022-21cat11:37:345
30734419960,3cyclictest3322779-21cat09:07:345
30734389964,1cyclictest0-21swapper/408:12:344
30734389963,2cyclictest3701189-21latency_hist12:17:344
30734389962,3cyclictest3619917-21date11:37:344
30734389962,3cyclictest3578099-21cut11:12:344
30734389962,3cyclictest3557474-21wc11:02:344
30734389962,3cyclictest3445547-21cut10:07:344
30734389962,3cyclictest3353352-21ls09:22:344
30734389962,3cyclictest3343152-21ls09:17:344
30734389962,3cyclictest3251423-21sort08:32:344
30734389961,4cyclictest639-21systemd-journal10:37:344
30734389961,4cyclictest639-21systemd-journal10:37:344
30734389961,4cyclictest3599656-21cut11:27:344
30734389961,4cyclictest3475912-21cat10:22:344
30734389960,5cyclictest3332984-21exim409:12:354
30734539954,1cyclictest0-21swapper/707:47:347
30734539954,0cyclictest0-21swapper/708:27:347
30734539953,1cyclictest0-21swapper/711:32:347
30734539953,1cyclictest0-21swapper/710:47:347
30734539953,1cyclictest0-21swapper/707:37:347
30734489953,1cyclictest3701810-21exim412:17:346
30734489951,3cyclictest3619934-21latency_hist11:37:346
30734489951,3cyclictest3139118-21latency_hist07:37:346
30734489950,5cyclictest0-21swapper/609:17:346
30734489950,4cyclictest3732401-21cat12:32:346
30734489950,4cyclictest3567370-21latency_hist11:07:346
30734489950,4cyclictest3536697-21latency_hist10:52:346
30734489950,4cyclictest3506663-21exim410:37:346
30734489950,4cyclictest3506663-21exim410:37:346
30734489950,4cyclictest3455717-21cat10:12:346
30734489950,4cyclictest3425098-21date09:57:346
30734489950,4cyclictest3394370-21latency_hist09:42:346
30734489950,4cyclictest3332564-21latency_hist09:12:346
30734489950,4cyclictest3312783-21exim409:02:356
30734489950,4cyclictest3282110-21sed08:47:356
30734489950,4cyclictest3271603-21idleruntime-cro08:42:346
30734489950,4cyclictest3073567-21cut07:07:356
30734419954,1cyclictest0-21swapper/508:57:345
30734419950,4cyclictest3701796-21cut12:17:345
30734419950,4cyclictest3578028-21cat11:12:345
30734419950,4cyclictest3567977-21sendmail11:07:345
30734419950,4cyclictest3537375-21cut10:52:345
30734419950,4cyclictest3506688-21latency_hist10:37:345
30734419950,4cyclictest3425115-21cat09:57:345
30734419950,4cyclictest3394415-21cut09:42:345
30734419950,4cyclictest3353391-21cat09:22:345
30734419950,4cyclictest3343136-21exim409:17:345
30734419950,4cyclictest3251442-21latency_hist08:32:345
30734419950,4cyclictest3073574-21exim407:07:355
30734389953,1cyclictest3732409-21latency_hist12:32:344
30734389951,3cyclictest3312125-21cron09:02:354
30734389950,4cyclictest3671062-21date12:02:334
30734389950,4cyclictest3609718-21exim411:32:344
30734389950,4cyclictest3567981-21cut11:07:344
30734389950,4cyclictest3547231-21munin-run10:57:344
30734389950,4cyclictest3537287-21grep10:52:344
30734389950,4cyclictest3496230-21exim410:32:344
30734389950,4cyclictest3455745-21exim410:12:344
30734389950,4cyclictest3424488-21cron09:57:344
30734389950,4cyclictest3394410-21latency_hist09:42:344
30734389950,4cyclictest3302129-21/usr/sbin/munin08:57:344
30734389950,4cyclictest3291644-21cron08:52:344
30734389950,4cyclictest3282133-21exim408:47:354
30734389950,4cyclictest3220682-21cut08:17:344
30734389950,4cyclictest3220682-21cut08:17:344
30734389950,4cyclictest3139077-21latency_hist07:37:344
30734389950,4cyclictest3108378-21cat07:22:344
30734389950,4cyclictest3073539-21wc07:07:354
30734539943,0cyclictest0-21swapper/710:22:347
30734539943,0cyclictest0-21swapper/709:07:357
30734539943,0cyclictest0-21swapper/708:32:347
30734539943,0cyclictest0-21swapper/707:22:347
30734539943,0cyclictest0-21swapper/707:07:357
30734539942,1cyclictest0-21swapper/712:27:347
30734539942,1cyclictest0-21swapper/708:17:347
30734539942,1cyclictest0-21swapper/708:17:347
30734539940,3cyclictest3599427-21turbostat11:27:347
30734539940,3cyclictest3547248-21sleep10:57:347
30734539940,3cyclictest3343171-21sleep09:17:347
30734539940,3cyclictest3292023-21sleep08:52:347
30734539940,2cyclictest0-21swapper/708:47:357
30734419940,3cyclictest3455718-21latency_hist10:12:345
30734419940,3cyclictest3312751-21latency_hist09:02:355
30734539933,0cyclictest0-21swapper/709:42:347
30734539933,0cyclictest0-21swapper/709:02:357
30734539932,1cyclictest0-21swapper/711:17:347
30734539932,0cyclictest0-21swapper/709:57:347
30734539931,1cyclictest0-21swapper/711:02:347
30734539931,1cyclictest0-21swapper/708:42:347
30734539931,1cyclictest0-21swapper/708:02:347
30734539930,2cyclictest3619850-21turbostat11:37:347
30734539930,2cyclictest3598360-21sleep11:22:347
30734539930,2cyclictest3210274-21turbostat08:12:347
30734539920,1cyclictest0-21swapper/712:32:347
30734539920,1cyclictest0-21swapper/712:02:337
30734539920,1cyclictest0-21swapper/710:52:347
30734539920,1cyclictest0-21swapper/710:37:347
30734539920,1cyclictest0-21swapper/710:37:347
30734539920,1cyclictest0-21swapper/710:12:347
30734539920,1cyclictest0-21swapper/709:12:347
30734539920,1cyclictest0-21swapper/708:57:347
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional