You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 03:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack0slot2.osadl.org (updated Fri Nov 22, 2024 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40111626250,6sleep70-21swapper/719:02:417
41057825040,7sleep50-21swapper/519:04:565
95521524946,1sleep70-21swapper/723:32:097
66962124846,1sleep70-21swapper/721:10:527
71121124644,1sleep70-21swapper/721:32:477
411040994644,2cyclictest587517-21kworker/u16:1+flush-8:020:57:546
49632224543,1sleep70-21swapper/719:44:407
97666923836,1sleep70-21swapper/723:46:357
41054023524,8sleep40-21swapper/419:04:234
411040992927,2cyclictest884490-21kworker/u16:5+events_unbound23:02:576
411040992826,2cyclictest587517-21kworker/u16:1+flush-8:021:32:506
41046622616,7sleep60-21swapper/619:03:206
411045992523,1cyclictest0-21swapper/719:17:557
411040992422,2cyclictest997902-21kworker/u16:3+events_unbound00:07:596
64933722321,1sleep70-21swapper/721:02:277
411040992321,2cyclictest878753-21kworker/u16:0+flush-8:022:57:096
411045992220,1cyclictest0-21swapper/722:53:057
411040992219,2cyclictest1007046-21kworker/u16:5+flush-8:000:27:406
411035992217,4cyclictest639-21systemd-journal19:17:404
411045992119,1cyclictest0-21swapper/722:22:407
411040992119,2cyclictest609368-21kworker/u16:0+events_unbound21:07:546
411040992018,2cyclictest476895-21kworker/u16:0+flush-8:019:43:056
411040992017,2cyclictest517656-21kworker/u16:4+flush-8:020:42:556
411037992017,2cyclictest587517-21kworker/u16:1+flush-8:021:33:105
411045991917,1cyclictest0-21swapper/721:48:067
411040991914,4cyclictest537639-21latency_hist20:07:416
411045991817,0cyclictest0-21swapper/723:07:397
411040991816,2cyclictest517656-21kworker/u16:4+flush-8:020:57:356
411037991814,3cyclictest306649-21kworker/u16:0+flush-8:019:17:535
411040991715,2cyclictest878753-21kworker/u16:0+flush-8:023:13:056
411040991715,2cyclictest853656-21kworker/u16:3+flush-8:022:47:406
411040991715,2cyclictest810786-21kworker/u16:4+flush-8:022:57:466
411037991715,2cyclictest410830-21kworker/u16:1+flush-8:019:42:555
411037991713,3cyclictest922475-21kworker/u16:2+flush-8:023:42:405
411035991714,2cyclictest420375-21kworker/u16:5+flush-8:019:27:554
41103599170,16cyclictest0-21swapper/419:18:034
411045991614,1cyclictest0-21swapper/721:23:067
411037991614,2cyclictest884490-21kworker/u16:5+flush-8:023:12:555
411037991613,2cyclictest1166-21snmpd20:45:325
411045991514,0cyclictest0-21swapper/723:22:397
411045991513,1cyclictest0-21swapper/719:42:417
41104599150,14cyclictest435349-21turbostat19:17:407
41104599150,14cyclictest0-21swapper/723:14:277
411040991513,2cyclictest922475-21kworker/u16:2+events_unbound23:44:116
411040991513,2cyclictest420375-21kworker/u16:5+flush-8:019:13:016
411037991513,2cyclictest410830-21kworker/u16:1+events_unbound19:27:535
411037991513,2cyclictest1870216-21kworker/u16:4+flush-8:019:13:095
411037991512,2cyclictest629680-21kworker/u16:3+events_unbound20:52:555
411035991512,2cyclictest884490-21kworker/u16:5+flush-8:023:12:534
411035991511,3cyclictest997902-21kworker/u16:3+flush-8:000:08:014
411035991511,3cyclictest587517-21kworker/u16:1+flush-8:020:57:584
411045991413,0cyclictest0-21swapper/720:47:297
411045991413,0cyclictest0-21swapper/700:09:207
411040991412,2cyclictest660287-21kworker/u16:4+events_unbound21:22:536
411040991412,2cyclictest587517-21kworker/u16:1+events_unbound20:38:046
411040991412,2cyclictest410830-21kworker/u16:1+events_unbound19:27:496
41104099140,1cyclictest0-21swapper/619:17:596
411037991411,2cyclictest732125-21kworker/u16:3+flush-8:021:47:555
411037991411,2cyclictest1166-21snmpd20:40:055
411035991412,2cyclictest587517-21kworker/u16:1+events_unbound22:06:124
411045991312,0cyclictest0-21swapper/720:42:267
41104599130,6cyclictest0-21swapper/720:54:597
411040991312,1cyclictest0-21swapper/623:28:036
411040991311,2cyclictest730667-21kworker/u16:2+events_unbound22:03:036
41103799138,4cyclictest731219-21exim421:42:405
411037991311,2cyclictest884490-21kworker/u16:5+flush-8:023:02:405
411037991311,2cyclictest832590-21kworker/u16:1+events_unbound23:21:505
41103599139,3cyclictest629680-21kworker/u16:3+flush-8:020:52:594
411035991311,2cyclictest587517-21kworker/u16:1+events_unbound20:39:114
411035991310,2cyclictest690850-21kworker/u16:0+flush-8:021:57:404
411035991310,2cyclictest690850-21kworker/u16:0+flush-8:021:57:404
411035991310,2cyclictest690850-21kworker/u16:0+flush-8:021:23:114
41104599129,2cyclictest1007742-21turbostat00:02:397
411045991211,0cyclictest0-21swapper/722:58:457
411045991210,1cyclictest0-21swapper/722:52:407
411040991210,2cyclictest690850-21kworker/u16:0+flush-8:021:48:406
411037991211,1cyclictest0-21swapper/523:29:075
411037991211,1cyclictest0-21swapper/522:54:265
411037991210,2cyclictest690850-21kworker/u16:0+events_unbound22:03:075
411037991210,2cyclictest517656-21kworker/u16:4+flush-8:021:01:495
41103599129,2cyclictest609368-21kworker/u16:0+flush-8:021:08:164
41103599127,4cyclictest639-21systemd-journal20:27:404
411035991211,1cyclictest0-21swapper/421:48:054
411035991210,2cyclictest609368-21kworker/u16:0+events_unbound20:44:494
41103599120,1cyclictest0-21swapper/423:04:144
411045991110,0cyclictest0-21swapper/722:06:447
411045991110,0cyclictest0-21swapper/719:27:597
411040991110,1cyclictest0-21swapper/623:22:226
41103799113,5cyclictest588646-21munin-run20:32:405
411037991110,1cyclictest0-21swapper/500:09:195
41103599119,2cyclictest517656-21kworker/u16:4+flush-8:020:02:404
41103599119,2cyclictest27269-21gvfs-afc-volume23:18:024
41103599119,2cyclictest26742-21gnome-shell22:52:494
41103599118,2cyclictest810786-21kworker/u16:4+events_unbound22:47:404
411035991110,1cyclictest0-21swapper/423:45:334
411035991110,1cyclictest0-21swapper/423:01:154
411035991110,1cyclictest0-21swapper/419:43:054
41104099108,1cyclictest823175-21sed22:27:406
41103799108,2cyclictest660287-21kworker/u16:4+flush-8:021:24:105
41103799107,2cyclictest885339-21irqcore22:57:535
4110409998,1cyclictest710995-21date21:32:406
4110379997,2cyclictest660287-21kworker/u16:4+flush-8:021:07:535
4110379997,1cyclictest792471-21awk22:12:405
4110359997,2cyclictest27932-21gmain21:34:494
4110359997,1cyclictest0-21swapper/421:42:404
4110359995,3cyclictest455922-21latency_hist19:27:404
4110459987,0cyclictest0-21swapper/721:47:407
4110459986,1cyclictest0-21swapper/722:47:407
4110459986,1cyclictest0-21swapper/722:12:407
4110409987,1cyclictest945748-21latency_hist23:27:406
4110409984,3cyclictest761836-21systemd-run21:57:406
4110409984,3cyclictest761836-21systemd-run21:57:406
4110409983,4cyclictest874120-21cat22:52:406
4110409983,4cyclictest1048367-21latency_hist00:22:406
4110409980,7cyclictest792772-21latency_hist22:12:406
4110379986,1cyclictest0-21swapper/521:57:405
4110379986,1cyclictest0-21swapper/521:57:405
4110379986,1cyclictest0-21swapper/520:27:405
4110379986,1cyclictest0-21swapper/519:52:405
4110379980,7cyclictest1038577-21latency_hist00:17:405
4110359986,2cyclictest639-21systemd-journal00:17:404
4110359986,1cyclictest0-21swapper/420:17:414
4110359985,2cyclictest1166-21snmpd23:29:284
4110359983,4cyclictest965459-21sh23:37:394
4110459976,1cyclictest0-21swapper/720:22:417
4110459976,0cyclictest0-21swapper/721:57:407
4110459976,0cyclictest0-21swapper/721:57:407
4110459976,0cyclictest0-21swapper/720:02:407
4110459976,0cyclictest0-21swapper/700:17:407
4110459975,1cyclictest0-21swapper/720:27:407
4110459975,1cyclictest0-21swapper/720:07:407
4110459975,1cyclictest0-21swapper/700:32:407
4110459975,1cyclictest0-21swapper/700:07:407
4110409976,1cyclictest812946-21latency_hist22:22:406
4110409976,1cyclictest731579-21latency_hist21:42:406
4110409976,1cyclictest507179-21latency_hist19:52:406
4110409975,1cyclictest0-21swapper/600:32:406
4110409973,3cyclictest660248-21sed21:07:406
4110409973,3cyclictest527599-21cat20:02:406
4110409973,3cyclictest476643-21cat19:37:406
4110409972,4cyclictest456289-21latency_hist19:27:406
4110409970,6cyclictest0-21swapper/620:52:406
4110379975,1cyclictest0-21swapper/522:22:405
4110379975,1cyclictest0-21swapper/520:22:415
4110379975,1cyclictest0-21swapper/520:17:405
4110379975,1cyclictest0-21swapper/520:02:405
4110379973,3cyclictest965839-21sendmail23:37:395
4110379973,3cyclictest853662-21cut22:42:405
4110379973,3cyclictest456415-21cut19:27:405
4110379973,3cyclictest1069265-21date00:32:405
4110379972,4cyclictest628992-21latency_hist20:52:405
4110379970,6cyclictest599187-21cut20:37:415
4110359975,2cyclictest1028567-21kworker/u16:0+flush-8:000:37:404
4110359975,1cyclictest0-21swapper/423:27:404
4110359975,1cyclictest0-21swapper/422:52:404
4110359975,1cyclictest0-21swapper/422:27:404
4110359975,1cyclictest0-21swapper/422:12:404
4110359975,1cyclictest0-21swapper/420:32:414
4110359975,1cyclictest0-21swapper/400:27:404
4110359974,2cyclictest0-21swapper/419:57:414
4110359973,3cyclictest476506-21sendmail19:37:404
4110459966,0cyclictest0-21swapper/721:17:407
4110459965,0cyclictest0-21swapper/720:52:407
4110459965,0cyclictest0-21swapper/720:17:407
4110459965,0cyclictest0-21swapper/719:52:417
4110459964,1cyclictest0-21swapper/723:52:407
4110459964,1cyclictest0-21swapper/723:12:407
4110459964,1cyclictest0-21swapper/722:02:407
4110459964,1cyclictest0-21swapper/720:32:407
4110459964,1cyclictest0-21swapper/700:27:397
4110459964,1cyclictest0-21swapper/700:22:407
4110459960,4cyclictest731252-21sleep21:42:407
4110409965,1cyclictest598954-21latency_hist20:37:416
4110409965,1cyclictest588682-21latency_hist20:32:416
4110409965,1cyclictest0-21swapper/622:17:406
4110409965,1cyclictest0-21swapper/621:22:406
4110409964,1cyclictest0-21swapper/600:02:396
4110409963,2cyclictest1018184-21latency_hist00:07:406
4110409962,3cyclictest987409-21sed23:52:406
4110409962,3cyclictest914967-21sed23:12:396
4110409962,3cyclictest864-21dbus-daemon23:37:396
4110409962,3cyclictest833595-21latency_hist22:32:406
4110409962,3cyclictest680382-21latency_hist21:17:406
4110409962,3cyclictest568479-21grep20:22:416
4110409962,3cyclictest421633-21latency_hist19:12:406
4110409962,3cyclictest1079578-21cut00:37:406
4110409960,5cyclictest578435-21journal-offline20:27:406
4110379965,1cyclictest0-21swapper/522:32:405
4110379964,1cyclictest14250irq/27-ahci[0000:00:1f.2]21:17:405
4110379964,1cyclictest0-21swapper/500:07:405
4110379962,3cyclictest987517-21cat23:52:405
4110379962,3cyclictest864-21dbus-daemon19:37:405
4110379962,3cyclictest741718-21cat21:47:405
4110379962,3cyclictest660127-21cat21:07:405
4110379962,3cyclictest1049013-21latency_hist00:22:405
4110379962,3cyclictest1007799-21sed00:02:395
4110379961,4cyclictest864-21dbus-daemon19:12:405
4110379961,4cyclictest843452-21idleruntime-cro22:37:405
4110379960,5cyclictest1058953-21cat00:27:405
4110379960,4cyclictest945670-21latency_hist23:27:405
4110359965,1cyclictest0-21swapper/420:22:414
4110359964,1cyclictest0-21swapper/422:17:404
4110359963,2cyclictest599029-21latency_hist20:37:414
4110359962,3cyclictest976362-21latency_hist23:42:404
4110359962,3cyclictest914993-21date23:12:394
4110359962,3cyclictest843557-21date22:37:404
4110359962,3cyclictest680596-21date21:17:404
4110359962,3cyclictest629553-21sed20:52:404
4110359962,3cyclictest1018132-21cut00:07:404
4110359960,5cyclictest507074-21pluginstate19:52:404
4110459954,0cyclictest0-21swapper/720:12:417
4110459954,0cyclictest0-21swapper/719:57:417
4110459953,1cyclictest0-21swapper/722:42:407
4110459953,1cyclictest0-21swapper/720:37:417
4110459953,1cyclictest0-21swapper/719:12:407
4110459950,4cyclictest456252-21taskset19:27:407
4110409951,3cyclictest517541-21cat19:57:416
4110409950,4cyclictest976317-21latency_hist23:42:406
4110409950,4cyclictest853762-21latency_hist22:42:406
4110409950,4cyclictest843439-21sendmail22:37:406
4110409950,4cyclictest772226-21date22:02:406
4110409950,4cyclictest741564-21latency_hist21:47:406
4110409950,4cyclictest558076-21turbostat.cron20:17:406
4110409950,4cyclictest486399-21latency_hist19:42:416
4110409950,4cyclictest1038376-21crond00:17:396
4110379952,2cyclictest1079482-21latency_hist00:37:405
4110379951,3cyclictest863784-21sh22:47:405
4110379951,3cyclictest772227-21cat22:02:405
4110379950,4cyclictest976301-21cat23:42:405
4110379950,4cyclictest914968-21latency_hist23:12:395
4110379950,4cyclictest873958-21sh22:52:405
4110379950,4cyclictest823203-21sed22:27:405
4110379950,4cyclictest802940-21latency_hist22:17:405
4110379950,4cyclictest710965-21cut21:32:405
4110379950,4cyclictest690841-21latency_hist21:22:405
4110379950,4cyclictest548234-21sendmail20:12:415
4110379950,4cyclictest537751-21systemd-run20:07:405
4110379950,4cyclictest517566-21taskset19:57:415
4110379950,4cyclictest486933-21sendmail19:42:405
4110359953,1cyclictest0-21swapper/423:57:404
4110359953,1cyclictest0-21swapper/423:57:404
4110359950,4cyclictest987371-21exim423:52:394
4110359950,4cyclictest853658-21latency_hist22:42:404
4110359950,4cyclictest833559-21sendmail22:32:404
4110359950,4cyclictest812859-21awk22:22:404
4110359950,4cyclictest772216-21sendmail22:02:404
4110359950,4cyclictest741148-21latency_hist21:47:404
4110359950,4cyclictest710988-21latency_hist21:32:404
4110359950,4cyclictest690787-21latency_hist21:22:404
4110359950,4cyclictest660139-21grep21:07:404
4110359950,4cyclictest537799-21latency_hist20:07:404
4110359950,4cyclictest486973-21exim419:42:404
4110359950,4cyclictest421635-21exim419:12:404
4110359950,4cyclictest1069039-21sh00:32:394
4110359950,4cyclictest1048857-21latency_hist00:22:404
4110359950,4cyclictest1007725-21sendmail00:02:394
4110459944,0cyclictest0-21swapper/723:37:397
4110459943,0cyclictest0-21swapper/722:37:407
4110459943,0cyclictest0-21swapper/722:32:407
4110459943,0cyclictest0-21swapper/722:27:407
4110459943,0cyclictest0-21swapper/721:07:407
4110459942,1cyclictest0-21swapper/700:37:407
4110459940,2cyclictest0-21swapper/721:32:407
4110459940,2cyclictest0-21swapper/719:37:407
4110409940,3cyclictest997859-21cat23:57:406
4110409940,3cyclictest997859-21cat23:57:406
4110409940,3cyclictest548238-21latency_hist20:12:416
4110379940,3cyclictest997208-21cron23:57:405
4110379940,3cyclictest997208-21cron23:57:405
4110359940,3cyclictest548237-21sed20:12:414
4110459932,0cyclictest0-21swapper/723:27:407
4110459930,2cyclictest0-21swapper/722:17:407
4110459921,1cyclictest0-21swapper/723:57:407
4110459921,1cyclictest0-21swapper/723:57:407
4110459920,1cyclictest0-21swapper/723:42:407
4110459920,1cyclictest0-21swapper/721:22:407
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional