You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5.osadl.org (updated Thu Apr 03, 2025 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2857999300,29cyclictest23091-21idleruntime-cro21:15:003
28578993028,1cyclictest25398-21missed_timers22:40:182
2857899300,29cyclictest609-21dbus-daemon00:10:012
28579992927,1cyclictest0-21swapper/321:55:113
2857999290,28cyclictest18603-21munin-run22:25:013
2857999290,1cyclictest3058-21cron19:25:003
28578992926,2cyclictest19655-21sh20:00:012
2857899290,28cyclictest30684-21munin-run00:20:012
2857899290,28cyclictest0-21swapper/223:00:162
28577992928,1cyclictest2989-21apt-get23:10:081
2857799290,28cyclictest8292-21cron22:00:001
2857799290,28cyclictest30794-21perf21:35:001
2857799290,1cyclictest26007-21cron00:10:011
28576992927,1cyclictest23165-21munin-run00:00:010
2857699290,28cyclictest0-21swapper/021:30:110
28579992828,0cyclictest16102-21apt-get21:00:123
28579992827,1cyclictest0-21swapper/320:30:153
2857999280,0cyclictest29307-21apt-get20:20:133
2857999280,0cyclictest0-21swapper/323:10:083
28578992827,0cyclictest14192-21apt-get22:15:142
28578992825,2cyclictest23107-21cat00:00:012
28577992828,0cyclictest0-21swapper/120:30:151
28577992826,1cyclictest19654-21sendmail-msp20:00:011
28577992826,1cyclictest0-21swapper/100:00:011
2857799280,1cyclictest30705-21sendmail-msp00:20:011
2857799280,0cyclictest0-21swapper/123:00:161
2857699280,28cyclictest0-21swapper/020:35:120
2857699280,28cyclictest0-21swapper/000:35:140
28579992727,0cyclictest32148-21apt-get23:00:153
28579992727,0cyclictest0-21swapper/323:30:113
28579992726,1cyclictest623-21polkitd20:10:013
28579992726,1cyclictest609-21dbus-daemon21:00:013
2857999270,0cyclictest0-21swapper/322:15:143
2857999270,0cyclictest0-21swapper/320:10:153
28578992727,0cyclictest4158-21apt-get20:35:122
28578992727,0cyclictest1840-21apt-get20:30:142
28578992727,0cyclictest0-21swapper/223:10:082
28578992727,0cyclictest0-21swapper/222:30:092
28578992727,0cyclictest0-21swapper/200:35:152
28578992725,1cyclictest21057-21apt-get21:10:152
2857899270,27cyclictest0-21swapper/220:00:122
2857899270,0cyclictest0-21swapper/220:20:132
2857899270,0cyclictest0-21swapper/219:40:132
28577992727,0cyclictest31062-21apt-get21:35:131
28577992727,0cyclictest0-21swapper/123:40:131
28577992727,0cyclictest0-21swapper/122:15:141
28577992727,0cyclictest0-21swapper/121:50:081
28577992727,0cyclictest0-21swapper/120:00:121
28577992727,0cyclictest0-21swapper/100:35:151
28576992727,0cyclictest0-21swapper/022:20:150
28576992727,0cyclictest0-21swapper/019:55:110
28576992727,0cyclictest0-21swapper/019:10:130
28576992726,1cyclictest3862-21cron20:35:000
28576992725,1cyclictest28355-21munin-run00:15:010
2857699270,27cyclictest0-21swapper/020:20:130
2857699270,0cyclictest21057-21apt-get21:10:150
28579992626,0cyclictest0-21swapper/323:35:123
28579992626,0cyclictest0-21swapper/320:40:133
28579992626,0cyclictest0-21swapper/320:35:123
28579992626,0cyclictest0-21swapper/300:15:133
28578992626,0cyclictest0-21swapper/223:40:132
28578992626,0cyclictest0-21swapper/221:50:082
28578992626,0cyclictest0-21swapper/221:35:132
28578992623,2cyclictest13897-21cron22:15:002
28577992626,0cyclictest31627-21apt-get20:25:121
28577992626,0cyclictest0-21swapper/123:35:111
28577992626,0cyclictest0-21swapper/122:05:011
28577992626,0cyclictest0-21swapper/121:10:151
28577992625,0cyclictest17332-21apt-get23:45:021
28577992624,1cyclictest609-21dbus-daemon22:15:001
28576992626,0cyclictest0-21swapper/022:35:120
28576992626,0cyclictest0-21swapper/021:05:120
28579992525,0cyclictest0-21swapper/323:45:023
28578992525,0cyclictest0-21swapper/222:05:012
28578992525,0cyclictest0-21swapper/220:25:122
28578992524,1cyclictest24587-21apt-get20:10:152
28578992523,1cyclictest28739-21apt-get21:30:122
28577992525,0cyclictest28788-21apt-get19:10:131
28577992525,0cyclictest16539-21apt-get22:20:161
28577992525,0cyclictest11142-21apt-get20:50:161
28576992523,1cyclictest0-21swapper/020:15:010
28579992424,0cyclictest0-21swapper/323:50:083
28579992424,0cyclictest0-21swapper/319:50:163
28579992422,1cyclictest0-21swapper/321:40:013
28578992424,0cyclictest0-21swapper/223:45:012
28578992424,0cyclictest0-21swapper/222:20:162
28578992424,0cyclictest0-21swapper/220:50:162
28578992424,0cyclictest0-21swapper/219:10:132
28578992423,1cyclictest26285-21apt-get00:10:092
28577992424,0cyclictest0-21swapper/122:50:081
28577992424,0cyclictest0-21swapper/120:10:151
28576992424,0cyclictest0-21swapper/000:15:120
28576992423,1cyclictest626-21gdbus23:45:010
28576992423,1cyclictest141rcuc/022:35:010
28579992323,0cyclictest0-21swapper/322:55:113
28579992323,0cyclictest0-21swapper/319:25:133
28578992323,0cyclictest27415-21apt-get22:50:082
28577992323,0cyclictest29802-21apt-get22:55:111
28577992323,0cyclictest25720-21apt-get21:20:141
28577992323,0cyclictest0-21swapper/100:30:091
28577992323,0cyclictest0-21swapper/100:10:091
28577992321,1cyclictest3072-21basename19:25:011
28576992323,0cyclictest0-21swapper/023:20:130
28576992323,0cyclictest0-21swapper/021:50:080
28576992323,0cyclictest0-21swapper/021:20:140
28576992323,0cyclictest0-21swapper/020:20:020
28579992221,1cyclictest0-21swapper/319:55:113
28578992222,0cyclictest1490-21apt-get00:30:092
28578992222,0cyclictest0-21swapper/222:55:112
28578992222,0cyclictest0-21swapper/221:20:142
28578992220,1cyclictest3062-21cron19:25:012
28577992222,0cyclictest10338-21apt-get19:40:121
28577992222,0cyclictest0-21swapper/120:20:121
28576992222,0cyclictest0-21swapper/022:15:130
28579992121,0cyclictest0-21swapper/323:40:133
28579992121,0cyclictest0-21swapper/320:45:133
2857699218,11cyclictest303-21systemd-journal21:55:010
28576992121,0cyclictest0-21swapper/019:50:160
28579992020,0cyclictest29545-21perf22:55:003
28578992018,1cyclictest303-21systemd-journal23:25:012
28577992019,1cyclictest9692-21awk23:25:011
28576992020,0cyclictest8000-21apt-get19:35:140
28578991918,1cyclictest0-21swapper/221:55:102
28578991917,1cyclictest609-21dbus-daemon20:10:012
28578991917,1cyclictest0-21swapper/221:00:012
28577991919,0cyclictest0-21swapper/123:30:111
28576991919,0cyclictest0-21swapper/020:25:120
28579991818,0cyclictest0-21swapper/320:50:163
28578991818,0cyclictest0-21swapper/221:00:122
28577991817,1cyclictest24303-21idleruntime-cro20:10:011
28576991818,0cyclictest0-21swapper/023:35:120
28579991717,0cyclictest0-21swapper/300:35:153
28578991717,0cyclictest0-21swapper/220:40:132
2857799173,10cyclictest607-21irqbalance22:34:481
28577991717,0cyclictest0-21swapper/120:40:131
28577991717,0cyclictest0-21swapper/120:35:121
28576991717,0cyclictest0-21swapper/020:05:140
28576991717,0cyclictest0-21swapper/019:40:130
2857999160,15cyclictest3688-21perf21:50:013
2857899160,15cyclictest9559-21modprobe20:45:162
28577991614,1cyclictest0-21swapper/121:30:001
28576991616,0cyclictest0-21swapper/023:05:030
28576991614,1cyclictest9867-21ntp_states20:45:190
2857699160,15cyclictest3877-21fschecks_time19:25:150
2857699160,15cyclictest0-21swapper/022:55:000
2857999150,1cyclictest0-21swapper/322:35:143
2857999150,1cyclictest0-21swapper/321:35:003
2857999150,14cyclictest31747-21tail19:15:183
2857999150,14cyclictest12410-21http19:45:013
2857999150,14cyclictest0-21swapper/323:15:193
2857999150,14cyclictest0-21swapper/300:20:133
2857999150,14cyclictest0-21swapper/300:05:003
28578991514,0cyclictest18543-21apt-get23:50:082
2857899150,1cyclictest32493-21find19:15:232
2857899150,1cyclictest20500-21http21:09:592
2857899150,14cyclictest0-21swapper/221:15:122
28577991515,0cyclictest15285-21apt-get19:50:161
28577991515,0cyclictest0-21swapper/123:50:081
28577991514,1cyclictest10466-21users20:45:221
2857799150,1cyclictest32005-21ntp_states00:20:191
2857799150,1cyclictest20665-21apt-key21:10:001
2857799150,14cyclictest14244-21systemctl19:45:211
28576991513,1cyclictest3662-21awk21:50:010
2857699150,1cyclictest4033-21ntp_states23:10:130
2857699150,14cyclictest0-21swapper/023:27:490
28579991414,0cyclictest0-21swapper/322:31:043
28579991414,0cyclictest0-21swapper/322:27:593
28579991414,0cyclictest0-21swapper/320:02:413
2857999140,14cyclictest0-21swapper/323:05:323
2857999140,14cyclictest0-21swapper/322:47:413
2857999140,14cyclictest0-21swapper/322:41:053
2857999140,14cyclictest0-21swapper/322:10:033
2857999140,14cyclictest0-21swapper/322:06:153
2857999140,14cyclictest0-21swapper/322:02:103
2857999140,14cyclictest0-21swapper/321:41:173
2857999140,14cyclictest0-21swapper/321:28:283
2857999140,14cyclictest0-21swapper/321:16:403
2857999140,14cyclictest0-21swapper/321:07:143
2857999140,14cyclictest0-21swapper/320:25:053
2857999140,14cyclictest0-21swapper/320:15:143
2857999140,14cyclictest0-21swapper/319:46:193
2857999140,14cyclictest0-21swapper/319:35:153
2857999140,14cyclictest0-21swapper/319:32:253
2857999140,14cyclictest0-21swapper/319:12:083
2857999140,14cyclictest0-21swapper/300:31:343
2857999140,14cyclictest0-21swapper/300:29:123
2857999140,14cyclictest0-21swapper/300:10:173
2857999140,14cyclictest0-21swapper/300:05:353
2857999140,0cyclictest0-21swapper/323:20:153
2857999140,0cyclictest0-21swapper/321:21:573
28578991414,0cyclictest0-21swapper/219:50:162
2857899140,14cyclictest0-21swapper/223:17:282
2857899140,14cyclictest0-21swapper/223:06:582
2857899140,14cyclictest0-21swapper/222:45:342
2857899140,14cyclictest0-21swapper/222:37:132
2857899140,14cyclictest0-21swapper/222:28:132
2857899140,14cyclictest0-21swapper/222:00:002
2857899140,14cyclictest0-21swapper/221:40:402
2857899140,14cyclictest0-21swapper/221:26:392
2857899140,14cyclictest0-21swapper/220:16:202
2857899140,14cyclictest0-21swapper/219:47:152
2857899140,14cyclictest0-21swapper/219:35:412
2857899140,14cyclictest0-21swapper/219:30:532
2857899140,14cyclictest0-21swapper/219:26:012
2857899140,14cyclictest0-21swapper/200:20:542
2857899140,14cyclictest0-21swapper/200:00:162
2857899140,0cyclictest0-21swapper/223:25:412
2857899140,0cyclictest0-21swapper/221:45:462
28577991414,0cyclictest4117-21kworker/1:323:25:371
28577991414,0cyclictest3354-21apt-get19:25:131
28577991414,0cyclictest0-21swapper/119:30:181
28577991412,1cyclictest22242-21diskstats22:35:131
2857799140,14cyclictest0-21swapper/123:06:331
2857799140,14cyclictest0-21swapper/121:45:231
2857799140,14cyclictest0-21swapper/121:40:591
2857799140,14cyclictest0-21swapper/119:16:021
2857799140,14cyclictest0-21swapper/100:03:071
2857799140,0cyclictest0-21swapper/122:41:021
2857799140,0cyclictest0-21swapper/122:28:291
2857799140,0cyclictest0-21swapper/122:01:541
2857799140,0cyclictest0-21swapper/121:15:121
2857799140,0cyclictest0-21swapper/119:35:361
2857799140,0cyclictest0-21swapper/100:26:051
28576991414,0cyclictest0-21swapper/022:05:290
28576991414,0cyclictest0-21swapper/021:35:130
28576991414,0cyclictest0-21swapper/020:40:150
28576991414,0cyclictest0-21swapper/019:33:240
28576991414,0cyclictest0-21swapper/000:27:560
28576991412,1cyclictest0-21swapper/019:15:200
2857699140,14cyclictest0-21swapper/023:47:420
2857699140,14cyclictest0-21swapper/023:30:300
2857699140,14cyclictest0-21swapper/023:16:220
2857699140,14cyclictest0-21swapper/023:03:550
2857699140,14cyclictest0-21swapper/023:00:000
2857699140,14cyclictest0-21swapper/022:45:140
2857699140,14cyclictest0-21swapper/022:42:200
2857699140,14cyclictest0-21swapper/022:26:030
2857699140,14cyclictest0-21swapper/022:11:390
2857699140,14cyclictest0-21swapper/021:29:470
2857699140,14cyclictest0-21swapper/021:15:560
2857699140,14cyclictest0-21swapper/020:52:290
2857699140,14cyclictest0-21swapper/019:21:340
2857699140,14cyclictest0-21swapper/000:01:420
2857699140,0cyclictest0-21swapper/022:00:200
2857699140,0cyclictest0-21swapper/021:41:130
2857699140,0cyclictest0-21swapper/020:56:300
2857699140,0cyclictest0-21swapper/020:01:260
2857699140,0cyclictest0-21swapper/019:46:010
2857699140,0cyclictest0-21swapper/000:33:250
2857699140,0cyclictest0-21swapper/000:23:370
2857699140,0cyclictest0-21swapper/000:05:180
28579991313,0cyclictest0-21swapper/323:55:403
28579991313,0cyclictest0-21swapper/323:25:243
28579991313,0cyclictest0-21swapper/321:50:353
2857899130,0cyclictest0-21swapper/200:27:382
28577991313,0cyclictest0-21swapper/123:15:091
28577991313,0cyclictest0-21swapper/122:45:451
28577991313,0cyclictest0-21swapper/120:15:081
28576991313,0cyclictest0-21swapper/021:01:420
28576991312,1cyclictest0-21swapper/023:50:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional