You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8.osadl.org (updated Thu Apr 03, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
320812285251,22sleep10-21swapper/119:08:541
299652277241,22sleep30-21swapper/319:05:013
319202273240,20sleep00-21swapper/019:06:520
320872272237,22sleep20-21swapper/219:08:592
203362590,0sleep10-21swapper/121:05:171
164472580,0sleep20-21swapper/223:51:132
290492570,1sleep20-21swapper/200:28:132
254242550,0sleep20-21swapper/221:59:162
195662550,0sleep00-21swapper/022:41:110
8742540,1sleep20-21swapper/200:03:452
67302530,0sleep00-21swapper/000:09:130
225592500,0sleep10-21swapper/121:56:441
100182210,0sleep10-21swapper/121:48:311
229062180,0sleep30-21swapper/323:08:563
216082140,0sleep00-21swapper/023:55:150
3228499105,3cyclictest4398-21awk00:35:011
322909994,3cyclictest19982-21systemd-journal22:43:593
322909993,4cyclictest7661-21lldpd23:56:533
322849994,3cyclictest508-21dbus-daemon00:28:421
322849992,4cyclictest12886-21cron23:25:001
322839994,3cyclictest19982-21systemd-journal23:43:120
322839993,5cyclictest0-21swapper/020:52:350
16685290,0sleep20-21swapper/223:28:182
322909985,2cyclictest508-21dbus-daemon00:26:113
322909984,3cyclictest508-21dbus-daemon19:48:443
322909983,3cyclictest19982-21systemd-journal21:30:013
322909983,3cyclictest19982-21systemd-journal19:40:483
322869986,1cyclictest13238-21ssh23:49:522
322869984,3cyclictest508-21dbus-daemon23:19:012
322869984,3cyclictest19982-21systemd-journal20:56:192
322869983,3cyclictest19982-21systemd-journal23:03:342
322869983,3cyclictest19982-21systemd-journal23:03:342
322869983,3cyclictest19982-21systemd-journal20:47:582
322869983,3cyclictest19982-21systemd-journal00:38:432
322849981,4cyclictest17208-21munin-run21:00:011
322839982,3cyclictest9272-21turbostat19:30:020
322839981,4cyclictest28569-21turbostat22:25:000
322909975,2cyclictest34-21ksoftirqd/323:15:003
322909974,2cyclictest7893-21if_enp4s022:10:113
322909974,2cyclictest508-21dbus-daemon21:15:313
322909974,2cyclictest508-21dbus-daemon00:00:203
322909974,2cyclictest30654-21snmpd22:49:483
322909974,2cyclictest11736-21in:imuxsock21:20:573
322909973,3cyclictest30623-21ntpq20:15:183
322909973,3cyclictest20323-21ntpq21:55:163
322909973,3cyclictest16530-21cut21:30:013
322909973,3cyclictest10307-21cron00:40:013
322909973,3cyclictest0-21swapper/322:20:073
322909973,1cyclictest0-21swapper/323:31:493
322909973,1cyclictest0-21swapper/323:25:103
322909973,1cyclictest0-21swapper/322:55:423
322909973,1cyclictest0-21swapper/322:17:433
322909973,1cyclictest0-21swapper/300:08:163
322909972,3cyclictest19982-21systemd-journal20:00:343
322869976,1cyclictest26777-21ssh22:47:252
322869974,2cyclictest29741-11brltty22:10:112
322869974,2cyclictest28-21ksoftirqd/219:55:002
322869973,3cyclictest508-21dbus-daemon21:20:572
322869973,3cyclictest508-21dbus-daemon20:00:002
322869973,3cyclictest30654-21snmpd22:28:382
322869973,3cyclictest30654-21snmpd19:25:012
322869973,3cyclictest19982-21systemd-journal19:17:032
322869973,1cyclictest0-21swapper/223:37:202
322869973,1cyclictest0-21swapper/222:43:002
322869973,1cyclictest0-21swapper/222:20:362
322869973,1cyclictest0-21swapper/222:06:552
322869973,1cyclictest0-21swapper/221:41:482
322869973,1cyclictest0-21swapper/221:16:232
322869973,1cyclictest0-21swapper/200:31:082
322869972,3cyclictest19982-21systemd-journal22:38:322
322869971,4cyclictest19982-21systemd-journal19:30:232
322849974,2cyclictest508-21dbus-daemon20:50:231
322849973,3cyclictest30654-21snmpd20:38:181
322849973,3cyclictest19982-21systemd-journal22:28:361
322849973,2cyclictest20917-21ssh22:42:261
322849973,1cyclictest0-21swapper/123:26:481
322849973,1cyclictest0-21swapper/123:05:031
322849973,1cyclictest0-21swapper/100:36:471
322849973,1cyclictest0-21swapper/100:17:131
322849972,3cyclictest19982-21systemd-journal21:44:441
322849972,3cyclictest19982-21systemd-journal20:10:021
322849971,3cyclictest31133-21munin-run23:15:011
322839975,1cyclictest18096-21ls19:50:010
322839974,2cyclictest508-21dbus-daemon20:22:570
322839974,2cyclictest508-21dbus-daemon20:05:200
322839973,3cyclictest30654-21snmpd20:26:180
322839973,3cyclictest30654-21snmpd00:29:140
322839973,3cyclictest19982-21systemd-journal23:49:590
322839973,3cyclictest14506-21grep19:40:150
322839973,3cyclictest11362-21grep20:45:160
322839973,3cyclictest0-21swapper/023:07:080
322839973,3cyclictest0-21swapper/021:19:100
322839972,3cyclictest19982-21systemd-journal21:37:130
322839972,1cyclictest0-21swapper/021:52:030
322839971,3cyclictest7311-21rm21:21:500
322839971,3cyclictest30219-21rm23:37:310
322839971,3cyclictest29430-21munin-run20:15:000
322839971,3cyclictest22502-21rm00:20:350
23625270,0sleep0131rcuc/023:33:040
322909964,1cyclictest31331-21smart_sda22:50:153
322909963,2cyclictest771-21users23:15:183
322909963,2cyclictest69112sleep319:23:183
322909963,2cyclictest5682-21sh22:31:503
322909963,2cyclictest30654-21snmpd20:52:223
322909963,2cyclictest25733-21tune2fs20:05:153
322909963,2cyclictest20216-21users19:50:233
322909963,2cyclictest18128-21rm23:04:293
322909963,2cyclictest18128-21rm23:04:293
322909963,2cyclictest1-21systemd21:05:203
322909963,2cyclictest0-21swapper/300:30:093
322909963,1cyclictest0-21swapper/321:50:073
322909962,3cyclictest9781-21cat22:35:133
322909962,3cyclictest9012-21cat20:40:143
322909962,3cyclictest7456-21cat20:35:213
322909962,3cyclictest6719-21fschecks_time23:20:143
322909962,3cyclictest31994-21fschecks_count21:40:163
322909962,3cyclictest30654-21snmpd21:10:173
322909962,3cyclictest30654-21snmpd20:55:583
322909962,3cyclictest27820-21expr20:10:153
322909962,3cyclictest19982-21systemd-journal23:38:383
322909962,3cyclictest19982-21systemd-journal20:25:283
322909962,3cyclictest18055-21tail21:00:133
322909962,3cyclictest14313-21fw_packets23:50:103
322909962,3cyclictest1293-21expr23:40:143
322909962,3cyclictest12574-21cat19:35:173
322909962,3cyclictest10221-21uname19:30:193
322909962,3cyclictest1011-21cat19:10:193
322909962,2cyclictest19982-21systemd-journal20:34:133
322909962,1cyclictest0-21swapper/322:26:153
322909962,1cyclictest0-21swapper/322:05:573
322909962,1cyclictest0-21swapper/321:48:123
322909962,1cyclictest0-21swapper/300:17:043
322909961,3cyclictest502-21systemd-logind21:35:333
322909961,3cyclictest19982-21systemd-journal19:56:153
322909961,3cyclictest19982-21systemd-journal19:25:233
322909961,3cyclictest19982-21systemd-journal00:24:313
322909960,6cyclictest0-21swapper/323:45:313
322909960,5cyclictest3063-21cpuspeed_turbos19:15:173
322909960,2cyclictest27384-21gltestperf22:00:193
322909960,1cyclictest29741-11brltty00:12:003
322869963,2cyclictest8405-21sh23:20:502
322869963,2cyclictest668-21runrttasks21:10:472
322869963,2cyclictest29678-21ssh22:01:292
322869963,2cyclictest2812-21sh00:05:202
322869963,2cyclictest1-21systemd21:50:592
322869963,1cyclictest19982-21systemd-journal20:35:022
322869962,3cyclictest7661-21lldpd22:56:202
322869962,3cyclictest30654-21snmpd23:40:252
322869962,3cyclictest30654-21snmpd21:25:082
322869962,3cyclictest30654-21snmpd20:27:222
322869962,3cyclictest30654-21snmpd20:20:072
322869962,3cyclictest30654-21snmpd20:17:312
322869962,3cyclictest30654-21snmpd19:48:022
322869962,3cyclictest30654-21snmpd19:43:412
322869962,3cyclictest30654-21snmpd19:35:252
322869962,3cyclictest30654-21snmpd00:19:292
322869962,3cyclictest2879-21smartctl22:30:122
322869962,3cyclictest25969-21cat20:05:172
322869962,3cyclictest23357-21cat20:00:132
322869962,3cyclictest21992-21memory23:55:162
322869962,3cyclictest17859-21diskstats21:00:132
322869962,3cyclictest0-21swapper/223:30:052
322869962,3cyclictest0-21swapper/221:35:342
322869962,3cyclictest0-21swapper/221:30:242
322869962,1cyclictest0-21swapper/223:10:132
322869962,1cyclictest0-21swapper/223:05:372
322869962,1cyclictest0-21swapper/222:52:302
322869962,1cyclictest0-21swapper/222:15:522
322869962,1cyclictest0-21swapper/221:48:392
322869962,1cyclictest0-21swapper/200:20:152
322869961,3cyclictest502-21systemd-logind19:14:072
322869960,6cyclictest0-21swapper/220:30:252
322869960,6cyclictest0-21swapper/200:10:102
322869960,5cyclictest10589-21sh20:45:002
322849966,0cyclictest0-21swapper/100:21:551
322849965,1cyclictest22-21ksoftirqd/121:30:101
322849963,2cyclictest9916-21sh19:30:161
322849963,2cyclictest6743-21chrt00:09:141
322849963,2cyclictest4481-21grep20:30:171
322849963,2cyclictest3890-21sh23:18:211
322849963,2cyclictest30654-21snmpd22:16:101
322849963,2cyclictest29849-21sh22:50:021
322849963,2cyclictest14979-21pmu-power19:40:191
322849963,2cyclictest13468-21sh22:37:181
322849963,2cyclictest1-21systemd21:17:101
322849963,2cyclictest11736-21in:imuxsock20:46:441
322849963,2cyclictest11343-21ssh23:48:031
322849963,2cyclictest10216-21ssh21:24:331
322849963,2cyclictest10198-21ssh00:10:511
322849962,3cyclictest8136-21date22:10:131
322849962,3cyclictest730-21cat19:10:161
322849962,3cyclictest3390-21tune2fs19:15:171
322849962,3cyclictest30654-21snmpd22:56:321
322849962,3cyclictest30654-21snmpd22:45:421
322849962,3cyclictest30654-21snmpd22:31:521
322849962,3cyclictest30654-21snmpd20:15:151
322849962,3cyclictest30654-21snmpd19:54:571
322849962,3cyclictest30654-21snmpd19:26:551
322849962,3cyclictest21535-21cat19:55:141
322849962,3cyclictest19982-21systemd-journal19:22:271
322849962,3cyclictest17101-21ntpq19:45:191
322849962,3cyclictest13200-21cat23:00:201
322849962,3cyclictest13200-21cat23:00:201
322849962,3cyclictest12777-21awk19:35:201
322849962,1cyclictest0-21swapper/123:36:031
322849962,1cyclictest0-21swapper/123:30:051
322849962,1cyclictest0-21swapper/122:05:081
322849962,1cyclictest0-21swapper/122:00:331
322849962,1cyclictest0-21swapper/121:10:091
322849962,1cyclictest0-21swapper/100:00:231
322849961,4cyclictest415-21grep20:20:171
322849961,4cyclictest0-21swapper/123:42:281
322849961,4cyclictest0-21swapper/122:20:051
322849961,4cyclictest0-21swapper/121:27:311
322849961,3cyclictest502-21systemd-logind20:06:161
322849960,6cyclictest0-21swapper/121:37:281
322849960,5cyclictest20874-21gdbus23:55:001
322849960,5cyclictest12741-21if_enp4s021:50:121
322849960,2cyclictest15077-21ssh23:50:131
322839966,0cyclictest0-21swapper/023:15:090
322839963,2cyclictest31792-21sh00:02:180
322839963,2cyclictest30654-21snmpd23:26:360
322839963,2cyclictest30654-21snmpd22:16:010
322839963,2cyclictest3017-21latency22:30:130
322839963,2cyclictest12132sleep021:40:460
322839963,2cyclictest10498-21taskset20:43:590
322839963,1cyclictest21621-21diskmemload21:45:340
322839962,3cyclictest5353-21expr19:20:140
322839962,3cyclictest508-21dbus-daemon22:10:350
322839962,3cyclictest508-21dbus-daemon21:10:550
322839962,3cyclictest30654-21snmpd23:01:270
322839962,3cyclictest30654-21snmpd23:01:270
322839962,3cyclictest30654-21snmpd22:25:100
322839962,3cyclictest30654-21snmpd21:55:190
322839962,3cyclictest30654-21snmpd20:35:190
322839962,3cyclictest30654-21snmpd20:32:220
322839962,3cyclictest30654-21snmpd19:18:110
322839962,3cyclictest30654-21snmpd00:35:230
322839962,3cyclictest30654-21snmpd00:10:020
322839962,3cyclictest27088-21expr22:00:180
322839962,3cyclictest24614-21cat23:10:090
322839962,3cyclictest22064-21smart_sda19:55:180
322839962,3cyclictest17408-21grep21:30:140
322839962,3cyclictest0-21swapper/022:47:590
322839962,3cyclictest0-21swapper/000:15:050
322839961,4cyclictest21008-21grep21:05:210
322839961,3cyclictest19982-21systemd-journal22:35:130
322839961,3cyclictest19982-21systemd-journal20:55:050
322839961,2cyclictest5364-21kworker/u8:122:56:510
322839960,6cyclictest0-21swapper/023:50:010
322839960,6cyclictest0-21swapper/023:20:050
322839960,5cyclictest31668-21idleruntime-cro20:20:000
322839960,3cyclictest1885-21sh22:05:310
322839960,3cyclictest12682-21rm21:26:340
322839960,2cyclictest29974-21/usr/sbin/munin22:50:120
322909951,3cyclictest32581-21basename20:20:163
322909951,2cyclictest19982-21systemd-journal20:45:033
322869952,3cyclictest1-21systemd19:20:192
322869952,2cyclictest28544-21proc_pri20:10:222
322869950,3cyclictest19982-21systemd-journal21:05:052
322869950,3cyclictest19982-21systemd-journal20:50:032
322849953,1cyclictest22-21ksoftirqd/120:40:001
322849952,2cyclictest30654-21snmpd20:25:061
322849952,2cyclictest23262-21grep20:00:131
322849952,2cyclictest18131-21cat21:00:151
322839953,1cyclictest18674-21grep19:50:160
322839952,2cyclictest9565-21cpu19:30:160
322839952,2cyclictest30654-21snmpd19:10:100
322839952,2cyclictest23760-21cat20:00:150
322839952,2cyclictest17783-21expr21:00:130
322839952,2cyclictest11775-21cpu19:35:140
322839950,4cyclictest30917-21ssh00:30:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional