You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-04 - 06:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Fri Apr 04, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
177552285257,17sleep20-21swapper/219:09:172
176842263233,18sleep30-21swapper/319:08:223
175362261227,22sleep00-21swapper/019:06:270
175122254221,22sleep10-21swapper/119:06:081
232812630,0sleep10-21swapper/100:20:171
316622600,0sleep00-21swapper/000:28:120
8412570,0sleep10-21swapper/122:44:281
277122570,0sleep00-21swapper/022:40:130
287502560,0sleep20-21swapper/222:40:202
177362530,0sleep00-21swapper/023:50:120
64212210,0sleep30-21swapper/321:33:083
243502180,0sleep00-21swapper/023:29:590
80112170,0sleep00-21swapper/000:07:220
294442160,0sleep30-21swapper/323:07:003
1791499163,3cyclictest765-21cron21:40:003
106032160,0sleep20-21swapper/223:43:462
157592150,0sleep20-21swapper/221:40:142
1791499148,4cyclictest13415-21apt-get20:10:013
1791499129,2cyclictest2014-21apt-get22:45:023
17914991210,1cyclictest27503-21apt-get22:40:123
17912991210,2cyclictest28-21ksoftirqd/221:40:012
1791499119,1cyclictest1514-21apt-get20:55:133
1791499118,2cyclictest346-21apt-get23:10:083
1791499117,3cyclictest22629-21apt-get00:20:153
1791499117,3cyclictest20664-21ssh23:00:143
1791499117,3cyclictest20394-21apt-get20:25:123
1791499117,3cyclictest19982-21systemd-journal23:30:003
1791499113,3cyclictest508-21dbus-daemon00:12:273
1791499112,4cyclictest26834-21ssh22:39:513
1791499107,2cyclictest20970-21apt-get19:15:113
1791499106,3cyclictest27916-21taskset19:30:163
1791499105,3cyclictest19982-21systemd-journal23:31:073
1791499104,5cyclictest27145-21/usr/sbin/munin22:15:143
1791499103,3cyclictest11208-21rm22:28:093
1791499103,2cyclictest18301-21expr19:10:133
1791499102,6cyclictest27679-21ssh23:56:413
1791499102,5cyclictest4168-21ssh21:56:003
1791499102,5cyclictest29204-21ssh21:50:183
1791499102,5cyclictest26017-21ssh22:14:213
1791499102,5cyclictest11841-21ssh22:02:133
1791499102,5cyclictest10386-21ssh22:51:473
1791499102,4cyclictest4157-21ssh00:31:433
1791499102,4cyclictest10072-21rm21:10:263
1791499102,3cyclictest319-21pmu-power00:00:193
1791499101,6cyclictest8636-21ssh23:41:403
1791499101,6cyclictest30606-21ssh21:26:553
1790399103,4cyclictest3459-21sh21:00:010
179149996,2cyclictest25393-21apt-get19:25:123
179149996,2cyclictest22036-21apt-get21:20:103
179149996,2cyclictest19982-21systemd-journal20:40:013
179149996,2cyclictest1-21systemd23:20:183
179149995,3cyclictest8052-21grep00:35:113
179149995,3cyclictest7576-21smartctl23:15:183
179149995,3cyclictest4076-21awk19:50:013
179149995,3cyclictest29278-21awk20:45:013
179149995,3cyclictest24059-21awk23:55:003
179149995,3cyclictest16859-21smartctl20:15:193
179149994,4cyclictest1904-21ssh23:36:273
179149994,4cyclictest18702-21rm00:15:393
179149994,3cyclictest19982-21systemd-journal19:57:323
179149993,2cyclictest29741-11brltty20:53:013
179149993,2cyclictest24286-21wc20:30:233
179149993,1cyclictest7892-21diskmemload00:27:483
179149992,5cyclictest19351-21ssh21:42:273
179149992,5cyclictest15600-21ssh22:05:553
179149992,3cyclictest7892-21diskmemload22:22:203
179149992,3cyclictest7892-21diskmemload00:07:283
179149991,6cyclictest15687-21ssh23:48:513
179149991,5cyclictest6053-21taskset19:52:303
179149991,5cyclictest22380-21ssh21:45:163
179149991,5cyclictest20543-21ssh21:18:533
179149991,5cyclictest15718-21ssh22:30:443
179149991,4cyclictest18389-21ssh22:58:113
179129995,2cyclictest19982-21systemd-journal19:42:052
1762290,5sleep20-21swapper/223:10:142
17058290,0sleep00-21swapper/023:22:340
179149987,0cyclictest17881-21apt-get20:20:133
179149985,2cyclictest1122-21smartctl19:40:203
179149985,2cyclictest10688-21apt20:05:003
179149984,3cyclictest6135-21cstates21:05:133
179149984,3cyclictest4247-21expr21:00:123
179149984,3cyclictest30681-21gltestperf19:35:183
179149984,3cyclictest30305-21latency20:45:173
179149984,3cyclictest23600-21sort19:20:133
179149984,3cyclictest12215-21cat20:05:183
179129984,3cyclictest7661-21lldpd20:01:422
179129983,3cyclictest19982-21systemd-journal19:55:262
179129980,3cyclictest19982-21systemd-journal21:30:002
179069985,2cyclictest30654-21snmpd23:25:091
179069983,3cyclictest7661-21lldpd19:37:561
179069983,3cyclictest22331-21apt-get20:30:011
179069983,3cyclictest19982-21systemd-journal20:31:361
179069980,7cyclictest0-21swapper/123:39:401
179039987,1cyclictest121ksoftirqd/019:45:000
179039984,3cyclictest7210-21cron00:35:010
179039984,3cyclictest508-21dbus-daemon19:15:480
179039984,1cyclictest141rcu_preempt00:00:000
179039983,3cyclictest19982-21systemd-journal22:11:530
179129975,1cyclictest28-21ksoftirqd/200:00:012
179129974,2cyclictest11736-21in:imuxsock21:09:162
179129973,3cyclictest8893-21grep22:00:152
179129973,3cyclictest508-21dbus-daemon20:14:372
179129973,3cyclictest508-21dbus-daemon19:55:002
179129973,3cyclictest30654-21snmpd00:26:192
179129973,3cyclictest19982-21systemd-journal20:21:432
179129973,1cyclictest0-21swapper/223:45:192
179129973,1cyclictest0-21swapper/222:08:322
179129973,1cyclictest0-21swapper/221:55:582
179129971,3cyclictest1452-21munin-run21:55:002
179069975,1cyclictest14902-21missed_timers22:55:161
179069974,2cyclictest508-21dbus-daemon19:46:151
179069974,2cyclictest17328-21rm23:22:491
179069974,2cyclictest11736-21in:imuxsock23:41:341
179069973,3cyclictest502-21systemd-logind22:07:411
179069973,3cyclictest19982-21systemd-journal19:16:131
179069973,3cyclictest1878-21cat00:30:101
179069973,3cyclictest0-21swapper/123:58:371
179069973,1cyclictest20078-21idleruntime-cro19:15:011
179069973,1cyclictest0-21swapper/122:28:431
179069972,3cyclictest19982-21systemd-journal21:35:581
179069972,3cyclictest19982-21systemd-journal20:11:431
179069972,3cyclictest19982-21systemd-journal19:26:341
179069971,4cyclictest19982-21systemd-journal22:54:251
179069971,3cyclictest31489-21basename20:50:001
179069971,3cyclictest16884-21munin-run23:50:011
179039975,1cyclictest0-21swapper/022:35:130
179039974,2cyclictest19982-21systemd-journal19:37:290
179039973,3cyclictest30654-21snmpd23:35:190
179039973,3cyclictest19982-21systemd-journal22:20:010
179039973,3cyclictest0-21swapper/021:59:010
179039973,1cyclictest0-21swapper/023:42:060
179039973,1cyclictest0-21swapper/022:22:140
179039973,1cyclictest0-21swapper/021:32:080
179039972,3cyclictest7661-21lldpd23:07:020
179129966,0cyclictest0-21swapper/221:00:132
179129963,2cyclictest9955-21ssh22:51:242
179129963,2cyclictest9597-21sh23:16:282
179129963,2cyclictest30654-21snmpd21:45:182
179129963,2cyclictest30654-21snmpd00:11:412
179129963,2cyclictest23130-21ssh22:36:002
179129963,2cyclictest20601-21expr22:10:122
179129963,2cyclictest13562-21apt-get22:30:082
179129962,3cyclictest9185-21latency21:10:182
179129962,3cyclictest7661-21lldpd20:35:132
179129962,3cyclictest3605-21awk19:45:202
179129962,3cyclictest32697-21ntpq23:35:202
179129962,3cyclictest30654-21snmpd20:41:252
179129962,3cyclictest30654-21snmpd20:27:012
179129962,3cyclictest30654-21snmpd19:36:522
179129962,3cyclictest30654-21snmpd19:28:112
179129962,3cyclictest29872-21expr20:45:142
179129962,3cyclictest23657-21ntpq00:20:192
179129962,3cyclictest22487-21tune2fs21:20:132
179129962,3cyclictest18335-21cat19:10:142
179129962,3cyclictest17467-21cat00:15:142
179129962,3cyclictest15015-21ntpq22:55:182
179129962,3cyclictest1-21systemd20:55:052
179129962,3cyclictest11736-21in:imuxsock20:05:512
179129962,3cyclictest0-21swapper/221:15:552
179129962,2cyclictest19982-21systemd-journal20:34:142
179129962,1cyclictest0-21swapper/223:50:402
179129962,1cyclictest0-21swapper/223:06:052
179129962,1cyclictest0-21swapper/222:47:002
179129962,1cyclictest0-21swapper/222:27:372
179129962,1cyclictest0-21swapper/222:20:532
179129962,1cyclictest0-21swapper/200:06:282
179129961,3cyclictest19982-21systemd-journal23:01:312
179129961,3cyclictest19982-21systemd-journal00:30:222
179129961,2cyclictest502-21systemd-logind00:00:012
179129961,2cyclictest15602-21ssh23:21:042
179129961,2cyclictest11478-21ssh00:37:462
179129960,6cyclictest0-21swapper/222:15:022
179129960,3cyclictest25520-21sh23:30:112
179129960,0cyclictest0-21swapper/223:25:172
179069965,1cyclictest7892-21diskmemload21:48:161
179069964,1cyclictest15627-21/usr/sbin/munin20:15:191
179069963,2cyclictest883-21head00:00:231
179069963,2cyclictest4006-21sh21:30:441
179069963,2cyclictest30654-21snmpd19:56:411
179069963,2cyclictest29278-21sh21:25:421
179069963,2cyclictest24728-21ssh21:21:251
179069963,2cyclictest2049-21uniq22:20:171
179069963,2cyclictest16515-21rm21:40:161
179069963,2cyclictest0-21swapper/100:15:301
179069962,3cyclictest9111-21cat21:10:191
179069962,3cyclictest4982-21cat19:50:171
179069962,3cyclictest4226-21cut21:00:131
179069962,3cyclictest30654-21snmpd23:04:411
179069962,3cyclictest30654-21snmpd22:35:381
179069962,3cyclictest30654-21snmpd22:00:551
179069962,3cyclictest30654-21snmpd21:56:331
179069962,3cyclictest30654-21snmpd21:05:201
179069962,3cyclictest30654-21snmpd20:37:091
179069962,3cyclictest30654-21snmpd20:06:561
179069962,3cyclictest30654-21snmpd00:10:011
179069962,3cyclictest28201-21cat21:50:101
179069962,3cyclictest19116-21gdbus19:25:011
179069962,3cyclictest17870-21fw_packets23:50:141
179069962,3cyclictest15850-21which21:15:151
179069962,1cyclictest0-21swapper/122:45:221
179069962,1cyclictest0-21swapper/122:30:341
179069962,1cyclictest0-21swapper/122:16:221
179069962,1cyclictest0-21swapper/100:26:311
179069962,1cyclictest0-21swapper/100:06:111
179069961,4cyclictest0-21swapper/122:10:021
179069961,2cyclictest2493-21ssh23:10:411
179069960,6cyclictest0-21swapper/123:05:011
179069960,6cyclictest0-21swapper/100:39:331
179069960,5cyclictest1347-21cat20:55:001
179069960,2cyclictest7661-21lldpd23:15:161
179039964,1cyclictest6904-21df_inode19:55:130
179039963,2cyclictest51950irq/137-enp4s0-21:40:200
179039963,2cyclictest508-21dbus-daemon19:45:200
179039963,2cyclictest508-21dbus-daemon00:16:360
179039963,2cyclictest24651-21sh23:30:040
179039963,2cyclictest24255-21tail00:20:220
179039963,2cyclictest23462-21sh23:01:260
179039963,2cyclictest21909-21unixbench_singl20:25:210
179039963,2cyclictest19982-21systemd-journal21:25:190
179039963,2cyclictest17669-21sh22:32:430
179039963,2cyclictest11736-21in:imuxsock21:06:220
179039963,2cyclictest10602-21seq22:00:540
179039963,1cyclictest0-21swapper/022:56:290
179039962,3cyclictest861-21fschecks_time23:10:090
179039962,3cyclictest4935-21cat19:50:180
179039962,3cyclictest30654-21snmpd22:28:350
179039962,3cyclictest30654-21snmpd22:05:320
179039962,3cyclictest30654-21snmpd21:03:560
179039962,3cyclictest30654-21snmpd20:35:120
179039962,3cyclictest30654-21snmpd20:23:390
179039962,3cyclictest30654-21snmpd20:16:500
179039962,3cyclictest30654-21snmpd19:21:460
179039962,3cyclictest28685-21cat19:30:190
179039962,3cyclictest28341-21expr21:50:110
179039962,3cyclictest26076-21cat19:25:160
179039962,3cyclictest16541-21awk21:15:180
179039962,3cyclictest1-21systemd21:20:070
179039962,3cyclictest1-21systemd19:12:270
179039962,3cyclictest10655-21cat00:10:130
179039962,1cyclictest0-21swapper/023:45:190
179039962,1cyclictest0-21swapper/022:46:390
179039962,1cyclictest0-21swapper/021:11:280
179039962,1cyclictest0-21swapper/000:02:360
179039961,3cyclictest502-21systemd-logind20:05:020
179039961,2cyclictest12321-21ssh21:37:130
179039960,6cyclictest0-21swapper/021:46:370
179039960,5cyclictest14257-21load20:10:230
179039960,4cyclictest19982-21systemd-journal23:15:340
179039960,3cyclictest8193-21cut22:50:130
179039960,0cyclictest0-21swapper/000:36:070
179129952,2cyclictest27887-21taskset19:30:152
179129952,2cyclictest23695-21/usr/sbin/munin19:20:142
179129952,2cyclictest20756-21fschecks_count19:15:022
179129952,2cyclictest16326-21cat20:15:142
179129952,2cyclictest1-21systemd21:30:082
179129950,3cyclictest19982-21systemd-journal20:50:042
179069953,1cyclictest7892-21diskmemload23:30:091
179069953,1cyclictest18298-21cut20:20:161
179069952,2cyclictest9550-21load20:00:201
179069952,2cyclictest32466-21tr19:40:131
179069952,2cyclictest28064-21tr19:30:161
179069952,2cyclictest27427-21grep20:40:121
179069952,2cyclictest1895-21df20:55:141
179039952,2cyclictest9135-21sh20:00:180
179039952,2cyclictest32425-21cat20:50:170
179039952,2cyclictest22819-21cat20:30:000
179039951,3cyclictest30212-21cat20:45:160
179039950,4cyclictest27326-21apt-get20:40:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional