You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-15 - 08:49
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Wed Jan 15, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
184345221190,1sleep10-21swapper/123:50:181
16826342960,1sleep11682639-21tr21:50:131
17103822350,1chrt1710388-21kthreadcore22:10:142
15603382330,1sleep00-21swapper/020:15:170
1475016992724,2cyclictest1613792-21cron21:00:001
1475023992623,1cyclictest1898834-21systemd-run00:35:002
1475016992422,1cyclictest1595292-21systemd-run20:45:001
1475031992320,1cyclictest0-21swapper/321:10:003
1475031992320,1cyclictest0-21swapper/320:20:003
1475031992219,1cyclictest0-21swapper/322:40:003
1475023992218,3cyclictest1890836-21awk00:30:002
1475016992219,2cyclictest1564056-21latency_hist20:20:001
1475012992219,2cyclictest277-21plymouthd20:00:000
1475012992218,2cyclictest1727727-21grep22:20:220
18354652210,1chrt0-21swapper/023:45:000
1475031992118,1cyclictest0-21swapper/320:00:003
1475031992118,1cyclictest0-21swapper/319:45:013
1475012992118,2cyclictest522-21in:imuxsock19:45:010
1475031992017,1cyclictest0-21swapper/323:05:013
1475031992017,1cyclictest0-21swapper/322:20:163
1475031992017,1cyclictest0-21swapper/321:15:013
1475031992017,1cyclictest0-21swapper/320:30:003
1475031992017,1cyclictest0-21swapper/300:40:003
1475023992017,2cyclictest0-21swapper/222:35:002
1475016992017,2cyclictest1837918-21latency_hist23:50:001
1475016992017,2cyclictest1748585-21latency_hist22:40:011
1475016992017,2cyclictest0-21swapper/120:25:001
1475012992017,2cyclictest1811750-21latency_hist23:30:010
1475031991916,1cyclictest0-21swapper/321:30:013
1475031991916,1cyclictest0-21swapper/319:25:013
1475023991916,2cyclictest501-21dbus-daemon00:10:002
1475023991916,2cyclictest1748831-21latency_hist22:40:012
1475023991916,2cyclictest1654236-21sed21:30:122
1475016991917,1cyclictest1880590-21apt-key00:20:001
1475016991917,1cyclictest1880590-21apt-key00:20:001
1475016991916,2cyclictest0-21swapper/122:15:001
1475016991915,3cyclictest3564018-21systemctl22:55:001
1475012991917,1cyclictest0-21swapper/020:25:000
1475012991916,2cyclictest1695999-21runrttasks22:00:130
1475012991914,3cyclictest1513872-21grep19:35:260
1475031991815,1cyclictest0-21swapper/323:40:153
1475031991815,1cyclictest0-21swapper/322:55:003
1475031991815,1cyclictest0-21swapper/322:34:193
1475031991815,1cyclictest0-21swapper/321:50:003
1475023991815,2cyclictest1780378-21ntp_kernel_pll_23:00:202
1475023991815,2cyclictest1631669-21ntp_kernel_pll_21:10:242
1475023991815,2cyclictest1520552-21idleruntime-cro19:45:012
1475016991815,2cyclictest0-21swapper/121:50:001
1475016991815,2cyclictest0-21swapper/119:10:011
1475012991816,1cyclictest1798548-21/usr/sbin/munin23:15:190
1475012991814,2cyclictest1782915-21latency_hist23:05:010
1475012991813,3cyclictest1619983-21cron21:05:000
1475031991714,1cyclictest0-21swapper/322:10:183
1475031991714,1cyclictest0-21swapper/321:45:013
1475023991714,2cyclictest1749451-21irqcore22:40:162
1475023991713,2cyclictest0-21swapper/223:00:002
1475023991713,2cyclictest0-21swapper/200:40:002
1475016991714,2cyclictest1812054-21fschecks_count23:30:101
1475016991714,2cyclictest1643056-21gltestperf21:20:141
1475016991714,2cyclictest0-21swapper/121:10:011
1475016991714,2cyclictest0-21swapper/119:40:011
1475016991713,2cyclictest0-21swapper/123:15:001
1475016991712,3cyclictest1582912-21apt20:35:001
1475012991715,1cyclictest0-21swapper/020:35:000
1475012991714,2cyclictest1754032-21needreboot22:40:200
1475012991714,2cyclictest1542545-21kthreadcore20:00:180
1475012991714,2cyclictest1526514-21unixbench_multi19:45:240
1475012991714,2cyclictest1502696-21irqcore19:30:190
1475012991714,2cyclictest1496177-21tr19:25:110
1475031991613,1cyclictest0-21swapper/323:35:013
1475031991613,1cyclictest0-21swapper/323:22:133
1475031991613,1cyclictest0-21swapper/323:15:223
1475031991613,1cyclictest0-21swapper/321:55:413
1475031991613,1cyclictest0-21swapper/321:20:533
1475031991613,1cyclictest0-21swapper/320:58:463
1475031991613,1cyclictest0-21swapper/320:55:003
1475031991613,1cyclictest0-21swapper/320:43:393
1475031991613,1cyclictest0-21swapper/320:36:323
1475031991613,1cyclictest0-21swapper/320:36:313
1475031991613,1cyclictest0-21swapper/320:22:523
1475031991613,1cyclictest0-21swapper/320:01:143
1475031991613,1cyclictest0-21swapper/319:54:283
1475031991613,1cyclictest0-21swapper/319:17:363
1475031991613,1cyclictest0-21swapper/319:12:383
1475031991613,1cyclictest0-21swapper/300:34:193
1475031991613,1cyclictest0-21swapper/300:15:553
1475031991613,1cyclictest0-21swapper/300:15:553
1475031991613,1cyclictest0-21swapper/300:05:003
1475031991612,1cyclictest0-21swapper/322:15:223
1475023991614,1cyclictest1722673-21irqcore22:20:142
1475023991613,2cyclictest761-21cups-browsed23:40:212
1475023991613,2cyclictest1881231-21gltestperf00:20:152
1475023991613,2cyclictest1864995-21gltestperf00:10:102
1475023991613,2cyclictest1732602-21latency_hist22:30:002
1475023991613,2cyclictest1693288-21switchtime21:55:222
1475023991613,2cyclictest1677001-21gpgv21:45:012
1475023991613,2cyclictest0-21swapper/220:00:002
1475023991612,3cyclictest1719336-21ntp_kernel_err22:15:212
1475016991613,2cyclictest7550irq/9-acpi23:57:311
1475016991613,2cyclictest524-21rs:main1
1475016991613,2cyclictest501-21dbus-daemon20:10:001
1475016991613,2cyclictest1865589-21irqrtprio00:10:121
1475016991613,2cyclictest1653485-21fschecks_count21:30:101
1475016991613,2cyclictest1554921-21kthreadcore20:10:231
1475016991613,2cyclictest1521242-21fschecks_count19:45:141
1475016991612,3cyclictest7550irq/9-acpi20:35:121
1475016991612,3cyclictest7550irq/9-acpi20:35:121
1475016991612,3cyclictest1827969-21fschecks_count23:40:131
1475016991612,3cyclictest1804562-21kthreadcore23:20:191
1475016991612,2cyclictest1481472-21latency_hist19:15:011
1475016991611,4cyclictest1520518-21cron19:45:011
1475012991613,2cyclictest1746287-21unixbench-2d22:35:260
1475012991613,2cyclictest1714578-21gltestperf22:15:150
1475012991613,2cyclictest1642356-21latency_hist21:20:010
1475012991613,2cyclictest1634175-21turbostat21:15:000
1475012991613,2cyclictest0-21swapper/019:20:010
1475012991612,3cyclictest1906817-21ls00:40:000
1475012991611,3cyclictest1873355-21irqcore00:15:160
1475012991611,3cyclictest1873355-21irqcore00:15:160
1475012991611,3cyclictest1527539-21gltestperf19:50:160
1475031991513,1cyclictest0-21swapper/323:10:453
1475031991513,1cyclictest0-21swapper/321:15:163
1475031991512,1cyclictest0-21swapper/323:50:463
1475031991512,1cyclictest0-21swapper/323:46:193
1475031991512,1cyclictest0-21swapper/323:36:413
1475031991512,1cyclictest0-21swapper/323:05:453
1475031991512,1cyclictest0-21swapper/322:56:123
1475031991512,1cyclictest0-21swapper/322:45:263
1475031991512,1cyclictest0-21swapper/322:25:173
1475031991512,1cyclictest0-21swapper/322:05:273
1475031991512,1cyclictest0-21swapper/321:50:433
1475031991512,1cyclictest0-21swapper/321:35:083
1475031991512,1cyclictest0-21swapper/321:30:143
1475031991512,1cyclictest0-21swapper/321:00:173
1475031991512,1cyclictest0-21swapper/320:31:413
1475031991512,1cyclictest0-21swapper/319:25:263
1475031991512,1cyclictest0-21swapper/300:25:203
1475031991512,1cyclictest0-21swapper/300:10:353
1475031991512,1cyclictest0-21swapper/300:05:213
1475031991511,1cyclictest0-21swapper/319:45:183
1475023991512,2cyclictest1851135-21needreboot23:55:192
1475023991512,2cyclictest1811711-21latency_hist23:30:012
1475023991512,2cyclictest1802055-21irqcore23:20:172
1475023991512,2cyclictest1788494-21kthreadcore23:10:132
1475023991512,2cyclictest1696297-21gltestperf22:00:152
1475023991512,2cyclictest1626318-21latency_hist21:10:012
1475023991512,2cyclictest1552050-21cpuspeed_turbos20:10:182
1475023991512,2cyclictest0-21swapper/200:05:002
1475023991511,3cyclictest1821170-21kernelversion23:35:162
1475023991510,4cyclictest1837483-21sshd23:49:192
1475016991513,1cyclictest1687908-21apt-get21:55:111
1475016991513,1cyclictest0-21swapper/123:20:001
1475016991512,2cyclictest1888497-21munin-plugin-st00:25:001
1475016991512,2cyclictest1754075-21ntp_kernel_err22:40:201
1475016991512,2cyclictest1719706-21timerwakeupswit22:15:251
1475016991512,2cyclictest1576143-21timerwakeupswit20:25:221
1475016991512,2cyclictest1540963-21kthreadcore20:00:181
1475016991512,2cyclictest1475002-21cyclictest22:00:161
1475016991511,2cyclictest0-21swapper/123:30:011
1475016991511,2cyclictest0-21swapper/121:10:181
147501699150,2cyclictest277-21plymouthd21:35:201
1475012991513,1cyclictest1548068-21kthreadcore20:05:180
1475012991513,1cyclictest1462524-21kworker/0:2+kacpid21:06:420
1475012991512,2cyclictest1881081-21sh00:20:130
1475012991512,2cyclictest1820632-21irqcore23:35:150
1475012991512,2cyclictest1761279-21kthreadcore22:50:120
1475012991512,2cyclictest1682638-21kthreadcore21:50:130
1475012991512,2cyclictest1462524-21kworker/0:2+kacpi_notify21:33:160
1475012991512,2cyclictest1462524-21kworker/0:2+kacpid22:29:450
1475012991511,3cyclictest1851619-21unixbench-2d23:55:240
1475012991511,2cyclictest0-21swapper/020:25:110
147501299150,4cyclictest1809179-21idleruntime-cro23:25:000
147501299150,4cyclictest1495796-21idleruntime-cro19:25:010
1475031991412,1cyclictest0-21swapper/323:25:043
1475031991412,1cyclictest0-21swapper/320:45:133
1475031991412,1cyclictest0-21swapper/320:10:073
1475031991412,1cyclictest0-21swapper/320:05:423
1475031991412,1cyclictest0-21swapper/319:30:063
1475031991412,1cyclictest0-21swapper/300:20:083
1475031991411,1cyclictest0-21swapper/323:55:023
1475031991411,1cyclictest0-21swapper/322:40:163
1475031991411,1cyclictest0-21swapper/322:00:093
1475031991411,1cyclictest0-21swapper/319:35:053
147502399148,4cyclictest1588019-21sshd20:35:192
147502399148,4cyclictest1588019-21sshd20:35:192
147502399144,4cyclictest1526969-21awk19:50:002
147502399143,2cyclictest15278172sed19:50:172
1475023991411,2cyclictest1873996-21irqrtprio00:15:172
1475023991411,2cyclictest1873996-21irqrtprio00:15:172
147502399141,12cyclictest1687577-21munin-run21:55:002
1475023991411,2cyclictest1653008-21systemd-run21:30:012
1475023991411,2cyclictest1645715-21kthreadcore21:20:172
1475023991411,2cyclictest1635129-21gltestperf21:15:132
1475023991411,2cyclictest1508515-21cpuspeed_turbos19:35:152
1475023991411,2cyclictest1489581-21sendmail19:20:012
1475023991411,2cyclictest0-21swapper/220:25:002
1475023991411,1cyclictest0-21swapper/222:10:002
1475023991410,3cyclictest1799130-21unixbench_multi23:15:242
1475023991410,3cyclictest1551208-21unixbench-2d20:05:242
147501699149,4cyclictest1501652-21sshd19:26:181
147501699149,1cyclictest191rcu_preempt21:00:181
147501699148,4cyclictest1532971-21sshd19:53:191
147501699147,5cyclictest3561480-21systemd-journal20:55:001
147501699143,2cyclictest581rcuop/319:25:011
1475016991412,1cyclictest0-21swapper/123:00:221
1475016991411,2cyclictest1899430-21cut00:35:161
1475016991411,2cyclictest1825427-21unixbench_singl23:35:241
1475016991411,2cyclictest1770274-21kthreadcore22:55:171
1475016991411,2cyclictest1732881-21tr22:30:081
1475016991411,2cyclictest1635251-21grep21:15:141
1475016991411,2cyclictest1481747-21gltestperf19:15:011
147501299149,2cyclictest471rcuog/223:50:000
147501299148,1cyclictest191rcu_preempt20:55:170
1475012991412,1cyclictest1853716-21kworker/0:0+kacpid00:13:460
1475012991412,1cyclictest1733119-21gltestperf22:30:100
1475012991412,1cyclictest1643195-21tr21:20:150
1475012991412,1cyclictest1462524-21kworker/0:2+kacpid21:48:110
1475012991412,1cyclictest0-21swapper/000:30:100
1475012991411,2cyclictest1872777-21kworker/0:2+kacpid00:29:070
1475012991411,2cyclictest1857064-21kthreadcore00:00:180
1475012991411,2cyclictest1772568-21systemctl22:55:210
1475012991411,2cyclictest1706192-21/usr/sbin/munin22:10:120
1475012991411,2cyclictest1688205-21gltestperf21:55:130
1475012991411,2cyclictest1669661-21/usr/sbin/munin21:40:140
1475012991411,2cyclictest1596909-21kthreadcore20:45:160
1475012991411,2cyclictest1584089-21irqrtprio20:35:160
1475012991411,2cyclictest1584089-21irqrtprio20:35:160
1475012991411,2cyclictest1554920-21kthreadcore20:10:230
1475012991411,1cyclictest1462524-21kworker/0:2+kacpid23:50:140
1475012991410,3cyclictest1475624-21fschecks_time19:10:140
147502399139,3cyclictest1761369-21kthreadcore22:50:122
147502399138,4cyclictest1571051-21fw_packets20:25:132
147502399137,4cyclictest547-21gdbus20:35:002
147502399137,4cyclictest1-21systemd21:00:012
1475023991310,2cyclictest1842088-21kthreadcore23:50:132
1475023991310,2cyclictest1545567-21latency_hist20:05:002
1475023991310,2cyclictest0-21swapper/219:30:182
1475023991310,1cyclictest0-21swapper/219:15:012
147502399130,13cyclictest0-21swapper/221:35:232
147501699139,2cyclictest0-21swapper/119:55:251
1475016991311,1cyclictest1722477-21fschecks_time22:20:131
1475016991311,1cyclictest1626198-21diskmemload22:05:441
1475016991311,1cyclictest1601427-21munin-plugin-st20:50:001
1475016991311,1cyclictest0-21swapper/119:30:181
1475016991310,2cyclictest515-21wpa_supplicant22:26:301
1475016991310,2cyclictest34-21ksoftirqd/100:34:591
1475016991310,2cyclictest1858517-21kthreadcore00:00:181
1475016991310,2cyclictest1674825-21unixbench-2d21:40:231
147501699130,2cyclictest1862596-21runrttasks00:06:281
147501299139,3cyclictest1462524-21kworker/0:2+kacpi_notify22:08:170
147501299139,3cyclictest1462524-21kworker/0:2+kacpid21:25:390
147501299139,2cyclictest0-21swapper/020:50:260
147501299139,2cyclictest0-21swapper/020:40:160
1475012991312,1cyclictest1462524-21kworker/0:2+kacpid23:30:190
1475012991310,2cyclictest1462524-21kworker/0:2+kacpi_notify22:45:300
1475012991310,2cyclictest1462524-21kworker/0:2+kacpi_notify21:35:460
1475012991310,2cyclictest0-21swapper/023:10:160
1475012991310,2cyclictest0-21swapper/000:10:000
147502399129,2cyclictest1601003-21sed20:45:232
147502399129,2cyclictest1594667-21timerandwakeup20:40:242
147502399129,2cyclictest0-21swapper/223:30:112
147502399129,2cyclictest0-21swapper/221:05:002
147502399129,2cyclictest0-21swapper/219:20:162
147502399126,4cyclictest1-21systemd23:10:002
147501299129,2cyclictest1462524-21kworker/0:2+kacpi_notify23:05:390
147503199119,1cyclictest0-21swapper/319:10:013
147502399117,3cyclictest1-21systemd21:46:162
147502399116,4cyclictest1501190-21systemctl19:25:202
147501699110,4cyclictest175891950ssh22:49:541
147501699110,1cyclictest0-21swapper/123:05:511
147501699110,10cyclictest0-21swapper/121:27:521
14747592111,4sleep069-21kcompactd019:09:530
147502399107,2cyclictest1563601-21unixbench_multi20:15:232
147502399107,2cyclictest0-21swapper/222:50:012
147502399107,2cyclictest0-21swapper/220:50:192
147502399107,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional