You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-13 - 02:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  Congatec
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Wed Mar 12, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
170123899748738,5cyclictest1707330-21kworker/3:0+events09:29:053
170123899746743,2cyclictest1707330-21kworker/3:0+events11:49:213
170123899746737,7cyclictest1707330-21kworker/3:0+events08:30:223
170123699746741,4cyclictest1721971-21kworker/2:1+events07:47:422
170123699745741,3cyclictest1747265-21kworker/2:2+events08:06:022
170123699743739,3cyclictest1747265-21kworker/2:2+events08:01:592
170123299743737,5cyclictest1860662-21turbostat09:29:561
170123299742739,2cyclictest1854493-21kworker/1:0+events11:57:571
170123299742736,5cyclictest1728179-21kworker/1:2+events08:17:411
170123299741736,4cyclictest1854493-21kworker/1:0+events10:46:471
170123699740737,2cyclictest1873873-21kworker/2:1+events11:29:502
170123699740736,3cyclictest1721971-21kworker/2:1+events07:51:252
170123299740736,3cyclictest1854493-21kworker/1:0+events10:52:431
170123899737724,6cyclictest1707330-21kworker/3:0+events10:35:213
170123299732728,3cyclictest1728179-21kworker/1:2+events08:48:451
170123299731727,3cyclictest1854493-21kworker/1:0+events10:08:251
170123899729724,4cyclictest1707330-21kworker/3:0+events11:42:293
170123899729723,5cyclictest1707330-21kworker/3:0+events11:01:083
170123299728725,2cyclictest1854493-21kworker/1:0+events10:12:301
170123899727722,4cyclictest1707330-21kworker/3:0+events09:22:173
170123899726722,3cyclictest1707330-21kworker/3:0+events11:23:333
170123299726718,3cyclictest1854493-21kworker/1:0+events12:03:251
170123299725721,3cyclictest1728179-21kworker/1:2+events07:36:061
170123899723718,4cyclictest1707330-21kworker/3:0+events09:38:343
170123299721713,6cyclictest1728179-21kworker/1:2+events08:43:441
170123299721713,6cyclictest1728179-21kworker/1:2+events08:43:431
170123899720716,3cyclictest3801366-21kworker/3:1+events07:14:463
170123899720716,3cyclictest1779135-21turbostat08:19:553
170123899720716,3cyclictest1707330-21kworker/3:0+events12:31:043
170123899720716,3cyclictest1707330-21kworker/3:0+events10:03:523
170123899720716,3cyclictest1707330-21kworker/3:0+events09:12:543
170123899720714,5cyclictest1707330-21kworker/3:0+events07:26:593
170123899720713,4cyclictest1707330-21kworker/3:0+events09:43:463
170123299720717,2cyclictest1728179-21kworker/1:2+events08:13:121
170123899719715,3cyclictest1707330-21kworker/3:0+events12:16:583
170123899719715,3cyclictest1707330-21kworker/3:0+events11:18:573
170123899719715,3cyclictest1707330-21kworker/3:0+events08:52:293
170123899719714,4cyclictest1707330-21kworker/3:0+events12:21:053
170123899719713,5cyclictest1707330-21kworker/3:0+events08:56:063
170123899719713,5cyclictest1707330-21kworker/3:0+events07:21:453
170123299719714,4cyclictest1854493-21kworker/1:0+events09:30:451
170123899718714,3cyclictest1707330-21kworker/3:0+events12:38:353
170123899718714,3cyclictest1707330-21kworker/3:0+events12:25:423
170123899718714,3cyclictest1707330-21kworker/3:0+events11:35:433
170123899718714,3cyclictest1707330-21kworker/3:0+events11:13:003
170123899718714,3cyclictest1707330-21kworker/3:0+events10:26:253
170123899718714,3cyclictest1707330-21kworker/3:0+events09:07:193
170123899718714,3cyclictest1707330-21kworker/3:0+events08:27:333
170123299718714,3cyclictest1854493-21kworker/1:0+events12:06:341
170123299718713,4cyclictest1728179-21kworker/1:2+events07:41:251
170123899717714,2cyclictest1912753-21turbostat10:24:543
170123899717713,3cyclictest3801366-21kworker/3:1+events07:18:333
170123899717713,3cyclictest1707330-21kworker/3:0+events11:09:493
170123899717713,3cyclictest1707330-21kworker/3:0+events09:52:533
170123299717714,2cyclictest3808393-21kworker/1:1+events07:30:451
170123299717712,4cyclictest1772278-21turbostat08:09:551
170123899716712,3cyclictest1707330-21kworker/3:0+events10:43:023
170123899716712,3cyclictest1707330-21kworker/3:0+events10:43:023
170123899716712,3cyclictest1707330-21kworker/3:0+events10:31:033
170123899716709,4cyclictest1707330-21kworker/3:0+events09:57:533
170123299716713,2cyclictest1854493-21kworker/1:0+events11:51:261
170123299716713,2cyclictest1854493-21kworker/1:0+events10:57:391
170123899715711,3cyclictest1707330-21kworker/3:0+events09:16:073
170123299715711,3cyclictest1854493-21kworker/1:0+events12:10:541
170123899714710,3cyclictest1707330-21kworker/3:0+events09:47:303
170123899714710,3cyclictest1707330-21kworker/3:0+events09:03:043
170123899714710,3cyclictest1707330-21kworker/3:0+events08:23:323
170123299714711,2cyclictest1728179-21kworker/1:2+events08:38:451
170123899712707,4cyclictest1861292-21turbostat09:34:563
170123699701697,3cyclictest1740917-21turbostat07:44:562
170123899699693,5cyclictest1977469-21turbostat11:34:563
170123699696686,5cyclictest1873873-21kworker/2:1+events10:15:212
170123299691688,2cyclictest1797821-21turbostat08:34:541
170123899689684,4cyclictest1816476-21turbostat08:49:553
170123899687683,3cyclictest1945097-21turbostat10:59:553
170123699681679,1cyclictest1747265-21kworker/2:2+events07:56:462
170123699679676,2cyclictest1970639-21turbostat11:24:562
170123299676672,3cyclictest1893457-21turbostat10:04:551
170123699667659,6cyclictest1873873-21kworker/2:1+events11:32:412
170123299650646,3cyclictest1996142-21turbostat11:49:561
170123299649645,3cyclictest1721927-21turbostat07:29:561
170123699645640,4cyclictest1873873-21kworker/2:1+events10:22:102
170123699643640,2cyclictest1900322-21turbostat10:14:542
170123899637634,2cyclictest2016303-21turbostat12:14:553
170123299595593,1cyclictest1932021-21turbostat10:44:551
170123299595593,1cyclictest1932021-21turbostat10:44:551
17015752970,1sleep10-21swapper/107:10:131
170123299710,69cyclictest1843899-21grep09:10:231
1701232992623,2cyclictest1893446-21sendmail09:59:591
1701232992522,2cyclictest1996134-21cron11:45:001
1701236992419,3cyclictest1842075-21munin-plugin-st09:10:012
1701229992219,2cyclictest3614154-21systemd-journal11:10:010
1701229992219,2cyclictest0-21swapper/008:00:000
1701236992118,2cyclictest1864588-21kthreadcore09:30:132
1701229992118,2cyclictest509-21rs:main0
1701229992116,3cyclictest491-21avahi-daemon12:20:000
1701236992017,2cyclictest2004961-21kthreadcore11:55:152
1701232992017,2cyclictest3614154-21systemd-journal11:23:591
1701229992017,2cyclictest1708118-21kthreadcore07:15:030
1701236991913,4cyclictest2002965-21cron11:54:592
1701229991916,2cyclictest1887193-21sh09:55:000
1701236991815,2cyclictest2038937-21kthreadcore12:30:122
1701236991813,3cyclictest1799542-21kthreadcore08:30:212
1701232991815,2cyclictest502-21systemd-logind10:23:591
1701229991815,2cyclictest0-21swapper/009:40:010
1701229991814,3cyclictest1754150-21gltestperf07:50:170
295798170,1rtkit-daemon0-21swapper/010:21:290
1701236991713,3cyclictest1868279-21tr09:35:182
1701229991715,1cyclictest1791602-21idleruntime-cro08:25:000
1701229991714,2cyclictest1701223-21cyclictest11:35:150
1701229991713,3cyclictest1926122-21cpuspeed_turbos10:35:130
1701229991713,3cyclictest1735153-21gltestperf07:35:140
1701229991711,4cyclictest3614154-21systemd-journal09:00:010
1701238991613,2cyclictest2873-21cron07:45:003
170123699168,6cyclictest3614154-21systemd-journal07:20:012
1701236991612,3cyclictest1785716-21cpuspeed_turbos08:20:112
1701236991612,3cyclictest0-21swapper/209:55:002
1701236991612,2cyclictest0-21swapper/210:05:002
1701232991613,2cyclictest1818820-21kthreadcore08:50:121
1701229991613,2cyclictest499-21polkitd11:55:000
1701229991613,2cyclictest2042469-21head12:35:110
1701229991613,2cyclictest1804408-21cpuspeed_turbos08:35:140
1701229991613,2cyclictest0-21swapper/009:05:010
1701229991613,2cyclictest0-21swapper/007:30:010
1701229991611,2cyclictest1861071-21kworker/0:0+kacpid11:29:150
295798150,1rtkit-daemon0-21swapper/008:15:370
1701238991512,2cyclictest1765610-21unixbench_multi07:55:333
1701238991511,3cyclictest506-21in:imuxsock10:20:003
1701238991510,4cyclictest1759688-21sshd07:53:273
1701236991513,1cyclictest1735328-21irqcore07:35:152
1701236991512,2cyclictest1958189-21sh11:10:012
1701236991512,2cyclictest1939232-21cpuspeed_turbos10:50:122
1701236991512,2cyclictest0-21swapper/209:05:002
1701236991511,3cyclictest1990465-21gltestperf11:40:142
1701236991511,2cyclictest0-21swapper/211:00:012
1701236991510,3cyclictest0-21swapper/209:00:012
170123699150,14cyclictest1954603-21kthreadcore11:00:162
1701232991512,2cyclictest1907247-21irqcore10:15:181
1701232991511,3cyclictest1712730-21kthreadcore07:15:191
1701232991510,3cyclictest1983653-21cron11:35:011
170122999156,8cyclictest1798269-21sort08:30:170
1701229991513,1cyclictest1861071-21kworker/0:0+kacpid11:01:480
1701229991512,2cyclictest502-21systemd-logind10:53:590
1701229991512,2cyclictest2010061-21gltestperf12:00:150
1701229991512,2cyclictest1766491-21cut08:00:150
1701238991410,3cyclictest1900661-21cpuspeed_turbos10:10:093
1701238991410,2cyclictest0-21swapper/308:35:253
1701236991412,1cyclictest1729279-21irqcore07:30:202
1701236991411,2cyclictest1842695-21tail09:10:212
1701236991411,2cyclictest0-21swapper/208:40:162
1701236991411,2cyclictest0-21swapper/208:40:152
1701236991411,2cyclictest0-21swapper/208:29:592
1701236991410,2cyclictest0-21swapper/212:20:002
1701232991411,2cyclictest0-21swapper/108:30:011
1701232991410,3cyclictest1874186-21cpuspeed_turbos09:40:131
1701232991410,3cyclictest1836178-21cpuspeed_turbos09:05:131
1701232991410,3cyclictest0-21swapper/107:54:591
170122999149,4cyclictest1975917-21ntp_kernel_err11:20:200
1701229991412,1cyclictest1861071-21kworker/0:0+kacpid10:25:490
1701229991412,1cyclictest1771853-21kworker/0:3+kacpid09:25:130
1701229991412,1cyclictest1771853-21kworker/0:3+kacpid08:43:240
1701229991412,1cyclictest1771853-21kworker/0:3+kacpid08:43:230
1701229991412,1cyclictest0-21swapper/007:45:000
1701229991411,2cyclictest3614154-21systemd-journal07:23:590
1701229991411,2cyclictest2016708-21cpuspeed_turbos12:10:090
1701229991411,2cyclictest1957292-21kworker/0:3+kacpid11:11:190
1701229991411,2cyclictest1907236-21irqcore10:15:180
1701229991411,2cyclictest1861071-21kworker/0:0+kacpid09:58:090
1701229991411,2cyclictest1715497-21kworker/0:0+kacpid07:49:210
1701229991411,2cyclictest0-21swapper/011:20:000
1701229991411,2cyclictest0-21swapper/007:30:210
1701229991410,3cyclictest1855030-21grep09:20:140
16950162142,7sleep30-21swapper/307:05:113
170123899139,3cyclictest1749111-21kthreadcore07:45:173
170123899137,4cyclictest2016300-21turbostat.cron12:09:593
170123899137,4cyclictest1815619-21ntpq08:40:213
170123899137,4cyclictest1815619-21ntpq08:40:203
1701238991310,2cyclictest0-21swapper/310:50:213
170123699139,3cyclictest0-21swapper/210:30:002
170123699139,2cyclictest0-21swapper/208:19:592
170123699137,4cyclictest1879220-21ntpq09:40:192
170123699136,5cyclictest3614154-21systemd-journal08:40:012
1701236991310,2cyclictest0-21swapper/210:40:202
1701236991310,2cyclictest0-21swapper/210:40:202
1701236991310,2cyclictest0-21swapper/209:55:152
170123299138,4cyclictest1860243-21grep09:20:251
1701232991310,2cyclictest0-21swapper/109:50:001
1701232991310,2cyclictest0-21swapper/107:20:161
170123299130,2cyclictest1931306-21/usr/sbin/munin10:35:211
170123299130,12cyclictest0-21swapper/112:24:591
170123299130,12cyclictest0-21swapper/109:00:111
170122999139,3cyclictest1900661-21cpuspeed_turbos10:10:090
170122999138,4cyclictest1898838-21ntpq10:00:210
170122999138,4cyclictest1853579-21ntpq09:15:230
170122999138,4cyclictest1706459-21awk07:10:190
170122999135,7cyclictest493-21dbus-daemon09:35:000
1701229991312,1cyclictest1715497-21kworker/0:0+kacpid08:13:110
1701229991311,1cyclictest2009405-21kworker/0:2+kacpid12:07:160
1701229991311,1cyclictest1957292-21kworker/0:3+kacpid11:55:510
1701229991311,1cyclictest1861071-21kworker/0:0+kacpid12:24:380
1701229991311,1cyclictest1861071-21kworker/0:0+kacpid11:48:080
1701229991311,1cyclictest1861071-21kworker/0:0+kacpid11:42:200
1701229991311,1cyclictest1861071-21kworker/0:0+kacpid10:06:050
1701229991311,1cyclictest1861071-21kworker/0:0+kacpid09:46:410
1701229991311,1cyclictest1791306-21kworker/0:2+kacpid08:51:550
1701229991311,1cyclictest1771853-21kworker/0:3+kacpid08:45:110
1701229991311,1cyclictest1771853-21kworker/0:3+kacpid08:25:580
1701229991311,1cyclictest0-21swapper/010:50:000
1701229991310,2cyclictest1950571-21systemctl10:55:230
1701229991310,2cyclictest1874243-21cpuspeed_turbos09:40:130
1701229991310,2cyclictest1842624-21gltestperf09:10:210
170122999130,2cyclictest295798rtkit-daemon11:30:130
170123899129,2cyclictest2003610-21cpuspeed_turbos11:55:113
170123899129,2cyclictest0-21swapper/308:10:003
170123899124,2cyclictest295798rtkit-daemon12:02:233
170123699129,2cyclictest1777447-21kthreadcore08:10:162
170123699129,2cyclictest0-21swapper/210:40:002
170123699129,2cyclictest0-21swapper/209:15:162
170123699129,2cyclictest0-21swapper/207:30:012
170123699128,3cyclictest401ktimers/211:10:072
170123699128,3cyclictest0-21swapper/208:50:012
170123699128,2cyclictest1706915-21grep07:10:242
170123699120,11cyclictest0-21swapper/211:35:222
170123299129,2cyclictest3614154-21systemd-journal11:26:581
170123299128,3cyclictest493-21dbus-daemon11:10:011
170123299128,3cyclictest1957542-21cron11:05:001
170123299128,3cyclictest1753005-21wc07:45:241
170123299123,1cyclictest295798rtkit-daemon09:16:451
1701232991210,1cyclictest6950irq/9-acpi11:15:461
170123299120,1cyclictest0-21swapper/111:38:081
170122999127,4cyclictest1932600-21grep10:40:150
170122999127,4cyclictest1932600-21grep10:40:140
170122999126,4cyclictest2040910-21needreboot12:30:150
1701229991210,1cyclictest1861071-21kworker/0:0+kacpid12:26:540
1701229991210,1cyclictest1771853-21kworker/0:3+kacpid09:05:090
1701229991210,1cyclictest1771853-21kworker/0:3+kacpid08:07:060
170123899117,3cyclictest1997179-21cpuspeed_turbos11:50:093
170123899116,4cyclictest1769009-21tr08:00:183
170123899115,4cyclictest1775623-21tr08:10:143
170123899114,5cyclictest493-21dbus-daemon07:35:003
170123699119,1cyclictest1819906-21kthreadcore08:50:132
170123699118,2cyclictest2042241-21cpuspeed_turbos12:35:112
170123699118,2cyclictest2029982-21irqrtprio12:20:162
170123699118,2cyclictest2014171-21kthreadcore12:00:192
170123699117,3cyclictest2016639-21cpuspeed_turbos12:10:092
170123699117,2cyclictest0-21swapper/209:20:172
170123699116,4cyclictest493-21dbus-daemon10:50:002
170123699116,4cyclictest1716313-21sed07:20:152
170123699116,3cyclictest1880948-21cat09:50:012
170123699114,6cyclictest493-21dbus-daemon11:50:012
170123699114,5cyclictest523-21gdbus11:20:012
170123699112,4cyclictest2956-21rtkit-daemon10:05:462
170123299119,1cyclictest0-21swapper/112:26:301
170123299118,2cyclictest2022848-21cpuspeed_turbos12:15:121
170123299118,2cyclictest0-21swapper/112:30:091
170123299117,3cyclictest1958534-21cut11:10:081
170123299116,4cyclictest1886239-21grep09:50:161
170123299116,4cyclictest1868094-21tune2fs09:35:161
170123299116,4cyclictest1767767-21cut08:00:171
170123299111,3cyclictest2956-21rtkit-daemon07:58:381
170123299110,11cyclictest0-21swapper/110:34:401
170123299110,11cyclictest0-21swapper/110:27:381
170123299110,11cyclictest0-21swapper/108:55:211
170123899108,1cyclictest0-21swapper/310:07:173
170123899105,4cyclictest1735239-21head07:35:153
170123699108,1cyclictest0-21swapper/212:30:012
170123699108,1cyclictest0-21swapper/212:09:592
170123699106,2cyclictest0-21swapper/210:30:092
170123299108,1cyclictest0-21swapper/108:20:211
170123299105,4cyclictest2042536-21hddtemp_smartct12:35:121
17012389990,2cyclictest0-21swapper/311:27:083
17012369990,8cyclictest0-21swapper/209:28:282
1700920296,2sleep20-21swapper/207:08:062
1700908295,2sleep00-21swapper/007:07:570
17012389985,2cyclictest491ktimers/310:47:493
17012329983,4cyclictest1701242-21cut07:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional