You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-13 - 18:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Thu Mar 13, 2025 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2085017993529,4cyclictest2289136-21cron00:05:002
2085013993523,10cyclictest2296837-21latency_hist00:15:011
2085017993424,8cyclictest0-21swapper/219:19:592
2085013993316,2cyclictest0-21swapper/100:20:011
2085013993315,2cyclictest0-21swapper/122:35:001
2085013993315,16cyclictest0-21swapper/121:40:291
2085013993313,7cyclictest2169542-21tr21:55:131
2085017993218,12cyclictest0-21swapper/222:20:002
2085017993216,2cyclictest0-21swapper/220:25:002
208501799321,18cyclictest2227014-21taskset22:56:112
2085017993116,2cyclictest0-21swapper/219:40:002
2085017993115,2cyclictest0-21swapper/221:30:002
2085013993110,8cyclictest2269818-21seq23:43:151
2085013993110,8cyclictest2269818-21seq23:43:141
2085010993117,10cyclictest2284986-21cut00:00:150
2085017993022,5cyclictest699-21dbus-daemon00:00:012
2085017993020,9cyclictest703-21polkitd22:34:592
2085013993017,11cyclictest2088896-21awk19:19:591
2085013993016,12cyclictest2139843-21threads21:20:241
2085013993015,3cyclictest2093753-21cron19:34:591
2085013993014,2cyclictest0-21swapper/122:45:001
208501399301,16cyclictest2185726-21taskset22:12:051
208501399294,4cyclictest2181205-21sh22:08:131
208501399291,17cyclictest2218372-21sh22:47:541
208501399290,18cyclictest2133933-21sh21:15:091
2085020992822,2cyclictest191rcu_preempt23:44:513
2085020992822,2cyclictest191rcu_preempt23:44:503
2085017992815,2cyclictest703-21polkitd23:20:222
2085017992810,6cyclictest699-21dbus-daemon20:29:592
208501399289,6cyclictest2104687-21grep20:00:191
208501399288,6cyclictest699-21dbus-daemon21:50:181
2085013992823,3cyclictest2234330-21latency_hist23:05:001
208501399281,9cyclictest2151230-21apt21:34:591
208501399281,6cyclictest2144732-21taskset21:27:101
2085013992815,2cyclictest2122934-21turbostat.cron20:55:001
2085013992814,3cyclictest2127259-21cpuspeed_turbos21:05:441
2085013992814,1cyclictest0-21swapper/122:15:151
208501399281,16cyclictest2123839-21head20:55:161
2085013992810,7cyclictest940-21runrttasks23:49:001
2085013992810,6cyclictest2249796-21ssh23:20:381
2085020992712,6cyclictest191rcu_preempt21:40:173
208501799278,8cyclictest2257443-21ntp_192.168.11423:30:122
2085017992715,2cyclictest0-21swapper/221:00:012
208501399279,6cyclictest2243510-21idleruntime23:15:141
2085013992710,5cyclictest262-21systemd-journal23:36:061
208502099263,18cyclictest52-21ksoftirqd/323:08:323
2085020992622,2cyclictest703-21polkitd19:45:013
2085020992621,3cyclictest712-21tpm2-abrmd20:30:013
2085020992621,3cyclictest2266295-21munin-plugin-st23:40:003
208502099260,8cyclictest2130039-21ntpq21:10:203
208501799263,7cyclictest2186154-21sh22:12:432
2085017992616,2cyclictest1-21systemd00:15:022
2085017992615,7cyclictest2158890-21sshd21:41:512
2085017992613,3cyclictest2125536-21gltestperf21:00:132
208501799261,17cyclictest2181426-21ssh22:08:322
2085017992611,6cyclictest2193080-21ssh22:20:082
208501799260,17cyclictest2245365-21ssh23:16:342
2085013992616,8cyclictest0-21swapper/121:15:001
2085013992612,3cyclictest2125306-21cpuspeed_turbos21:00:121
2085013992610,6cyclictest2162738-21sh21:47:211
2085010992622,2cyclictest0-21swapper/023:20:000
2085020992520,1cyclictest191rcu_preempt23:01:013
2085020992516,2cyclictest191rcu_preempt20:40:293
208501799258,7cyclictest2271177-21grep23:45:012
208501799258,7cyclictest2271177-21grep23:45:002
208501799250,17cyclictest2212426-21ssh22:40:202
208501399259,5cyclictest2206950-21ls22:35:171
208501399259,5cyclictest2206950-21ls22:35:161
2085013992513,3cyclictest22821102sleep123:56:481
2085013992512,3cyclictest942-21snmpd22:24:231
208501399251,16cyclictest2100157-21/usr/sbin/munin19:50:151
2085013992510,5cyclictest2109183-21kworker/u16:320:15:181
208501399250,6cyclictest2286251-21ssh00:00:351
208501399250,13cyclictest2311729-21/usr/sbin/munin00:35:281
208501399250,13cyclictest2311729-21/usr/sbin/munin00:35:271
208502099248,10cyclictest52-21ksoftirqd/322:35:003
2085020992419,3cyclictest0-21swapper/323:00:003
2085020992419,1cyclictest191rcu_preempt21:31:153
208502099241,8cyclictest52-21ksoftirqd/323:10:123
208501399244,9cyclictest726-21gdbus20:40:001
208501399240,16cyclictest2236645-21ssh23:08:091
2085020992320,2cyclictest628-21rpcbind22:16:173
2085020992320,2cyclictest0-21swapper/322:40:003
2085020992320,2cyclictest0-21swapper/322:39:593
2085020992313,7cyclictest699-21dbus-daemon23:20:003
208502099231,18cyclictest2282623-21runrttasks23:57:373
208502099230,16cyclictest2134565-21sed21:15:183
2085017992311,5cyclictest2133506-21sshd21:14:502
2085013992319,2cyclictest2116107-21turbostat.cron20:35:001
208501399231,8cyclictest2258400-21ssh23:30:391
2085013992313,3cyclictest331ktimers/122:26:461
208501399231,15cyclictest2128849-21diskmemload22:56:121
208501399231,13cyclictest2089874-21if_enp1s019:20:161
2085010992318,3cyclictest2218327-21sh22:47:500
2085010992312,7cyclictest2311904-21df00:35:160
2085010992312,7cyclictest2311904-21df00:35:150
208502099229,3cyclictest0-21swapper/323:20:223
208502099223,15cyclictest2128849-21diskmemload22:48:283
2085020992217,1cyclictest191rcu_preempt20:10:123
2085020992217,1cyclictest191rcu_preempt19:55:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional