You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Thu Apr 03, 2025 00:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
210609911871177,7cyclictest0-21swapper/019:50:290
210619910911066,20cyclictest0-21swapper/119:25:121
209492577531,31sleep10-21swapper/119:24:261
209562561517,29sleep00-21swapper/019:24:290
2106099214189,22cyclictest0-21swapper/019:25:120
59242760,5sleep0111rcuc/021:35:300
21061997266,3cyclictest0-21swapper/119:50:281
21061995117,6cyclictest22-21ksoftirqd/121:29:521
21061995116,6cyclictest22-21ksoftirqd/123:21:421
21061995116,5cyclictest22-21ksoftirqd/120:26:401
21061995115,5cyclictest22-21ksoftirqd/120:05:271
21061995027,11cyclictest22-21ksoftirqd/122:35:121
21061995017,6cyclictest22-21ksoftirqd/123:14:071
21061995017,6cyclictest22-21ksoftirqd/123:04:511
21061995017,6cyclictest22-21ksoftirqd/121:08:411
21061995016,6cyclictest22-21ksoftirqd/123:47:091
21061995016,6cyclictest22-21ksoftirqd/123:40:501
21061995016,6cyclictest22-21ksoftirqd/123:34:081
21061995016,6cyclictest22-21ksoftirqd/123:09:481
21061995016,6cyclictest22-21ksoftirqd/121:49:581
21061995016,6cyclictest22-21ksoftirqd/121:04:151
21061995016,6cyclictest22-21ksoftirqd/120:49:291
21061995016,6cyclictest22-21ksoftirqd/120:39:211
21061995016,5cyclictest22-21ksoftirqd/122:57:101
21061994918,6cyclictest22-21ksoftirqd/120:40:151
21061994917,6cyclictest22-21ksoftirqd/123:36:021
21061994917,5cyclictest22-21ksoftirqd/121:56:131
21061994916,6cyclictest22-21ksoftirqd/123:27:241
21061994916,6cyclictest22-21ksoftirqd/123:17:551
21061994916,6cyclictest22-21ksoftirqd/122:53:091
21061994916,6cyclictest22-21ksoftirqd/122:00:191
21061994916,6cyclictest22-21ksoftirqd/121:20:101
21061994916,6cyclictest22-21ksoftirqd/121:15:111
21061994916,6cyclictest22-21ksoftirqd/120:55:431
21061994916,6cyclictest22-21ksoftirqd/120:31:361
21061994916,5cyclictest22-21ksoftirqd/122:07:511
21061994916,5cyclictest22-21ksoftirqd/121:40:351
21061994916,5cyclictest22-21ksoftirqd/121:37:161
21061994916,5cyclictest22-21ksoftirqd/121:14:281
21061994916,5cyclictest22-21ksoftirqd/120:52:541
21061994916,5cyclictest22-21ksoftirqd/100:02:021
21061994816,5cyclictest22-21ksoftirqd/123:55:521
21061994816,5cyclictest22-21ksoftirqd/121:50:041
21061994816,5cyclictest22-21ksoftirqd/100:37:221
21061994815,6cyclictest22-21ksoftirqd/100:40:111
21061994717,5cyclictest22-21ksoftirqd/123:50:231
21061994716,5cyclictest22-21ksoftirqd/100:06:141
21061994715,6cyclictest22-21ksoftirqd/100:33:491
21061994624,4cyclictest22-21ksoftirqd/122:40:231
21061994615,6cyclictest22-21ksoftirqd/119:40:251
21061994615,6cyclictest22-21ksoftirqd/100:50:101
21061994613,5cyclictest22-21ksoftirqd/100:46:351
251562450,8sleep02106099cyclictest00:20:220
21061994515,10cyclictest22-21ksoftirqd/120:01:271
21061994510,6cyclictest16775-21kworker/1:300:15:201
269152441,3sleep026935-21latency_hist21:05:020
21061994423,4cyclictest22-21ksoftirqd/121:30:121
21061994414,9cyclictest22-21ksoftirqd/120:20:121
21061994324,4cyclictest22-21ksoftirqd/122:15:121
21061994311,6cyclictest22-21ksoftirqd/119:45:251
21061994311,6cyclictest22-21ksoftirqd/119:45:251
2106199429,4cyclictest81rcu_preempt19:30:511
2106199428,6cyclictest22-21ksoftirqd/100:13:231
21061994227,4cyclictest22-21ksoftirqd/100:25:531
21061994226,5cyclictest22-21ksoftirqd/120:10:241
21061994223,4cyclictest22-21ksoftirqd/120:15:191
158022420,7sleep12106199cyclictest22:10:121
2106199416,6cyclictest26244-21latency19:35:231
21061994126,4cyclictest22-21ksoftirqd/122:46:471
21061994026,4cyclictest22-21ksoftirqd/122:30:201
21061994026,4cyclictest22-21ksoftirqd/120:00:011
21061994026,4cyclictest22-21ksoftirqd/100:23:091
21061994022,7cyclictest22-21ksoftirqd/122:24:351
21060994016,9cyclictest18487-21diskstats00:00:170
21061993924,4cyclictest22-21ksoftirqd/122:27:331
2106099364,5cyclictest23770-21unixbench_multi00:15:300
2106099337,14cyclictest14031-21aten_r4power_vo20:30:140
2106099336,5cyclictest81rcu_preempt23:40:170
2106099331,18cyclictest23151-21cron20:55:010
2106099321,14cyclictest27276-21sort00:30:120
2106099321,14cyclictest17129-21ntp_states23:55:260
2106099319,19cyclictest30575-21sendmail00:40:000
2106099312,3cyclictest81rcu_preempt21:32:560
2106099312,14cyclictest25277-21cron22:40:000
2106099311,8cyclictest0-21swapper/000:40:300
2106099311,4cyclictest81rcu_preempt23:17:130
2106099311,4cyclictest81rcu_preempt22:52:190
2106099311,4cyclictest81rcu_preempt21:16:230
21060993112,9cyclictest31789-21aten_r4power_en23:00:140
2106099304,4cyclictest81rcu_preempt19:40:170
2106099302,15cyclictest16721-21/usr/sbin/munin20:35:270
2106099302,12cyclictest3249-21iostat_ios20:00:190
2106099301,5cyclictest81rcu_preempt21:40:290
2106099301,4cyclictest81rcu_preempt22:58:340
2106099301,4cyclictest81rcu_preempt20:13:170
21060993012,6cyclictest8843-21/usr/sbin/munin23:30:210
21060993011,6cyclictest19474-21df22:20:180
21060993010,6cyclictest149682chrt22:07:010
2106099300,4cyclictest81rcu_preempt00:46:440
139892301,5sleep00-21swapper/023:47:120
35182291,5sleep00-21swapper/023:14:120
24602291,5sleep00-21swapper/000:54:070
2106099299,5cyclictest10003-21/usr/sbin/munin21:50:210
2106099292,14cyclictest0-21swapper/021:48:340
2106099291,4cyclictest81rcu_preempt23:27:550
21060992911,6cyclictest9558-21taskset20:15:330
21060992911,6cyclictest15997-21iostat22:10:140
21060992910,5cyclictest23196-21diskstats19:30:030
202542291,5sleep00-21swapper/000:07:100
187232291,5sleep00-21swapper/022:17:050
115122291,5sleep00-21swapper/020:21:110
72932281,5sleep00-21swapper/023:24:150
327192281,5sleep00-21swapper/021:20:050
272512281,5sleep00-21swapper/019:39:300
22692281,5sleep00-21swapper/021:26:200
222712281,5sleep00-21swapper/000:13:370
2106099289,6cyclictest21771-21aten_r4power_po22:30:080
2106099282,9cyclictest26063-21iostat_ios22:40:210
2106099281,3cyclictest81rcu_preempt20:26:420
2106099281,3cyclictest81rcu_preempt19:49:390
2106099281,3cyclictest81rcu_preempt19:49:380
21060992811,6cyclictest12563-21chrt21:58:150
21060992811,5cyclictest0-21swapper/022:26:120
209922281,5sleep00-21swapper/020:47:080
20942281,5sleep00-21swapper/019:57:100
157812281,5sleep00-21swapper/023:51:220
144372281,5sleep00-21swapper/022:03:020
267312271,5sleep00-21swapper/000:29:170
2106099272,12cyclictest23912-21iostat_ios20:55:200
2106099271,4cyclictest81rcu_preempt23:37:560
2106099271,3cyclictest81rcu_preempt20:09:010
2106099271,14cyclictest29558-21sh21:10:230
21060992710,5cyclictest27096-21aten_r4power_cu21:05:110
13302271,5sleep00-21swapper/023:07:070
2106099268,4cyclictest18130-21fschecks_time20:40:170
2106099249,4cyclictest27426-21taskset22:46:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional