You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-11 - 23:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Fri Apr 11, 2025 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
288309911841159,22cyclictest0-21swapper/007:25:070
288319911181109,6cyclictest0-21swapper/107:25:091
288319911091083,22cyclictest804-21apt-get07:35:001
285212634589,30sleep00-21swapper/007:22:160
286542464417,32sleep10-21swapper/107:23:421
2883099240215,20cyclictest0-21swapper/007:35:000
114422880,9sleep12883199cyclictest08:00:221
28830995710,11cyclictest2062-21vmstat09:00:280
2883099532,13cyclictest2220-21dbus-daemon07:56:080
96272410,8sleep12883199cyclictest07:55:251
2883099394,11cyclictest81rcu_preempt11:38:370
43702381,32sleep04511-21cut12:35:010
2883099381,23cyclictest8169-21pidof12:45:000
2883199364,25cyclictest6740-21/usr/sbin/munin07:50:341
64322352,29sleep02220-21dbus-daemon07:48:350
2883199352,12cyclictest7117-21memory12:40:201
28831993517,14cyclictest22-21ksoftirqd/110:20:071
28831993517,14cyclictest22-21ksoftirqd/110:20:071
2883099351,11cyclictest81rcu_preempt08:49:440
28830993510,12cyclictest25602-21users11:55:320
20492351,30sleep12376-21NetworkManager12:26:241
28831993417,14cyclictest22-21ksoftirqd/111:19:131
2883199341,11cyclictest312222cut07:30:031
139822341,4sleep114063-21users09:35:331
2883199336,4cyclictest81rcu_preempt09:01:351
2883199336,4cyclictest81rcu_preempt08:28:461
2883199331,14cyclictest93102cut12:50:121
2883099338,12cyclictest31570-21mailstats12:15:270
2883099335,17cyclictest1486-21grep10:40:160
137492331,4sleep113822-21users08:05:301
299392321,4sleep129966-21hddtemp_smartct08:50:211
2883099322,26cyclictest31844-21smartctl08:55:170
2883099322,16cyclictest0-21swapper/008:19:580
2883099321,18cyclictest11624-21rt-features08:00:260
2883199315,4cyclictest81rcu_preempt10:54:041
2883199315,4cyclictest81rcu_preempt10:43:291
2883199313,4cyclictest81rcu_preempt09:19:141
2883199311,11cyclictest31012-21fschecks_time12:15:151
28830993121,6cyclictest20650-21memory11:40:220
2883099311,4cyclictest81rcu_preempt09:29:070
2883099311,4cyclictest81rcu_preempt09:14:460
28830993112,7cyclictest13539-21/usr/sbin/munin08:05:270
28830993112,6cyclictest26891-21latency12:00:220
171842311,27sleep117287-21which08:15:271
69472301,5sleep10-21swapper/110:56:571
2883199305,4cyclictest81rcu_preempt10:37:311
2883199304,4cyclictest81rcu_preempt11:11:001
2883199304,4cyclictest81rcu_preempt07:49:101
2883199303,4cyclictest81rcu_preempt08:11:181
2883199301,4cyclictest81rcu_preempt07:43:141
2883099307,20cyclictest16395-21cron09:45:000
2883099305,13cyclictest27451-21ls10:20:120
2883099305,13cyclictest27451-21ls10:20:120
2883099301,4cyclictest81rcu_preempt08:43:310
28830993012,5cyclictest8020-21iostat11:00:200
28830993012,5cyclictest328-21df_inode12:20:160
28830993012,5cyclictest13981-21df11:20:140
2883099301,17cyclictest23760-21latency_hist08:35:000
2883099301,11cyclictest313272aten_r4power_en07:30:030
157712301,5sleep00-21swapper/008:11:100
83542291,5sleep00-21swapper/012:45:230
78132291,5sleep00-21swapper/009:15:370
43762291,5sleep00-21swapper/007:41:430
292352291,5sleep00-21swapper/010:26:350
2883199295,4cyclictest81rcu_preempt10:46:071
2883199295,4cyclictest81rcu_preempt09:41:361
2883199295,4cyclictest81rcu_preempt09:05:391
2883199295,3cyclictest81rcu_preempt09:26:491
2883199294,4cyclictest81rcu_preempt12:34:271
2883199294,4cyclictest81rcu_preempt11:42:461
2883199294,4cyclictest81rcu_preempt09:31:351
2883199294,4cyclictest81rcu_preempt09:21:581
2883199293,4cyclictest81rcu_preempt12:45:191
2883199293,16cyclictest0-21swapper/110:33:271
2883199293,12cyclictest0-21swapper/111:57:201
2883099298,5cyclictest0-21swapper/010:59:380
2883099293,4cyclictest81rcu_preempt10:08:560
2883099292,14cyclictest4342-21/usr/sbin/munin10:50:180
28830992919,7cyclictest3-21ksoftirqd/009:05:280
28830992918,7cyclictest386-21unixbench_singl10:35:340
2883099291,4cyclictest81rcu_preempt12:29:140
28830992912,13cyclictest9254-21diskstats12:50:120
28830992910,6cyclictest5714-21awk12:35:300
28830992910,6cyclictest5714-21awk12:35:300
284272291,5sleep00-21swapper/012:08:380
271972291,5sleep10-21swapper/108:42:401
190582291,5sleep119114-21sed11:35:291
4082281,6sleep10-21swapper/108:58:101
2883199288,7cyclictest0-21swapper/112:05:241
2883199288,16cyclictest0-21swapper/111:30:151
2883199284,15cyclictest0-21swapper/108:31:251
2883199283,4cyclictest81rcu_preempt11:04:501
2883199283,21cyclictest0-21swapper/112:10:221
28831992821,5cyclictest0-21swapper/108:36:391
2883199281,22cyclictest19923-21taskset09:55:241
2883099282,13cyclictest24050-21/usr/sbin/munin10:10:210
2883099282,13cyclictest22226-21vmstat10:00:290
28830992812,5cyclictest30857-21taskset08:51:400
2883099281,14cyclictest10138-21/usr/sbin/munin09:25:010
2883099281,13cyclictest20715-21runrttasks09:59:260
28830992810,6cyclictest12222-21runrttasks09:30:450
241652281,5sleep10-21swapper/110:10:261
218632281,5sleep00-21swapper/011:45:280
2883199276,3cyclictest81rcu_preempt10:26:291
2883199274,4cyclictest81rcu_preempt10:08:511
2883199273,4cyclictest81rcu_preempt12:20:101
2883199273,4cyclictest81rcu_preempt11:28:241
2883199273,4cyclictest81rcu_preempt11:07:281
2883199273,4cyclictest81rcu_preempt10:18:441
2883199272,4cyclictest81rcu_preempt09:12:141
28831992718,5cyclictest22-21ksoftirqd/112:00:141
2883199271,11cyclictest22217-21users10:00:291
2883099279,6cyclictest13518-21missed_timers09:35:240
2883099273,8cyclictest10228-21diskstats11:10:190
2883099272,13cyclictest2376-21NetworkManager08:22:500
2883099271,14cyclictest24440-21uname08:35:230
2883099271,13cyclictest29850-21grep10:30:140
2883099271,13cyclictest22741-21df_inode11:50:100
2883099270,4cyclictest81rcu_preempt11:27:030
197232271,4sleep10-21swapper/108:23:031
178502271,4sleep00-21swapper/011:34:290
166232271,5sleep10-21swapper/109:45:521
136922271,4sleep10-21swapper/111:20:101
292042261,5sleep10-21swapper/108:49:011
2883199262,11cyclictest23285-21ntp_states11:50:201
2883199261,5cyclictest81rcu_preempt12:35:271
2883199261,5cyclictest81rcu_preempt12:35:271
2883099262,12cyclictest9437-21chrt11:06:370
2883099261,4cyclictest81rcu_preempt10:17:360
2883099261,16cyclictest0-21swapper/008:30:000
2883099261,13cyclictest29036-21/usr/sbin/munin12:10:140
28830992610,5cyclictest8099-21apt-get07:51:280
2883199251,12cyclictest17183-21/usr/sbin/munin09:50:251
2883099253,17cyclictest11880-21wget11:15:110
28830992516,6cyclictest3-21ksoftirqd/009:45:370
2883099241,12cyclictest18512-21cron09:50:590
2883199234,10cyclictest0-21swapper/111:46:241
2883099231,11cyclictest3089-21runrttasks10:45:500
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional