You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 04:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Wed Apr 02, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
263962115104,6sleep20-21swapper/219:09:422
26105211595,14sleep10-21swapper/119:05:571
26319211493,15sleep30-21swapper/319:08:433
25668210895,6sleep00-21swapper/019:05:230
145762720,0sleep10-21swapper/100:14:341
242982310,0sleep20-21swapper/221:27:522
197922290,1sleep219793-21munin-run00:20:002
29422250,1sleep20-21swapper/221:39:472
63762210,0sleep20-21swapper/222:18:272
2644999210,18cyclictest0-21swapper/221:05:122
272162200,0sleep00-21swapper/022:05:580
2644999191,3cyclictest10979-21kworker/u8:023:35:132
271232180,0sleep10-21swapper/100:28:241
26447991815,2cyclictest757-21kworker/u8:122:49:050
26450991613,2cyclictest31372-21modprobe19:20:173
26450991612,2cyclictest227762sleep321:25:333
2644999160,13cyclictest0-21swapper/223:20:292
2644899161,3cyclictest27251-21users20:35:261
2644899161,13cyclictest21642-21perl22:35:141
26447991612,3cyclictest19083-21kworker/u8:123:45:050
2644799161,13cyclictest26043-21ssh23:50:370
26450991510,3cyclictest30315-21sh21:34:333
26450991510,3cyclictest0-21swapper/323:05:193
2645099150,13cyclictest2272-21ssh00:35:263
2644999151,12cyclictest7484-21diskmemload22:00:472
26449991510,3cyclictest7772-21taskset22:20:122
26449991510,3cyclictest0-21swapper/221:33:512
2644999150,3cyclictest29986-21basename20:45:132
2644999150,3cyclictest0-21swapper/200:29:512
26448991512,2cyclictest0-21swapper/122:57:521
2644899151,12cyclictest6119-21ssh22:54:141
2644899151,12cyclictest28377-21ssh22:07:441
26448991510,3cyclictest3886-21df_abs21:40:141
2644899150,14cyclictest7484-21diskmemload22:42:521
2644899150,13cyclictest0-21swapper/123:36:121
2644899150,13cyclictest0-21swapper/122:01:321
2644799152,11cyclictest0-21swapper/000:35:300
2644799151,3cyclictest31487-21cat19:20:170
26447991512,2cyclictest2444-21kworker/u8:322:56:050
2644799151,12cyclictest0-21swapper/022:15:390
2644799150,3cyclictest7438-21ps19:40:200
2644799150,2cyclictest0-21swapper/022:25:170
2644799150,13cyclictest31083-21munin-node0
2645099141,2cyclictest17345-21munin-node3
26450991411,2cyclictest0-21swapper/321:35:193
26450991411,2cyclictest0-21swapper/319:35:193
2645099141,11cyclictest19285-21ssh21:20:323
2645099140,12cyclictest7484-21diskmemload22:15:113
2645099140,12cyclictest25907-21munin-node3
2645099140,12cyclictest22210-21diskstats22:00:123
2645099140,12cyclictest0-21swapper/322:40:143
2644999149,3cyclictest29656-21munin-run20:45:002
2644999149,3cyclictest0-21swapper/200:01:132
26449991411,2cyclictest0-21swapper/222:13:432
2644999140,3cyclictest21622-21latency_hist22:00:002
2644999140,2cyclictest0-21swapper/222:25:132
2644999140,2cyclictest0-21swapper/221:15:132
2644999140,2cyclictest0-21swapper/221:10:232
2644999140,13cyclictest0-21swapper/223:33:532
2644999140,13cyclictest0-21swapper/223:00:332
2644999140,12cyclictest7077-21ssh00:05:362
2644999140,12cyclictest21173-21ssh23:45:382
2644999140,12cyclictest0-21swapper/222:43:372
2644999140,12cyclictest0-21swapper/221:40:112
2644999140,12cyclictest0-21swapper/220:00:062
2644999140,12cyclictest0-21swapper/219:40:272
2644999140,12cyclictest0-21swapper/200:20:122
2644999140,12cyclictest0-21swapper/200:10:032
2644899149,3cyclictest19441-21ssh00:19:291
2644899142,9cyclictest3594-21Xorg00:05:591
2644899141,7cyclictest0-21swapper/119:15:011
26448991411,2cyclictest19596-21kworker/u8:321:21:051
26448991411,2cyclictest0-21swapper/123:40:021
26448991411,2cyclictest0-21swapper/123:25:521
26448991411,2cyclictest0-21swapper/122:20:421
26448991411,2cyclictest0-21swapper/121:29:121
26448991411,2cyclictest0-21swapper/100:04:021
2644899141,11cyclictest0-21swapper/120:20:201
2644899140,3cyclictest0-21swapper/122:19:071
2644899140,3cyclictest0-21swapper/120:30:181
2644899140,2cyclictest7484-21diskmemload21:35:521
2644899140,2cyclictest0-21swapper/100:21:321
2644899140,13cyclictest0-21swapper/122:13:521
2644899140,13cyclictest0-21swapper/121:14:221
2644899140,12cyclictest7484-21diskmemload23:55:201
2644899140,12cyclictest31725-21cut19:20:201
2644899140,12cyclictest0-21swapper/121:50:271
2644899140,12cyclictest0-21swapper/121:30:561
26447991411,2cyclictest26655-21kworker/u8:000:24:030
26447991411,2cyclictest1053-21ssh22:13:070
26447991411,2cyclictest0-21swapper/023:32:470
26447991411,2cyclictest0-21swapper/000:07:370
2644799140,2cyclictest0-21swapper/021:42:050
2644799140,2cyclictest0-21swapper/021:37:580
2644799140,13cyclictest0-21swapper/023:08:070
2644799140,12cyclictest0-21swapper/023:36:270
2644799140,12cyclictest0-21swapper/022:00:490
2644799140,12cyclictest0-21swapper/019:55:130
2644799140,12cyclictest0-21swapper/000:10:100
231182140,1sleep10-21swapper/123:48:361
2645099136,5cyclictest0-21swapper/321:40:593
26450991310,2cyclictest9626-21python19:45:243
26450991310,2cyclictest7484-21diskmemload00:05:253
26450991310,2cyclictest32224-21ssh22:47:173
26450991310,2cyclictest0-21swapper/323:35:213
26450991310,2cyclictest0-21swapper/323:15:143
26450991310,2cyclictest0-21swapper/323:10:083
26450991310,2cyclictest0-21swapper/323:01:393
26450991310,2cyclictest0-21swapper/322:23:193
26450991310,2cyclictest0-21swapper/322:05:223
2645099130,2cyclictest0-21swapper/323:32:193
2645099130,2cyclictest0-21swapper/322:10:103
2645099130,2cyclictest0-21swapper/320:30:003
2645099130,2cyclictest0-21swapper/319:15:123
2645099130,2cyclictest0-21swapper/300:20:093
2645099130,11cyclictest7484-21diskmemload00:00:553
26449991310,2cyclictest0-21swapper/223:55:512
26449991310,2cyclictest0-21swapper/223:28:492
26449991310,2cyclictest0-21swapper/223:11:132
26449991310,2cyclictest0-21swapper/222:52:372
26449991310,2cyclictest0-21swapper/222:35:032
26449991310,2cyclictest0-21swapper/200:30:432
2644999130,2cyclictest0-21swapper/223:44:152
2644999130,12cyclictest0-21swapper/219:10:172
2644999130,11cyclictest0-21swapper/223:05:202
2644999130,11cyclictest0-21swapper/222:57:492
2644899132,6cyclictest0-21swapper/119:40:011
2644899132,5cyclictest0-21swapper/119:10:011
2644899131,9cyclictest0-21swapper/122:30:091
2644899131,3cyclictest0-21swapper/123:50:091
26448991310,2cyclictest0-21swapper/123:31:231
26448991310,2cyclictest0-21swapper/123:14:421
26448991310,2cyclictest0-21swapper/122:25:361
2644899130,2cyclictest0-21swapper/123:21:521
2644899130,2cyclictest0-21swapper/123:00:131
2644899130,2cyclictest0-21swapper/100:39:121
2644899130,12cyclictest0-21swapper/123:18:421
2644899130,12cyclictest0-21swapper/123:05:521
2644799136,5cyclictest13993-21sh21:51:040
2644799132,6cyclictest0-21swapper/021:15:000
2644799131,9cyclictest0-21swapper/022:30:220
26447991310,2cyclictest3745-21runrttasks21:20:080
26447991310,2cyclictest19083-21kworker/u8:123:44:040
26447991310,2cyclictest0-21swapper/023:00:000
26447991310,2cyclictest0-21swapper/022:52:000
26447991310,2cyclictest0-21swapper/022:35:170
26447991310,2cyclictest0-21swapper/022:24:480
26447991310,2cyclictest0-21swapper/000:17:570
2644799130,2cyclictest7484-21diskmemload00:04:040
2644799130,2cyclictest0-21swapper/023:10:200
2644799130,2cyclictest0-21swapper/022:40:180
2644799130,2cyclictest0-21swapper/021:31:110
2644799130,2cyclictest0-21swapper/000:28:170
189802130,2sleep00-21swapper/021:56:130
2645099129,2cyclictest0-21swapper/323:56:113
2645099129,2cyclictest0-21swapper/323:26:233
2645099129,2cyclictest0-21swapper/322:50:093
2645099129,2cyclictest0-21swapper/321:15:353
2645099129,2cyclictest0-21swapper/300:10:143
2645099121,8cyclictest0-21swapper/321:09:593
2645099120,2cyclictest26486-21ssh23:51:173
2645099120,2cyclictest15851-21ssh00:15:173
2645099120,2cyclictest0-21swapper/323:45:223
2645099120,2cyclictest0-21swapper/321:55:473
2645099120,2cyclictest0-21swapper/321:50:493
2645099120,2cyclictest0-21swapper/321:45:183
2645099120,2cyclictest0-21swapper/321:10:443
2645099120,2cyclictest0-21swapper/320:20:193
2645099120,11cyclictest7484-21diskmemload00:25:483
2645099120,11cyclictest0-21swapper/323:40:103
2645099120,11cyclictest0-21swapper/323:20:543
2645099120,11cyclictest0-21swapper/322:36:353
2645099120,11cyclictest0-21swapper/322:26:493
2645099120,11cyclictest0-21swapper/320:50:223
2645099120,11cyclictest0-21swapper/320:45:273
2645099120,11cyclictest0-21swapper/320:40:253
2645099120,11cyclictest0-21swapper/300:34:293
2644999129,2cyclictest7484-21diskmemload23:50:202
2644999129,2cyclictest0-21swapper/222:31:012
2644999129,2cyclictest0-21swapper/222:05:142
2644999129,2cyclictest0-21swapper/221:20:382
2644999129,2cyclictest0-21swapper/220:55:192
2644999129,2cyclictest0-21swapper/220:15:212
2644999129,2cyclictest0-21swapper/220:10:112
2644999129,2cyclictest0-21swapper/219:55:192
2644999129,2cyclictest0-21swapper/200:36:382
2644999122,7cyclictest0-21swapper/220:36:592
2644999121,6cyclictest0-21swapper/220:10:002
2644999120,2cyclictest0-21swapper/222:45:542
2644999120,11cyclictest0-21swapper/223:15:172
2644999120,11cyclictest0-21swapper/221:50:202
2644999120,11cyclictest0-21swapper/221:46:162
2644999120,11cyclictest0-21swapper/220:30:152
2644899129,2cyclictest0-21swapper/119:50:291
2644899127,3cyclictest5693-21munin-node1
2644899127,3cyclictest12525-21munin-node1
2644899121,9cyclictest19190-21munin-node1
2644899120,3cyclictest0-21swapper/120:00:251
2644899120,11cyclictest18579-21ssh21:55:341
2644899120,11cyclictest0-21swapper/122:45:341
2644899120,11cyclictest0-21swapper/121:45:031
2644899120,11cyclictest0-21swapper/121:15:181
2644899120,11cyclictest0-21swapper/120:50:211
2644899120,11cyclictest0-21swapper/119:45:191
2644899120,11cyclictest0-21swapper/119:40:231
2644899120,11cyclictest0-21swapper/100:30:061
2644799129,2cyclictest28552-21ssh00:30:130
2644799129,2cyclictest0-21swapper/021:26:400
2644799129,2cyclictest0-21swapper/020:25:000
2644799122,6cyclictest0-21swapper/020:20:250
2644799122,6cyclictest0-21swapper/019:25:010
2644799120,2cyclictest0-21swapper/021:45:480
2644799120,2cyclictest0-21swapper/021:10:320
2644799120,2cyclictest0-21swapper/020:35:130
2644799120,2cyclictest0-21swapper/019:30:200
2644799120,11cyclictest0-21swapper/023:56:420
2644799120,11cyclictest0-21swapper/023:28:100
2644799120,11cyclictest0-21swapper/023:16:410
2644799120,11cyclictest0-21swapper/020:50:200
2645099118,2cyclictest0-21swapper/320:15:263
2645099118,2cyclictest0-21swapper/320:10:243
2645099118,2cyclictest0-21swapper/319:30:163
2645099112,6cyclictest0-21swapper/319:55:583
2645099111,7cyclictest0-21swapper/319:41:583
2645099110,10cyclictest0-21swapper/322:56:573
2644999116,3cyclictest0-21swapper/219:25:142
2644999110,2cyclictest0-21swapper/219:15:212
2644899118,2cyclictest1177-21kworker/u8:120:55:051
2644899116,3cyclictest32731-21munin-node1
2644899110,9cyclictest0-21swapper/120:40:121
2644899110,9cyclictest0-21swapper/119:30:151
2644899110,3cyclictest17180-21munin-node1
2644899110,3cyclictest0-21swapper/120:45:131
2644799118,2cyclictest8502-21kworker/u8:019:51:050
2644799118,2cyclictest0-21swapper/020:15:260
2644799116,3cyclictest86722sleep019:45:140
2644799116,3cyclictest4914-21cut19:35:150
2644799110,9cyclictest0-21swapper/020:55:260
2644799110,2cyclictest0-21swapper/020:30:140
2644799110,10cyclictest0-21swapper/021:00:120
2645099102,7cyclictest0-21swapper/320:31:593
2645099102,5cyclictest0-21swapper/320:09:593
2645099100,9cyclictest0-21swapper/319:50:023
2645099100,7cyclictest0-21swapper/319:29:593
2644999107,2cyclictest0-21swapper/220:20:122
2644999102,5cyclictest0-21swapper/219:20:582
2644999100,9cyclictest0-21swapper/220:50:232
2644999100,4cyclictest0-21swapper/219:50:012
2644799107,2cyclictest26664-21apt-get19:10:130
2644799105,3cyclictest5854-21telnet21:05:140
2644799105,3cyclictest28734-21ntp_kernel_pll_20:40:190
2644799100,7cyclictest0-21swapper/020:49:580
2644799100,7cyclictest0-21swapper/019:15:010
264509996,2cyclictest0-21swapper/320:00:173
264509990,6cyclictest0-21swapper/321:01:593
264509990,6cyclictest0-21swapper/319:11:593
264499994,3cyclictest3745-21runrttasks20:26:192
264499991,3cyclictest6249-21cron19:40:002
264489996,2cyclictest1177-21kworker/u8:121:00:051
264489994,3cyclictest23316-21sshd20:26:231
264479996,2cyclictest0-21swapper/020:10:100
264509984,2cyclictest0-21swapper/320:55:043
264499984,2cyclictest4786-21switchtime21:00:252
264499983,3cyclictest10555-21aten2.4-expect19:50:182
264499981,5cyclictest3222-21ntpq19:30:212
264489983,3cyclictest3745-21runrttasks20:09:181
264479983,3cyclictest3745-21runrttasks20:07:120
264479983,3cyclictest15760-21sed20:00:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional