You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Thu Apr 03, 2025 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
140189911953,35cyclictest0-21swapper/023:55:230
309912870,4sleep30-21swapper/323:35:083
1401899871,68cyclictest2955-21rm22:10:090
14018998621,46cyclictest17421-21munin-run19:20:080
1401899841,61cyclictest19722-21wget20:55:220
1401899809,48cyclictest0-21swapper/023:14:290
14018997718,47cyclictest8050irq/56-dwc_otg_22:10:300
1401899771,12cyclictest0-21swapper/023:25:450
256142760,5sleep31402399cyclictest22:40:293
1401899766,15cyclictest0-21swapper/021:50:180
14018997514,47cyclictest0-21swapper/021:20:230
14018997514,47cyclictest0-21swapper/021:20:230
14018997416,5cyclictest6017-21ssh23:44:110
14018997414,10cyclictest17119-21/usr/sbin/munin00:00:260
1401899736,38cyclictest0-21swapper/022:00:270
1401899731,65cyclictest0-21swapper/019:20:250
14018997312,47cyclictest9048-21ssh22:17:110
144072720,1sleep30-21swapper/321:39:533
1401899723,52cyclictest0-21swapper/019:10:110
14018997218,4cyclictest19572-21aten_rbpower_vo19:25:230
1401899721,5cyclictest8050irq/56-dwc_otg_20:00:210
1401899721,5cyclictest8050irq/56-dwc_otg_20:00:210
14018997017,40cyclictest0-21swapper/000:32:420
14018997017,40cyclictest0-21swapper/000:32:420
1401899701,58cyclictest0-21swapper/023:33:340
1401899701,20cyclictest21060-21basename21:50:080
1401899701,14cyclictest0-21swapper/022:55:270
60702690,1sleep10-21swapper/121:28:521
60702690,1sleep10-21swapper/121:28:521
1402299696,16cyclictest7950irq/56-dwc_otg_23:10:172
14018996920,5cyclictest31858-21latency_hist20:00:090
14018996913,9cyclictest0-21swapper/019:30:380
14023996812,25cyclictest8050irq/56-dwc_otg_21:15:203
14018996814,9cyclictest1-21systemd21:37:090
1401899676,42cyclictest0-21swapper/023:15:200
14018996713,4cyclictest16593-21ssh22:29:070
86992660,2sleep30-21swapper/321:31:083
1401899669,3cyclictest7950irq/56-dwc_otg_21:59:070
1401899662,29cyclictest0-21swapper/023:20:200
1401899662,18cyclictest0-21swapper/022:48:160
14018996519,4cyclictest7950irq/56-dwc_otg_23:52:210
14018996512,9cyclictest0-21swapper/020:05:330
1401899651,20cyclictest0-21swapper/020:50:090
1401899651,17cyclictest0-21swapper/000:05:470
1401899651,12cyclictest0-21swapper/021:42:390
14018996510,11cyclictest0-21swapper/023:01:250
1401899649,17cyclictest0-21swapper/020:15:210
1401899646,16cyclictest0-21swapper/000:10:170
14018996437,7cyclictest7950irq/56-dwc_otg_19:50:250
1401899642,51cyclictest0-21swapper/022:40:100
1401899642,17cyclictest0-21swapper/020:30:360
14018996417,4cyclictest7850irq/56-dwc_otg00:20:330
14018996416,12cyclictest762350ssh00:35:230
14018996412,10cyclictest12629-21ssh23:09:310
14018996412,10cyclictest12629-21ssh23:09:310
1401899641,20cyclictest17636-21kworker/u8:322:20:320
1401899633,4cyclictest0-21swapper/021:30:090
1401899633,4cyclictest0-21swapper/021:30:090
14018996321,37cyclictest22507-21unixbench_singl22:35:370
14018996314,9cyclictest0-21swapper/023:38:370
14018996313,9cyclictest0-21swapper/020:38:400
1401899631,34cyclictest0-21swapper/021:10:240
1401899631,18cyclictest0-21swapper/023:47:180
1401899631,18cyclictest0-21swapper/023:47:180
14018996214,40cyclictest0-21swapper/022:53:200
14018996213,3cyclictest0-21swapper/000:30:000
1401899621,12cyclictest0-21swapper/022:30:280
1401899621,12cyclictest0-21swapper/022:30:280
1402399618,17cyclictest8050irq/56-dwc_otg_23:55:233
1402199618,14cyclictest20951-21ssh23:20:141
1402199613,43cyclictest0-21swapper/122:40:221
1401899605,4cyclictest0-21swapper/000:15:220
95052590,2sleep00-21swapper/021:32:460
14023995920,11cyclictest7850irq/56-dwc_otg00:15:223
1401899592,20cyclictest1131-21runrttasks21:17:350
1401899591,18cyclictest0-21swapper/019:41:330
1402299577,22cyclictest0-21swapper/222:52:392
1402299575,16cyclictest0-21swapper/200:35:212
1402299573,9cyclictest0-21swapper/221:05:212
1401899579,9cyclictest0-21swapper/021:03:450
1401899572,22cyclictest0-21swapper/021:09:180
62042560,4sleep231-21ksoftirqd/222:14:232
1402299567,6cyclictest7950irq/56-dwc_otg_22:30:182
1402299567,6cyclictest7950irq/56-dwc_otg_22:30:182
1401899568,4cyclictest7950irq/56-dwc_otg_20:25:270
1401899562,18cyclictest0-21swapper/019:38:520
1401899561,20cyclictest0-21swapper/019:49:190
1401899561,12cyclictest0-21swapper/020:25:040
1401899561,12cyclictest0-21swapper/020:25:040
136552563,12sleep10-21swapper/119:08:561
1402299552,13cyclictest7850irq/56-dwc_otg19:10:202
14018995422,5cyclictest0-21swapper/020:40:210
1401899531,16cyclictest0-21swapper/020:50:090
1401899531,16cyclictest0-21swapper/020:50:090
14018995310,8cyclictest0-21swapper/020:10:560
14021995222,16cyclictest8050irq/56-dwc_otg_19:10:111
1402299515,12cyclictest7850irq/56-dwc_otg22:00:242
1402299505,18cyclictest8050irq/56-dwc_otg_21:55:212
136592503,12sleep00-21swapper/019:09:000
1402399494,4cyclictest7950irq/56-dwc_otg_22:15:223
14023994915,9cyclictest8050irq/56-dwc_otg_21:15:083
1402199497,27cyclictest0-21swapper/120:40:131
1402399482,7cyclictest7950irq/56-dwc_otg_23:15:183
1402199473,14cyclictest0-21swapper/120:30:241
1402399467,24cyclictest0-21swapper/323:00:253
1402399463,30cyclictest0-21swapper/319:10:113
14023994614,15cyclictest8050irq/56-dwc_otg_00:35:403
1402199467,24cyclictest0-21swapper/119:50:251
1402199466,10cyclictest7950irq/56-dwc_otg_23:35:201
1402399457,24cyclictest0-21swapper/323:35:223
14023994521,8cyclictest8050irq/56-dwc_otg_23:41:533
1402299457,24cyclictest0-21swapper/219:45:232
1402199453,35cyclictest8050irq/56-dwc_otg_20:20:201
1402199453,35cyclictest8050irq/56-dwc_otg_20:20:201
1402399447,23cyclictest0-21swapper/320:15:233
14023994414,12cyclictest8050irq/56-dwc_otg_23:20:283
14023994411,11cyclictest8050irq/56-dwc_otg_00:21:543
1402299448,23cyclictest0-21swapper/220:35:222
1402299447,24cyclictest0-21swapper/220:00:212
1402299447,24cyclictest0-21swapper/220:00:212
1402199446,25cyclictest0-21swapper/120:05:231
1402199443,10cyclictest13316-21rm23:55:211
1402399438,12cyclictest21921-21nfs4_client19:30:333
1402399437,23cyclictest0-21swapper/320:10:243
1402399437,23cyclictest0-21swapper/319:15:253
1402399436,21cyclictest24140-21runrttasks21:53:403
14023994311,17cyclictest8050irq/56-dwc_otg_20:05:303
14023994311,16cyclictest32273-21df20:00:233
14023994311,16cyclictest32273-21df20:00:233
14021994317,6cyclictest23-21ksoftirqd/121:20:081
1402399427,23cyclictest0-21swapper/320:40:133
1402399427,23cyclictest0-21swapper/319:50:253
1402399427,21cyclictest0-21swapper/319:35:233
1402399425,24cyclictest23543-21ssh00:10:143
14023994223,7cyclictest16810-21ssh23:15:083
14023994219,11cyclictest8050irq/56-dwc_otg_00:30:173
14023994219,11cyclictest8050irq/56-dwc_otg_00:30:173
14023994215,13cyclictest8050irq/56-dwc_otg_22:01:523
14023994212,9cyclictest8050irq/56-dwc_otg_20:30:333
1402299427,21cyclictest0-21swapper/220:40:232
1402299427,21cyclictest0-21swapper/219:40:222
14022994216,9cyclictest0-21swapper/200:00:292
14022994215,5cyclictest23245-21apt-key00:10:092
1402299421,30cyclictest0-21swapper/223:10:092
1402299421,30cyclictest0-21swapper/223:10:092
1402199427,22cyclictest8050irq/56-dwc_otg_21:35:091
1402199427,22cyclictest0-21swapper/120:15:231
1402199422,13cyclictest0-21swapper/120:45:281
14021994213,9cyclictest22477-21unixbench_multi22:35:361
1402399417,21cyclictest0-21swapper/320:20:223
1402399417,21cyclictest0-21swapper/320:20:223
14022994122,5cyclictest7850irq/56-dwc_otg00:20:232
1402199412,16cyclictest8050irq/56-dwc_otg_21:00:201
14021994120,13cyclictest221ktimersoftd/100:15:331
129552410,1sleep10-21swapper/121:36:581
267662401,34sleep10-21swapper/121:56:111
1402399407,18cyclictest8050irq/56-dwc_otg_21:56:243
1402399406,20cyclictest18453-21ssh00:00:413
1402399404,19cyclictest426-21ssh21:20:233
1402399404,19cyclictest426-21ssh21:20:233
1402399402,19cyclictest7950irq/56-dwc_otg_22:07:423
14023994020,9cyclictest8050irq/56-dwc_otg_22:21:003
14023994018,9cyclictest8050irq/56-dwc_otg_21:40:143
14023994018,5cyclictest7950irq/56-dwc_otg_21:28:413
14023994018,5cyclictest7950irq/56-dwc_otg_21:28:413
14023994017,8cyclictest8050irq/56-dwc_otg_20:35:493
14023994013,13cyclictest381ktimersoftd/320:30:083
14023994012,14cyclictest8050irq/56-dwc_otg_22:10:263
14023994011,12cyclictest18830-21sshd21:45:383
1402299401,4cyclictest8050irq/56-dwc_otg_23:50:152
14022994011,19cyclictest0-21swapper/221:25:142
14022994011,19cyclictest0-21swapper/221:25:142
1402399396,13cyclictest8050irq/56-dwc_otg_20:00:083
1402399393,25cyclictest19687-21chrt20:55:213
14023993924,12cyclictest31935-21fschecks_time22:50:213
14023993913,10cyclictest8050irq/56-dwc_otg_19:20:233
1402399391,20cyclictest0-21swapper/322:30:093
1402199399,6cyclictest23-21ksoftirqd/100:30:081
1402199399,21cyclictest1469-21ssh22:52:391
1402199398,15cyclictest7950irq/56-dwc_otg_21:55:091
1402199396,22cyclictest23601-21ssh00:10:181
1402199396,15cyclictest7950irq/56-dwc_otg_19:40:291
121922396,23sleep212204-21/usr/sbin/munin19:05:212
14022993820,4cyclictest29-21rcuc/221:30:372
14022993811,6cyclictest5210-21unixbench_multi00:30:332
14022993811,6cyclictest5210-21unixbench_multi00:30:332
1402199384,19cyclictest7850irq/56-dwc_otg22:15:261
1402199381,3cyclictest0-21swapper/122:35:091
1402199381,3cyclictest0-21swapper/122:35:091
1402199381,25cyclictest30801-21ls19:55:261
308772370,2sleep10-21swapper/123:34:541
1402399377,15cyclictest8050irq/56-dwc_otg_23:06:463
1402399377,15cyclictest8050irq/56-dwc_otg_23:06:463
1402299374,4cyclictest5304-21fschecks_count20:15:272
1402299372,8cyclictest13585-21ssh23:55:272
1402299372,19cyclictest21350-21ssh23:20:272
14022993713,10cyclictest0-21swapper/223:30:362
1402199379,6cyclictest7850irq/56-dwc_otg21:10:271
1402199372,17cyclictest7950irq/56-dwc_otg_19:45:231
14021993720,4cyclictest23-21ksoftirqd/119:25:251
14021993717,5cyclictest1131-21runrttasks23:18:221
14021993713,11cyclictest9650irq/80-mmc020:40:091
14021993710,6cyclictest0-21swapper/121:23:391
14021993710,6cyclictest0-21swapper/121:23:391
36512360,2sleep20-21swapper/200:29:582
1402399369,15cyclictest11381-21ssh23:51:493
1402399369,11cyclictest8050irq/56-dwc_otg_00:08:273
1402399368,16cyclictest30187-21ssh22:47:543
1402399366,16cyclictest8050irq/56-dwc_otg_23:46:573
1402399366,16cyclictest8050irq/56-dwc_otg_23:46:573
1402399364,19cyclictest19253-21taskset20:52:523
1402399364,19cyclictest19253-21taskset20:52:523
1402399364,16cyclictest19865-21ssh22:32:543
1402399364,16cyclictest19865-21ssh22:32:543
14023993615,4cyclictest7950irq/56-dwc_otg_22:58:393
14023993615,4cyclictest7950irq/56-dwc_otg_19:41:103
14023993614,4cyclictest8050irq/56-dwc_otg_22:37:283
1402299366,5cyclictest7850irq/56-dwc_otg20:50:292
1402299366,5cyclictest7850irq/56-dwc_otg20:50:292
1402299365,8cyclictest7950irq/56-dwc_otg_23:40:082
14022993620,10cyclictest23562-21/usr/sbin/munin00:10:302
1402299361,24cyclictest9081-21cut20:25:272
1402199367,24cyclictest0-21swapper/100:35:241
14021993618,6cyclictest32409-21fschecks_time20:00:261
14021993618,6cyclictest32409-21fschecks_time20:00:261
14021993617,6cyclictest16790-21latency_hist23:15:081
14023993510,10cyclictest8050irq/56-dwc_otg_23:28:343
1402299357,6cyclictest4286-21grep22:55:352
1402299352,15cyclictest7850irq/56-dwc_otg21:10:322
1402299351,23cyclictest21719-21cut21:00:252
14022993511,4cyclictest0-21swapper/223:15:352
1402199358,6cyclictest0-21swapper/100:25:081
1402199358,13cyclictest8050irq/56-dwc_otg_23:50:321
1402199352,8cyclictest32548-21apt22:05:081
14021993517,5cyclictest309150wget23:40:241
1402199351,4cyclictest0-21swapper/122:10:291
1402399347,4cyclictest0-21swapper/319:45:373
1402299346,14cyclictest8050irq/56-dwc_otg_00:15:232
1402299345,17cyclictest21395-21aten_rbpower_en21:50:182
1402299342,25cyclictest0-21swapper/222:49:592
1402299342,21cyclictest7013-21ssh23:00:272
1402299342,14cyclictest0-21swapper/220:30:402
14022993414,4cyclictest7042-21aten_rbpower_po20:20:232
14022993414,4cyclictest7042-21aten_rbpower_po20:20:232
1402199348,14cyclictest0-21swapper/121:05:221
1402199347,14cyclictest7950irq/56-dwc_otg_23:00:081
1402199345,6cyclictest8121-21unixbench_singl23:00:411
14021993415,4cyclictest23-21ksoftirqd/122:09:571
14021993412,4cyclictest5653-21sh00:31:211
14021993412,4cyclictest5653-21sh00:31:211
14021993412,10cyclictest7950irq/56-dwc_otg_20:58:211
1402399333,4cyclictest7850irq/56-dwc_otg21:00:203
1402399332,12cyclictest19460-21awk19:25:203
1402399331,11cyclictest0-21swapper/321:05:213
1402299338,17cyclictest8050irq/56-dwc_otg_23:40:132
1402299336,12cyclictest8050irq/56-dwc_otg_19:55:242
1402299335,7cyclictest7950irq/56-dwc_otg_21:40:382
1402299331,23cyclictest17113-21latency_hist22:30:092
14022993310,10cyclictest0-21swapper/223:48:352
14022993310,10cyclictest0-21swapper/223:48:352
1402199337,5cyclictest7850irq/56-dwc_otg20:25:371
1402199336,7cyclictest14443-21modprobe22:25:081
1402199334,5cyclictest7850irq/56-dwc_otg19:20:261
1402199334,4cyclictest7850irq/56-dwc_otg19:30:251
14021993319,8cyclictest21-21rcuc/123:06:541
14021993319,8cyclictest21-21rcuc/123:06:541
1402199331,21cyclictest17439-21grep00:00:281
1402199331,20cyclictest0-21swapper/123:45:251
1402199331,20cyclictest0-21swapper/123:45:251
253122320,2sleep10-21swapper/123:26:201
1402399328,5cyclictest2506-21ssh00:27:163
1402299327,15cyclictest0-21swapper/220:55:222
1402299326,6cyclictest7850irq/56-dwc_otg20:45:292
14022993216,4cyclictest16286-21if_wlan019:15:302
1402299321,22cyclictest11631-21ssh21:35:302
1402199329,12cyclictest0-21swapper/122:46:281
1402199323,23cyclictest0-21swapper/121:49:541
1402199321,24cyclictest15263-21ssh21:40:281
14021993210,4cyclictest0-21swapper/100:05:291
1402299316,11cyclictest8050irq/56-dwc_otg_21:15:312
14022993122,6cyclictest19224-21munin-plugin-st19:25:082
14022993111,4cyclictest0-21swapper/222:35:482
1402199311,20cyclictest0-21swapper/119:18:091
1402199311,10cyclictest23327-21cut19:35:271
1402399308,9cyclictest8050irq/56-dwc_otg_20:50:093
1402299307,4cyclictest0-21swapper/223:28:132
1402299303,4cyclictest7950irq/56-dwc_otg_22:08:092
1402299303,12cyclictest7850irq/56-dwc_otg21:20:292
1402299303,12cyclictest7850irq/56-dwc_otg21:20:292
1402299302,5cyclictest7850irq/56-dwc_otg22:20:372
14022993015,5cyclictest27592-21sh22:42:582
1402299301,13cyclictest7850irq/56-dwc_otg20:12:402
14022993010,4cyclictest1949350wget19:25:232
1402199307,4cyclictest0-21swapper/122:28:481
1402199300,22cyclictest7850irq/56-dwc_otg20:15:081
1402299298,6cyclictest2522-21grep20:05:372
1402299298,4cyclictest0-21swapper/222:15:392
1402299296,12cyclictest15658-1kworker/u9:121:45:392
1402299295,11cyclictest8050irq/56-dwc_otg_19:50:332
1402199284,5cyclictest23-21ksoftirqd/120:55:091
1402199284,5cyclictest23-21ksoftirqd/120:55:091
1402299274,9cyclictest7850irq/56-dwc_otg19:35:282
1402299261,15cyclictest21257-21/usr/sbin/munin19:30:332
121832253,14sleep30-21swapper/319:05:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional