You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 04:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 02, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
616521740,5sleep12789899cyclictest21:57:391
120802950,2sleep311905-21latency_hist00:25:093
2789499936,52cyclictest0-21swapper/020:45:220
2789499855,42cyclictest11971-21ssh21:20:200
13872850,2sleep20-21swapper/221:50:362
13872850,2sleep20-21swapper/221:50:362
54942800,2sleep20-21swapper/223:30:082
27894997927,22cyclictest0-21swapper/023:45:160
27894997425,9cyclictest0-21swapper/019:10:210
27894997315,35cyclictest31043-21idleruntime-cro00:05:070
2789499726,9cyclictest13-21rcuc/020:10:180
27894997214,10cyclictest0-21swapper/022:20:190
27894996931,9cyclictest0-21swapper/021:12:440
27894996813,12cyclictest0-21swapper/020:20:220
2790199665,45cyclictest0-21swapper/223:10:142
2790199665,45cyclictest0-21swapper/223:10:142
2790199653,43cyclictest31193-21/usr/sbin/munin20:55:332
2790199652,6cyclictest5680-21apt-get23:30:172
2789499652,43cyclictest0-21swapper/019:20:210
2790199641,4cyclictest1158-21snmpd23:19:302
2790199641,45cyclictest3427-21/usr/sbin/munin22:40:342
2790199641,45cyclictest3427-21/usr/sbin/munin22:40:342
2790199633,4cyclictest11951-21/usr/sbin/munin21:20:382
2790199625,38cyclictest7850irq/56-dwc_otg20:35:332
2790199616,43cyclictest8050irq/56-dwc_otg_21:42:572
2790199615,42cyclictest8050irq/56-dwc_otg_00:12:232
2790199612,4cyclictest7950irq/56-dwc_otg_22:50:292
2790199612,3cyclictest7850irq/56-dwc_otg22:10:182
27898996114,21cyclictest7950irq/56-dwc_otg_00:30:201
2789499613,12cyclictest17956-21ssh22:15:190
2789499613,12cyclictest17956-21ssh22:15:190
27894996118,3cyclictest0-21swapper/022:00:190
2789499608,7cyclictest4714-21grep22:40:370
2789499608,7cyclictest4714-21grep22:40:370
27894996018,22cyclictest32248-21sh22:35:230
27894996011,27cyclictest0-21swapper/019:55:260
2790199594,32cyclictest0-21swapper/200:00:072
2790199592,5cyclictest7950irq/56-dwc_otg_00:15:372
27901995918,38cyclictest21022-21latency_hist23:05:092
2790199591,3cyclictest0-21swapper/222:02:292
2789499596,32cyclictest0-21swapper/020:00:230
27894995917,3cyclictest0-21swapper/000:15:220
2789499591,34cyclictest21101-21latency_hist00:40:080
2790199586,33cyclictest0-21swapper/200:35:372
2790199584,38cyclictest8050irq/56-dwc_otg_21:59:492
2790199581,6cyclictest29203-21latency_hist20:50:082
27901995812,36cyclictest15174-21latency_hist20:05:082
2789499581,31cyclictest29530-21wget20:50:290
2789499581,31cyclictest29530-21wget20:50:290
2790199572,4cyclictest0-21swapper/219:40:332
2790199572,4cyclictest0-21swapper/219:40:332
2790199571,4cyclictest7950irq/56-dwc_otg_23:35:342
27894995727,5cyclictest0-21swapper/019:15:360
2789499572,51cyclictest0-21swapper/022:10:310
27894995724,6cyclictest29144-21head23:15:380
2789499572,34cyclictest2892-21latency_hist23:25:080
297792560,2sleep10-21swapper/123:16:541
2790199564,38cyclictest8050irq/56-dwc_otg_00:23:532
2790199562,39cyclictest23304-21sh22:21:052
27901995617,36cyclictest15167-21if_wlan000:30:282
2790199561,45cyclictest25778-21latency_hist21:40:082
2790199561,38cyclictest8050irq/56-dwc_otg_19:35:252
27894995635,10cyclictest8050irq/56-dwc_otg_21:50:250
27894995635,10cyclictest8050irq/56-dwc_otg_21:50:250
27894995626,6cyclictest22565-21cut21:35:270
27894995615,22cyclictest0-21swapper/020:15:230
137302560,2sleep30-21swapper/321:21:343
2790199553,39cyclictest8050irq/56-dwc_otg_22:37:362
2790199552,3cyclictest0-21swapper/220:50:262
2790199552,3cyclictest0-21swapper/220:50:262
2790199551,7cyclictest19263-21rm22:15:382
2790199551,7cyclictest19263-21rm22:15:382
2790199551,4cyclictest0-21swapper/221:46:442
2790199551,41cyclictest0-21swapper/200:29:032
2789499557,45cyclictest0-21swapper/023:30:170
2789499552,40cyclictest0-21swapper/019:30:300
2789499552,37cyclictest0-21swapper/019:35:190
2789499552,33cyclictest18518-21ssh21:30:240
2789499551,6cyclictest14700-21grep23:40:350
2789499551,28cyclictest26490-21/usr/sbin/munin21:40:270
2790199546,28cyclictest0-21swapper/221:10:352
2790199543,47cyclictest13430-21ssh23:40:172
2790199541,4cyclictest0-21swapper/223:54:132
2790199541,42cyclictest11413-21ssh22:05:112
27901995412,39cyclictest3457-21proc_pri19:30:362
27898995439,12cyclictest11990-21seq21:20:221
2789499542,30cyclictest0-21swapper/021:05:370
2789499541,49cyclictest0-21swapper/020:30:080
2789499541,33cyclictest10236-21fschecks_time19:50:310
27894995413,31cyclictest29887-21ssh21:45:240
2790199534,37cyclictest8050irq/56-dwc_otg_22:48:102
2790199532,48cyclictest12324-21missed_timers19:55:302
2790199531,3cyclictest8050irq/56-dwc_otg_23:45:552
2790199531,3cyclictest0-21swapper/222:26:052
2789899535,34cyclictest0-21swapper/120:20:191
27894995326,7cyclictest14960-21cut00:30:260
2789499532,43cyclictest0-21swapper/022:30:010
2789499532,18cyclictest0-21swapper/023:55:090
2790299524,6cyclictest8050irq/56-dwc_otg_22:30:163
2790199526,36cyclictest27285-21taskset20:44:232
2790199523,42cyclictest8050irq/56-dwc_otg_22:58:242
2790199522,38cyclictest0-21swapper/221:25:232
2790199521,3cyclictest7850irq/56-dwc_otg23:25:052
2790199521,3cyclictest0-21swapper/223:05:262
2789899527,29cyclictest7850irq/56-dwc_otg21:35:231
2789499523,9cyclictest0-21swapper/022:30:320
2789499521,27cyclictest0-21swapper/023:10:260
2789499521,27cyclictest0-21swapper/023:10:260
2789499521,16cyclictest18152-21ls21:30:080
2790199516,37cyclictest8050irq/56-dwc_otg_20:30:152
2790199512,45cyclictest30816-21unixbench_multi19:15:362
2790199511,6cyclictest1172-21fschecks_count19:25:262
2790199511,5cyclictest20686-21sed20:20:362
2790199511,4cyclictest8050irq/56-dwc_otg_21:15:232
2790199511,4cyclictest18399-21apt-get21:30:202
2790199511,42cyclictest0-21swapper/219:22:022
2790199511,3cyclictest0-21swapper/220:18:142
2789499512,27cyclictest0-21swapper/021:59:110
27894995122,5cyclictest1692-21sh00:10:230
27902995025,4cyclictest8050irq/56-dwc_otg_21:10:233
2790199502,5cyclictest15934-21/usr/sbin/munin20:10:252
2790199502,43cyclictest28195-21/usr/sbin/munin22:30:152
27901995023,4cyclictest28550-21sh00:00:342
27901995011,37cyclictest28469-21sed19:10:292
27894995023,16cyclictest9-21ksoftirqd/000:20:360
27894995016,17cyclictest14090-21fschecks_time22:55:260
27894995012,15cyclictest19175-21sh23:01:160
27901994918,29cyclictest2649-21fschecks_count21:05:262
2790199491,3cyclictest0-21swapper/220:29:132
2789499497,27cyclictest13-21rcuc/020:40:550
2789499492,27cyclictest1131-21runrttasks22:07:150
27894994916,17cyclictest23543-21sh23:09:430
2789499491,29cyclictest0-21swapper/020:55:330
2789499491,28cyclictest1486-21timerandwakeup21:00:350
2790199482,5cyclictest0-21swapper/200:07:322
2790199482,3cyclictest553-21apt-get21:00:202
2790199481,3cyclictest0-21swapper/219:50:402
2790199481,37cyclictest8050irq/56-dwc_otg_20:05:302
2789899487,26cyclictest27748-21ssh00:00:231
2789899486,12cyclictest11-21rcu_preempt00:15:211
2789499482,7cyclictest9282-21grep19:45:360
27894994819,5cyclictest0-21swapper/022:50:150
27894994811,19cyclictest9-21ksoftirqd/023:35:350
27894994810,25cyclictest0-21swapper/019:40:200
27894994810,25cyclictest0-21swapper/019:40:200
269352480,2sleep00-21swapper/019:05:350
2789899475,30cyclictest7850irq/56-dwc_otg00:20:381
2789899474,29cyclictest7850irq/56-dwc_otg00:10:151
2789499472,23cyclictest0-21swapper/020:40:070
2789499471,30cyclictest24713-21proc_pri23:55:330
241642470,2sleep30-21swapper/322:22:543
97752460,13sleep0101ktimersoftd/022:50:080
27902994629,9cyclictest8050irq/56-dwc_otg_20:30:153
27902994616,25cyclictest0-21swapper/321:00:223
27898994612,25cyclictest0-21swapper/119:15:221
2789499462,22cyclictest23663-21awk20:35:070
2790299458,21cyclictest0-21swapper/322:10:183
2790299453,20cyclictest8050irq/56-dwc_otg_19:40:313
2790299453,20cyclictest8050irq/56-dwc_otg_19:40:313
27902994518,4cyclictest13501-21latency_hist22:55:083
2790199452,3cyclictest0-21swapper/219:48:472
2789899451,3cyclictest5692-21ssh23:30:151
27894994529,4cyclictest8050irq/56-dwc_otg_21:15:250
2790299447,22cyclictest0-21swapper/323:10:173
2790299447,22cyclictest0-21swapper/323:10:173
2790299442,8cyclictest0-21swapper/322:40:203
2790299442,8cyclictest0-21swapper/322:40:203
2790299441,9cyclictest7950irq/56-dwc_otg_19:45:193
2789899447,24cyclictest0-21swapper/120:35:211
27894994424,3cyclictest0-21swapper/000:10:080
2789499442,26cyclictest0-21swapper/023:29:240
201452440,5sleep32790299cyclictest23:03:203
193012440,3sleep123-21ksoftirqd/100:35:421
2790299437,23cyclictest0-21swapper/321:55:233
27902994323,9cyclictest7950irq/56-dwc_otg_21:31:453
2789899437,23cyclictest0-21swapper/120:30:141
2789899431,10cyclictest0-21swapper/120:50:251
2789899431,10cyclictest0-21swapper/120:50:251
27894994315,23cyclictest0-21swapper/019:27:510
2789499431,31cyclictest0-21swapper/000:28:130
2790299427,23cyclictest0-21swapper/323:05:143
2790299427,14cyclictest25782-21ssh21:40:083
2790299426,22cyclictest0-21swapper/321:50:173
2790299426,22cyclictest0-21swapper/321:50:173
27898994224,10cyclictest29567-21latency_hist21:45:081
2790299417,21cyclictest0-21swapper/323:20:243
2789899417,21cyclictest7850irq/56-dwc_otg22:30:321
265942410,2sleep30-21swapper/322:27:243
2790299409,15cyclictest9702-21ssh22:50:083
2790299408,20cyclictest18088-21aten_rbpower_vo22:15:223
2790299408,20cyclictest18088-21aten_rbpower_vo22:15:223
2790299403,12cyclictest3211-21cut22:40:083
27902994029,7cyclictest381ktimersoftd/300:15:233
27902994021,8cyclictest7950irq/56-dwc_otg_21:40:293
2789899408,14cyclictest11-21rcu_preempt20:10:181
2790299397,24cyclictest0-21swapper/320:15:233
27902993910,4cyclictest8050irq/56-dwc_otg_23:40:183
2789899399,22cyclictest7850irq/56-dwc_otg20:30:081
2789899398,8cyclictest7950irq/56-dwc_otg_21:00:201
2789899397,24cyclictest7950irq/56-dwc_otg_21:10:071
2789899393,13cyclictest0-21swapper/122:50:161
2789899391,20cyclictest27742-21ls23:15:091
2789899391,20cyclictest27742-21ls23:15:091
27894993911,15cyclictest15667-21taskset20:09:480
2790299388,5cyclictest7950irq/56-dwc_otg_21:15:263
2790299387,22cyclictest0-21swapper/319:55:223
2790299387,19cyclictest0-21swapper/323:55:223
2790299387,12cyclictest6597-21sensors23:30:293
2790299385,15cyclictest8050irq/56-dwc_otg_19:10:303
2790299384,20cyclictest32300-21ssh00:07:303
2790299384,19cyclictest17619-21seq00:34:443
27902993819,8cyclictest7950irq/56-dwc_otg_22:55:423
27902993816,14cyclictest7950irq/56-dwc_otg_21:29:003
2790299381,26cyclictest7950irq/56-dwc_otg_00:35:303
2789899389,13cyclictest8050irq/56-dwc_otg_19:40:081
2789899387,18cyclictest11846-21latency_hist21:20:091
2789899384,27cyclictest7950irq/56-dwc_otg_22:55:271
2789899382,23cyclictest7850irq/56-dwc_otg22:40:261
2789899382,23cyclictest7850irq/56-dwc_otg22:40:261
2789899381,9cyclictest7850irq/56-dwc_otg19:20:211
27898993818,6cyclictest17938-21sort23:00:261
27898993816,11cyclictest11754-21aten_rbpower_cu19:55:221
2789899381,21cyclictest7850irq/56-dwc_otg22:15:081
2790299379,12cyclictest31923-21open_files20:55:323
2790299377,5cyclictest0-21swapper/300:10:243
2790299373,22cyclictest0-21swapper/319:20:223
2790299373,20cyclictest9573-21/usr/sbin/munin23:35:323
27902993724,4cyclictest8050irq/56-dwc_otg_20:35:293
2789899375,21cyclictest7269-21ssh22:45:141
2789899372,21cyclictest7850irq/56-dwc_otg20:10:081
2789899371,9cyclictest1132-21fschecks_count19:25:251
27898993718,5cyclictest7950irq/56-dwc_otg_20:45:231
2790299368,16cyclictest0-21swapper/320:40:133
2790299363,23cyclictest19199-21ssh23:49:073
27902993617,8cyclictest8050irq/56-dwc_otg_00:25:363
27902993613,15cyclictest8050irq/56-dwc_otg_22:07:263
27902993611,12cyclictest4514-21apt-get19:35:213
27902993610,16cyclictest381ktimersoftd/322:04:443
2789899365,9cyclictest0-21swapper/122:00:191
2789899365,21cyclictest7950irq/56-dwc_otg_20:00:231
2789899365,18cyclictest7950irq/56-dwc_otg_22:20:191
2789899364,3cyclictest8050irq/56-dwc_otg_19:40:271
2789899364,3cyclictest8050irq/56-dwc_otg_19:40:271
27898993612,6cyclictest19593-21unixbench-2d21:30:371
2790299354,13cyclictest8050irq/56-dwc_otg_23:17:153
27902993520,8cyclictest8050irq/56-dwc_otg_23:51:513
27902993517,8cyclictest8050irq/56-dwc_otg_20:10:333
2790299351,24cyclictest10205-21cut19:50:313
2790299351,21cyclictest0-21swapper/319:15:223
2789899352,22cyclictest28124-21ls19:10:241
27898993520,5cyclictest817550wget19:45:221
2789899351,6cyclictest14580-21ssh23:40:341
2789899351,26cyclictest27584-21ls00:00:091
27898993511,13cyclictest0-21swapper/100:09:351
2790299342,18cyclictest8050irq/56-dwc_otg_20:50:093
27902993414,9cyclictest7850irq/56-dwc_otg21:45:583
2789899347,17cyclictest25886-21aten_rbpower_po20:40:231
2789899346,21cyclictest7850irq/56-dwc_otg23:20:391
2789899345,4cyclictest7950irq/56-dwc_otg_21:30:081
2789899341,8cyclictest19856-21latency_hist23:50:091
2790299336,14cyclictest0-21swapper/321:05:373
2790299334,12cyclictest7950irq/56-dwc_otg_19:30:303
2789899338,11cyclictest0-21swapper/119:50:311
2789899334,19cyclictest7950irq/56-dwc_otg_19:30:281
2789899332,4cyclictest7850irq/56-dwc_otg21:10:411
2789899332,22cyclictest7850irq/56-dwc_otg21:50:201
2789899332,22cyclictest7850irq/56-dwc_otg21:50:201
2790299327,6cyclictest8050irq/56-dwc_otg_19:25:223
2790299326,3cyclictest8050irq/56-dwc_otg_20:02:433
2790299325,6cyclictest7850irq/56-dwc_otg20:50:423
2790299325,6cyclictest7850irq/56-dwc_otg20:50:423
2790299324,22cyclictest0-21swapper/320:20:223
2790299323,14cyclictest30876-21ssh00:04:483
27902993215,8cyclictest8050irq/56-dwc_otg_23:25:093
2789899329,4cyclictest0-21swapper/123:08:471
2789899329,4cyclictest0-21swapper/122:15:331
2789899329,4cyclictest0-21swapper/122:15:331
2789899327,3cyclictest0-21swapper/122:06:431
2789899326,22cyclictest0-21swapper/120:15:231
2789899323,8cyclictest10058-21ssh23:35:291
2789899322,6cyclictest19972-21/usr/sbin/munin23:50:191
262982326,10sleep20-21swapper/219:05:262
2789899319,12cyclictest0-21swapper/121:47:131
2789899317,5cyclictest23-21ksoftirqd/122:35:331
27898993111,9cyclictest0-21swapper/123:25:131
27898993110,4cyclictest0-21swapper/122:25:191
2789899301,3cyclictest8050irq/56-dwc_otg_00:30:091
2790299284,10cyclictest7950irq/56-dwc_otg_20:28:303
2789899282,5cyclictest7850irq/56-dwc_otg20:55:291
268862286,9sleep30-21swapper/319:05:353
267812286,12sleep10-21swapper/119:05:331
2790299277,7cyclictest15323-21mv20:05:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional