You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-01 - 02:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System (updated Sun Sep 01, 2024 00:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2738499662,58cyclictest24553-21ssh21:55:151
2738499662,58cyclictest24553-21ssh21:55:151
2739599633,36cyclictest14560-21ssh23:45:132
2737399636,9cyclictest131rcu_preempt20:45:120
2739599603,38cyclictest900-21snmpd20:45:142
2739599585,8cyclictest131rcu_preempt21:20:122
2737399540,40cyclictest10296-21ssh22:25:140
2739599501,31cyclictest11428-21ssh22:45:132
2737399501,8cyclictest12-21ksoftirqd/019:55:110
2737399473,35cyclictest18750-21ssh22:30:150
2739599451,30cyclictest0-21swapper/222:30:152
2737399452,40cyclictest4084-21ssh23:00:120
2740799432,25cyclictest25866irq/58-eth022:40:123
27384994326,11cyclictest21-21ksoftirqd/121:45:131
27384994326,11cyclictest21-21ksoftirqd/121:45:131
2740799427,23cyclictest25866irq/58-eth023:50:133
2737399421,25cyclictest25989-21ssh00:15:120
2737399421,25cyclictest25989-21ssh00:15:110
2739599405,27cyclictest261rcuc/221:10:132
27395994028,8cyclictest27-21ksoftirqd/222:10:132
2738499391,32cyclictest9028-21rm22:05:131
27407993819,10cyclictest25866irq/58-eth023:30:133
2739599382,29cyclictest29277-21ssh23:15:122
2739599382,29cyclictest29277-21ssh23:15:112
27373993721,11cyclictest632-21Async17:44:200
2740799364,20cyclictest25866irq/58-eth021:25:143
2739599362,18cyclictest21601-21ssh21:33:272
2738499362,30cyclictest6174-21ssh23:40:151
2740799352,21cyclictest25866irq/58-eth021:15:133
2740799352,21cyclictest25866irq/58-eth021:15:123
27395993524,7cyclictest27-21ksoftirqd/222:40:132
2737399355,22cyclictest12-21ksoftirqd/021:40:110
2737399351,22cyclictest23123-21sh21:35:120
27395993416,12cyclictest0-21swapper/223:35:122
27395993416,12cyclictest0-21swapper/223:35:112
2738499342,27cyclictest27114-21ssh22:35:141
2740799332,19cyclictest25866irq/58-eth022:45:133
27407993319,9cyclictest0-21swapper/323:15:123
27407993319,9cyclictest0-21swapper/323:15:113
2739599338,16cyclictest0-21swapper/200:25:102
2739599331,17cyclictest1259-21ssh00:20:172
2739599331,17cyclictest1259-21ssh00:20:172
27395993311,11cyclictest131rcu_preempt21:55:232
27395993311,11cyclictest131rcu_preempt21:55:222
2737399332,23cyclictest6037-21awk22:20:290
2740799324,17cyclictest25866irq/58-eth021:35:123
2739599323,14cyclictest261rcuc/223:25:122
2739599322,20cyclictest22146irq/40-dwc2_hso20:10:122
2738499312,20cyclictest28151-21ssh22:55:141
2740799302,16cyclictest25866irq/58-eth000:00:133
2740799302,15cyclictest25866irq/58-eth019:50:133
2739599303,19cyclictest0-21swapper/222:20:292
2739599302,18cyclictest22146irq/40-dwc2_hso19:30:142
2738499303,23cyclictest10298-21ssh22:25:151
2738499302,4cyclictest21-21ksoftirqd/122:11:271
2740799293,19cyclictest0-21swapper/321:32:283
2740799293,17cyclictest0-21swapper/322:23:283
2740799292,13cyclictest25866irq/58-eth022:30:123
27407992913,9cyclictest25866irq/58-eth020:35:123
2740799291,23cyclictest25866irq/58-eth019:10:123
27407992911,10cyclictest25866irq/58-eth000:35:133
2739599295,18cyclictest22146irq/40-dwc2_hso20:20:132
2739599295,18cyclictest22146irq/40-dwc2_hso20:20:122
2739599293,14cyclictest900-21snmpd00:15:122
2739599293,14cyclictest900-21snmpd00:15:112
2739599292,20cyclictest900-21snmpd21:50:282
2739599292,20cyclictest0-21swapper/219:45:052
2739599292,20cyclictest0-21swapper/219:45:042
2739599291,16cyclictest22146irq/40-dwc2_hso20:05:132
2738499293,18cyclictest20794-21ssh23:10:121
2737399295,9cyclictest0-21swapper/021:55:160
2737399295,9cyclictest0-21swapper/021:55:150
2740799285,13cyclictest25866irq/58-eth023:25:143
2739599283,14cyclictest22146irq/40-dwc2_hso19:15:112
2739599282,21cyclictest32077-21ssh22:17:282
2739599282,18cyclictest22146irq/40-dwc2_hso23:20:122
27395992818,7cyclictest27-21ksoftirqd/222:50:132
2738499283,21cyclictest1263-21ssh00:20:181
2738499283,21cyclictest1263-21ssh00:20:171
2738499283,16cyclictest25901-21rm22:15:141
2737399282,20cyclictest23201-21ssh00:33:420
2740799273,15cyclictest0-21swapper/300:33:423
27407992710,12cyclictest25866irq/58-eth022:25:143
2739599275,14cyclictest0-21swapper/222:35:142
2739599273,3cyclictest131rcu_preempt23:10:132
2739599273,17cyclictest22146irq/40-dwc2_hso00:00:132
2739599273,16cyclictest22146irq/40-dwc2_hso20:30:152
2739599272,17cyclictest22146irq/40-dwc2_hso19:50:122
2738499277,3cyclictest131rcu_preempt22:50:171
2738499274,15cyclictest12416-21ssh23:05:131
2738499273,6cyclictest21-21ksoftirqd/123:20:031
2738499273,6cyclictest21-21ksoftirqd/123:20:021
2738499271,20cyclictest12980-21diskmemload23:50:131
2738499271,19cyclictest21-21ksoftirqd/120:55:141
2737399273,18cyclictest12-21ksoftirqd/019:10:120
2737399272,19cyclictest2519-21ssh22:58:200
2740799268,11cyclictest25966irq/59-eth000:25:113
2739599269,13cyclictest0-21swapper/221:40:112
2739599264,18cyclictest22146irq/40-dwc2_hso00:30:132
2739599262,15cyclictest22146irq/40-dwc2_hso20:35:112
2738499268,6cyclictest21-21ksoftirqd/123:05:011
2738499264,6cyclictest21-21ksoftirqd/121:20:161
2738499261,5cyclictest21-21ksoftirqd/122:40:271
27384992611,4cyclictest21-21ksoftirqd/121:32:311
2737399262,19cyclictest21940-21ssh23:30:150
2740799253,12cyclictest25866irq/58-eth023:45:133
2739599252,14cyclictest22146irq/40-dwc2_hso19:35:122
2738499256,3cyclictest131rcu_preempt23:36:461
2738499256,3cyclictest131rcu_preempt23:36:451
2738499256,3cyclictest131rcu_preempt23:20:281
27384992515,4cyclictest21-21ksoftirqd/121:40:181
27384992512,5cyclictest21-21ksoftirqd/100:16:081
27384992512,5cyclictest21-21ksoftirqd/100:16:071
27384992510,8cyclictest0-21swapper/100:04:541
27384992510,7cyclictest0-21swapper/100:10:231
27407992413,7cyclictest25866irq/58-eth021:40:143
2740799241,14cyclictest0-21swapper/322:35:153
2739599242,15cyclictest22146irq/40-dwc2_hso23:05:122
2739599242,15cyclictest22146irq/40-dwc2_hso19:20:142
2739599242,15cyclictest0-21swapper/221:05:132
2738499249,4cyclictest21-21ksoftirqd/100:05:421
2738499248,3cyclictest131rcu_preempt00:32:041
2738499243,6cyclictest21-21ksoftirqd/122:20:211
2737399242,17cyclictest3026-21apt-get22:40:110
2740799232,13cyclictest0-21swapper/320:05:123
27407992318,3cyclictest25966irq/59-eth023:35:113
27407992318,3cyclictest25966irq/59-eth023:35:113
2740799231,16cyclictest0-21swapper/323:10:113
2739599236,2cyclictest131rcu_preempt20:00:182
2739599231,7cyclictest2498266ssh00:35:202
2739599231,18cyclictest10252-21ssh22:25:122
2739599231,12cyclictest900-21snmpd20:40:132
2739599231,12cyclictest23130-21ssh21:35:122
2738499239,8cyclictest0-21swapper/121:38:101
2738499239,4cyclictest21-21ksoftirqd/121:52:051
2738499238,4cyclictest21-21ksoftirqd/123:27:121
2738499237,5cyclictest21-21ksoftirqd/100:26:121
2738499237,4cyclictest131rcu_preempt20:20:161
2738499237,4cyclictest131rcu_preempt20:20:151
2738499236,3cyclictest131rcu_preempt22:30:251
2738499235,3cyclictest131rcu_preempt23:45:211
2738499233,6cyclictest21-21ksoftirqd/121:10:111
2738499233,3cyclictest131rcu_preempt19:30:221
2738499231,4cyclictest21-21ksoftirqd/120:30:141
27384992313,4cyclictest21-21ksoftirqd/121:27:311
2738499231,17cyclictest21760-21ssh21:15:131
2738499231,17cyclictest21760-21ssh21:15:121
2737399239,8cyclictest12-21ksoftirqd/000:39:280
2737399238,9cyclictest27360-21cyclictest23:05:130
2737399234,14cyclictest9045-21diskmemload22:05:140
2737399233,4cyclictest131rcu_preempt19:35:140
27373992311,4cyclictest0-21swapper/021:49:220
27373992311,4cyclictest0-21swapper/021:49:210
27373992311,3cyclictest0-21swapper/023:47:260
2740799223,12cyclictest0-21swapper/323:40:053
2740799222,14cyclictest25966irq/59-eth022:55:143
27407992210,6cyclictest0-21swapper/322:17:283
2739599224,14cyclictest9296-21cut22:05:162
2739599223,9cyclictest24521-21cut21:15:222
2739599223,9cyclictest24521-21cut21:15:212
2739599223,9cyclictest16429-1kworker/u9:223:00:142
2739599223,16cyclictest22146irq/40-dwc2_hso20:15:142
2739599222,15cyclictest0-21swapper/223:50:142
2739599222,15cyclictest0-21swapper/222:57:282
2739599221,8cyclictest580-21ssh22:00:142
2739599221,8cyclictest580-21ssh22:00:132
2739599221,5cyclictest7839-21cut21:45:162
2739599221,5cyclictest7839-21cut21:45:152
2739599221,12cyclictest3624-21apt21:00:142
2738499227,7cyclictest21-21ksoftirqd/120:15:231
2738499227,7cyclictest21-21ksoftirqd/100:38:571
2738499226,8cyclictest21-21ksoftirqd/121:05:161
2738499224,8cyclictest21-21ksoftirqd/119:20:191
2738499223,5cyclictest21-21ksoftirqd/119:40:161
2738499223,4cyclictest21-21ksoftirqd/120:25:211
2738499222,12cyclictest21-21ksoftirqd/122:03:211
2738499222,12cyclictest21-21ksoftirqd/122:03:201
2738499221,4cyclictest21-21ksoftirqd/123:55:241
2738499221,4cyclictest0-21swapper/120:10:221
27384992212,4cyclictest21-21ksoftirqd/122:45:061
2738499221,18cyclictest16819-21find20:40:121
27384992210,4cyclictest21-21ksoftirqd/120:00:171
2737399224,3cyclictest131rcu_preempt23:25:120
2737399222,14cyclictest11442-21ssh22:45:150
2737399221,6cyclictest19791-21cut22:10:200
27373992212,5cyclictest22487-21kthreadcore21:15:210
27373992212,5cyclictest22487-21kthreadcore21:15:200
2737399221,18cyclictest11444-21kthreadcore20:00:190
2737399221,16cyclictest8573-21cut21:05:150
2737399221,16cyclictest5194-21ssh23:20:120
27373992211,3cyclictest0-21swapper/000:13:180
27373992210,3cyclictest12-21ksoftirqd/000:25:020
27373992210,3cyclictest12-21ksoftirqd/000:25:010
27373992210,2cyclictest12-21ksoftirqd/023:20:020
27407992110,7cyclictest25866irq/58-eth023:00:123
2739599216,7cyclictest3404-21sendmail-msp21:00:002
2739599215,13cyclictest22770-21cat00:10:272
2739599213,3cyclictest0-21swapper/220:25:152
2739599213,12cyclictest0-21swapper/223:40:132
2739599212,9cyclictest25772-21switchtime23:30:302
2739599211,10cyclictest8449-21kthreadcore21:25:232
2738499218,3cyclictest131rcu_preempt19:35:211
2738499215,5cyclictest131rcu_preempt20:40:021
2738499215,4cyclictest131rcu_preempt23:30:261
2738499215,3cyclictest131rcu_preempt20:05:181
2738499214,3cyclictest131rcu_preempt19:25:231
2737399217,3cyclictest22017-21kthreadcore22:50:200
2737399212,15cyclictest15292-21ls20:35:200
2737399211,16cyclictest3625-21apt-get21:00:130
27373992111,4cyclictest0-21swapper/023:50:310
27373992111,2cyclictest12-21ksoftirqd/022:18:130
2740799204,9cyclictest0-21swapper/300:15:123
2740799204,9cyclictest0-21swapper/300:15:113
2740799203,12cyclictest0-21swapper/322:13:113
2740799203,11cyclictest0-21swapper/321:50:273
27407992013,4cyclictest0-21swapper/322:50:153
2739599209,9cyclictest0-21swapper/219:40:042
2739599202,10cyclictest32268-21kthreadcore23:55:202
2739599201,15cyclictest14281-21grep19:30:012
27395992010,7cyclictest6612-21kthreadcore19:55:182
2739599200,18cyclictest26429-21acpi20:50:112
2738499207,3cyclictest131rcu_preempt19:15:121
2738499205,3cyclictest131rcu_preempt20:45:141
27384992015,3cyclictest201rcuc/120:50:191
2738499201,4cyclictest21-21ksoftirqd/119:45:151
2738499201,4cyclictest21-21ksoftirqd/119:45:141
27384992014,4cyclictest201rcuc/119:10:111
27384992010,4cyclictest21-21ksoftirqd/121:00:161
2737399209,7cyclictest16343-21ssh21:50:170
2737399209,4cyclictest131rcu_preempt20:15:130
2737399205,4cyclictest12-21ksoftirqd/020:20:150
2737399205,4cyclictest12-21ksoftirqd/020:20:140
2737399202,15cyclictest21367-21kthreadcore20:10:180
2737399202,12cyclictest15176-21ssh00:05:040
2737399201,2cyclictest131rcu_preempt23:57:230
2737399201,16cyclictest0-21swapper/021:30:110
27373992010,6cyclictest12-21ksoftirqd/019:15:120
2740799193,9cyclictest0-21swapper/319:15:123
2740799191,13cyclictest0-21swapper/300:20:213
2740799191,13cyclictest0-21swapper/300:20:203
2740799191,12cyclictest0-21swapper/320:15:283
27407991910,6cyclictest0-21swapper/322:08:083
2739599191,14cyclictest18120-21latency_hist00:10:032
2739599191,10cyclictest29892-21kthreadcore19:10:212
2738499195,4cyclictest131rcu_preempt19:55:141
27384991913,4cyclictest0-21swapper/119:50:181
2737399199,7cyclictest5728-21sendmail-msp23:40:050
2737399198,9cyclictest1666-21irqrtprio19:50:180
2737399195,3cyclictest131rcu_preempt20:05:120
2737399194,4cyclictest131rcu_preempt19:25:200
2737399194,4cyclictest12-21ksoftirqd/023:18:450
2737399193,3cyclictest131rcu_preempt19:40:190
2737399192,2cyclictest131rcu_preempt22:37:370
2737399192,2cyclictest131rcu_preempt21:27:490
27373991913,4cyclictest12-21ksoftirqd/019:30:220
2737399191,16cyclictest654-21cpuspeed22:00:140
2737399191,16cyclictest654-21cpuspeed22:00:130
27373991911,6cyclictest27688-21sshd23:13:230
2737399191,14cyclictest30181-21apt-get21:20:130
2737399191,10cyclictest9083-21fschecks_time00:25:130
2740799189,6cyclictest25866irq/58-eth023:22:273
2740799189,6cyclictest0-21swapper/323:05:153
2740799185,9cyclictest25866irq/58-eth021:55:173
2740799185,9cyclictest25866irq/58-eth021:55:163
2740799182,10cyclictest0-21swapper/320:55:143
2740799182,10cyclictest0-21swapper/320:30:173
2740799182,10cyclictest0-21swapper/320:25:153
2740799182,10cyclictest0-21swapper/320:20:143
2740799182,10cyclictest0-21swapper/320:20:133
2740799182,10cyclictest0-21swapper/319:35:123
2740799181,13cyclictest0-21swapper/319:55:263
2740799181,12cyclictest0-21swapper/320:00:163
2740799181,11cyclictest0-21swapper/323:55:223
2740799181,11cyclictest0-21swapper/320:10:133
27407991810,6cyclictest0-21swapper/321:47:193
27407991810,6cyclictest0-21swapper/321:47:183
27407991810,6cyclictest0-21swapper/321:22:333
27407991810,6cyclictest0-21swapper/300:12:273
2739599181,8cyclictest27481-21latency_hist19:10:012
2737399188,7cyclictest16615-21sendmail-msp20:40:050
2737399186,6cyclictest131rcu_preempt00:02:490
2737399184,3cyclictest131rcu_preempt20:55:010
2737399182,2cyclictest131rcu_preempt21:13:330
2737399181,3cyclictest0-21swapper/023:36:410
2737399181,3cyclictest0-21swapper/023:36:400
2737399181,2cyclictest131rcu_preempt20:25:150
27373991811,5cyclictest0-21swapper/020:55:290
27373991811,4cyclictest12-21ksoftirqd/019:20:180
2740799179,6cyclictest0-21swapper/322:00:283
2740799179,6cyclictest0-21swapper/322:00:273
2740799179,6cyclictest0-21swapper/321:12:103
2740799172,10cyclictest0-21swapper/321:05:133
2740799172,10cyclictest0-21swapper/319:30:143
2740799172,10cyclictest0-21swapper/319:20:133
2737399171,4cyclictest131rcu_preempt19:50:020
2737399171,4cyclictest131rcu_preempt19:50:010
2740799168,6cyclictest0-21swapper/300:05:083
2740799161,12cyclictest0-21swapper/319:40:093
2738499151,3cyclictest131rcu_preempt19:10:021
2740799149,3cyclictest0-21swapper/320:41:283
2740799144,7cyclictest25866irq/58-eth020:45:143
2740799142,7cyclictest0-21swapper/319:45:053
2740799142,7cyclictest0-21swapper/319:45:043
2737399141,10cyclictest27290-21latency_hist19:10:010
2740799123,7cyclictest0-21swapper/319:25:123
2740799122,6cyclictest25866irq/58-eth020:50:113
2740799111,8cyclictest0-21swapper/321:00:123
2740799101,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional