You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-21 - 21:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu Nov 21, 2024 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1077998036,36cyclictest26840-21sendmail-msp09:40:122
106199623,9cyclictest131rcu_preempt09:35:120
1061995836,15cyclictest0-21swapper/011:20:120
1061995820,8cyclictest131rcu_preempt10:05:120
106199581,47cyclictest23539-21sendmail-msp08:40:120
107799549,37cyclictest0-21swapper/208:35:122
106999523,37cyclictest9940-21sendmail-msp12:00:121
107799511,29cyclictest2072-21ssh09:45:122
106999501,37cyclictest17058-21sendmail-msp08:00:171
1090994932,11cyclictest25866irq/58-eth012:10:173
1069994910,27cyclictest23159-21sendmail-msp10:00:131
1069994910,27cyclictest23159-21sendmail-msp10:00:121
109099475,27cyclictest25866irq/58-eth009:15:143
109099475,27cyclictest25866irq/58-eth009:15:133
106199463,37cyclictest10689-21sendmail-msp07:20:110
106999453,33cyclictest10956-21sendmail-msp11:20:121
1077994432,8cyclictest27112-21apt-get08:10:132
106199444,33cyclictest6625-21ssh12:35:110
106199443,33cyclictest18474-21sendmail-msp10:20:140
106199441,35cyclictest28268-21ssh09:15:140
106199441,35cyclictest28268-21ssh09:15:130
106199436,10cyclictest131rcu_preempt08:05:140
106999423,5cyclictest131rcu_preempt11:30:141
106999421,2cyclictest131rcu_preempt10:40:151
1061994219,19cyclictest12-21ksoftirqd/008:35:130
106199421,37cyclictest20180-21ssh11:05:140
1077994113,18cyclictest0-21swapper/208:50:112
1077994010,24cyclictest0-21swapper/207:55:112
106199401,33cyclictest26840-21sendmail-msp09:40:110
1090993917,10cyclictest25866irq/58-eth012:35:113
106999393,30cyclictest9517-21sendmail-msp11:40:121
106199395,25cyclictest14996-21sendmail-msp10:40:130
1090993831,4cyclictest25866irq/58-eth012:25:123
107799383,22cyclictest16410-21ssh09:55:142
107799383,22cyclictest16410-21ssh09:55:132
107799382,26cyclictest522-21usb-storage07:30:122
1090993722,8cyclictest33-21ksoftirqd/309:35:123
107799377,22cyclictest0-21swapper/209:35:142
1077993723,8cyclictest25515-21apt-get07:35:132
107799372,18cyclictest8318-21ssh12:16:202
1077993714,17cyclictest22146irq/40-dwc2_hso09:10:112
1077993710,12cyclictest0-21swapper/211:08:202
107799367,20cyclictest30343-21ssh10:05:022
107799367,20cyclictest30343-21ssh10:05:012
1077993629,3cyclictest26017-21apt-get10:25:112
107799362,28cyclictest3733-21grep11:55:202
107799362,28cyclictest26198-21ssh12:09:202
107799362,28cyclictest13947-21ssh12:38:202
1061993622,6cyclictest12-21ksoftirqd/007:50:130
106199361,27cyclictest12070-21sendmail-msp11:00:130
106199351,29cyclictest11386-21sendmail-msp12:20:150
106199351,29cyclictest11386-21sendmail-msp12:20:140
107799343,25cyclictest0-21swapper/210:30:202
107799343,24cyclictest0-21swapper/210:39:202
107799343,24cyclictest0-21swapper/210:39:202
107799341,20cyclictest26314-21ssh11:50:202
106999341,29cyclictest30139-21sendmail-msp07:40:111
1077993315,13cyclictest0-21swapper/212:00:142
106999333,25cyclictest4654-21ssh10:10:121
106999331,26cyclictest4158-21sendmail-msp08:20:171
106999331,22cyclictest2873-21sendmail-msp09:20:131
1061993316,7cyclictest12-21ksoftirqd/011:25:120
107799329,20cyclictest0-21swapper/212:10:162
107799323,10cyclictest28279-21sh09:15:142
107799323,10cyclictest28279-21sh09:15:142
107799322,20cyclictest20872-21ssh12:25:122
1077993217,9cyclictest0-21swapper/209:50:152
1077993217,11cyclictest0-21swapper/211:45:142
1077993216,11cyclictest22146irq/40-dwc2_hso11:35:132
1077993211,11cyclictest18736-21/usr/sbin/munin10:20:322
106199324,20cyclictest9508-21ssh09:50:160
106199321,24cyclictest12-21ksoftirqd/011:35:130
1061993211,16cyclictest632-21Async17:44:200
107799319,8cyclictest0-21swapper/207:15:112
1077993119,9cyclictest30092-21ssh12:30:162
1077993118,9cyclictest22146irq/40-dwc2_hso07:45:122
1077993118,9cyclictest22146irq/40-dwc2_hso07:45:122
1077993111,9cyclictest0-21swapper/211:40:132
1077993111,16cyclictest0-21swapper/210:55:132
1069993110,14cyclictest21-21ksoftirqd/111:25:111
106199316,17cyclictest12-21ksoftirqd/008:55:120
1077993022,4cyclictest26682-21apt-get11:30:142
107799302,10cyclictest0-21swapper/211:20:302
1077993016,9cyclictest0-21swapper/208:30:122
1077993014,13cyclictest10740-21apt-get09:00:112
106199309,15cyclictest12-21ksoftirqd/007:45:130
106199309,15cyclictest12-21ksoftirqd/007:45:120
106199308,3cyclictest131rcu_preempt09:30:090
106199302,22cyclictest3170-21cut10:30:210
106199302,21cyclictest10520-21sendmail-msp09:00:120
107799299,17cyclictest900-21snmpd08:45:322
107799293,7cyclictest131rcu_preempt10:05:132
107799293,21cyclictest0-21swapper/211:03:192
106199292,20cyclictest2873-21sendmail-msp09:20:270
109099285,11cyclictest0-21swapper/309:40:123
109099284,18cyclictest25866irq/58-eth009:55:133
109099284,18cyclictest25866irq/58-eth009:55:133
109099282,15cyclictest25866irq/58-eth011:50:193
1090992818,6cyclictest0-21swapper/307:55:123
1090992817,8cyclictest25866irq/58-eth010:05:133
107799286,7cyclictest13065-21cut10:15:242
107799285,11cyclictest13410-21cut07:20:222
107799282,18cyclictest22146irq/40-dwc2_hso07:10:112
1077992816,8cyclictest15678-21apt-get09:05:122
1077992816,8cyclictest0-21swapper/208:55:112
1077992812,12cyclictest8361-21cat08:20:362
1077992811,13cyclictest0-21swapper/211:10:172
106999282,19cyclictest9927-21ssh12:18:211
106999282,18cyclictest3490-21sh11:55:191
106999281,21cyclictest29814-21ssh10:50:131
1061992815,8cyclictest0-21swapper/012:10:170
1061992813,11cyclictest12033-21cron11:00:010
109099273,18cyclictest25866irq/58-eth010:15:163
109099272,18cyclictest0-21swapper/310:23:203
109099272,12cyclictest25866irq/58-eth010:50:123
107799271,17cyclictest22446-21rm10:45:122
107799271,17cyclictest22446-21rm10:45:122
106999272,20cyclictest24815-21ssh12:07:211
106999272,19cyclictest12313-21ssh12:36:241
106199272,19cyclictest710-21rs:main0
106199271,19cyclictest20882-21ssh12:25:130
109099263,15cyclictest0-21swapper/310:30:213
109099262,17cyclictest321rcuc/308:35:123
107799269,13cyclictest3083-21awk08:15:282
107799269,13cyclictest26056-21sed08:05:302
107799269,13cyclictest19277-21fschecks_time11:25:152
107799267,11cyclictest19335-21wc07:25:282
107799262,15cyclictest3374-21ssh11:15:142
1077992615,6cyclictest22146irq/40-dwc2_hso09:30:102
1077992610,12cyclictest19031-21/usr/sbin/munin10:40:292
106999263,5cyclictest19911-21apt-get09:35:121
1069992611,11cyclictest30571-21sh10:05:131
106199261,18cyclictest23159-21sendmail-msp10:00:290
106199261,18cyclictest23159-21sendmail-msp10:00:290
109099252,17cyclictest0-21swapper/310:26:023
107799257,8cyclictest27-21ksoftirqd/208:00:172
107799254,15cyclictest0-21swapper/210:51:512
107799253,17cyclictest0-21swapper/212:23:202
107799253,17cyclictest0-21swapper/212:23:202
1077992512,10cyclictest0-21swapper/208:25:112
1077992510,12cyclictest5005-21ssh10:10:172
106999253,17cyclictest721-21ssh10:30:121
106199253,11cyclictest131rcu_preempt08:30:120
1061992517,6cyclictest12-21ksoftirqd/007:40:130
1061992511,7cyclictest0-21swapper/009:45:140
1090992412,9cyclictest0-21swapper/309:30:103
107799249,12cyclictest12307-21ssh09:29:462
107799248,12cyclictest31466-21cut07:40:212
107799248,12cyclictest0-21swapper/207:50:122
1077992415,6cyclictest3140-21ssh09:20:122
106999242,6cyclictest0-21swapper/112:34:211
106999242,17cyclictest11788-21ssh12:20:161
106999242,17cyclictest11788-21ssh12:20:151
106999241,18cyclictest26526-21ssh12:26:221
106999241,18cyclictest1006-21ls08:50:021
109099233,16cyclictest0-21swapper/310:35:213
109099233,16cyclictest0-21swapper/310:35:203
109099233,11cyclictest0-21swapper/312:05:123
109099231,11cyclictest25866irq/58-eth008:20:163
106999238,4cyclictest131rcu_preempt09:05:151
106199234,11cyclictest0-21swapper/007:25:130
106199233,9cyclictest12-21ksoftirqd/009:10:110
1061992310,6cyclictest0-21swapper/011:10:190
109099223,16cyclictest0-21swapper/311:55:193
109099222,14cyclictest0-21swapper/312:16:203
107799227,8cyclictest24808-21cat08:40:192
106999221,17cyclictest4687-21ssh10:55:131
106999221,17cyclictest30896-21grep09:40:301
1061992214,5cyclictest0-21swapper/010:50:170
1061992213,7cyclictest3690-21sshd12:15:200
1061992213,5cyclictest522-21usb-storage07:30:130
1061992211,8cyclictest17058-21sendmail-msp08:00:020
1061992210,9cyclictest9517-21sendmail-msp11:40:080
109099214,10cyclictest25866irq/58-eth010:55:123
109099212,14cyclictest0-21swapper/311:40:133
106999219,9cyclictest18474-21sendmail-msp10:20:171
106999218,10cyclictest10520-21sendmail-msp09:00:221
106999211,6cyclictest52246usb-storage08:40:141
1069992112,5cyclictest0-21swapper/111:05:241
106199217,11cyclictest4158-21sendmail-msp08:20:210
106199215,12cyclictest18654-21ssh12:05:120
1061992115,4cyclictest0-21swapper/007:15:230
106199211,15cyclictest3364-21ssh11:15:140
106199211,12cyclictest0-21swapper/011:51:430
1090992011,7cyclictest0-21swapper/312:00:193
109099201,12cyclictest0-21swapper/309:20:143
106999209,6cyclictest0-21swapper/111:36:141
106999206,9cyclictest1051-21cyclictest11:50:211
1069992010,8cyclictest11535-21open_files10:35:261
1069992010,8cyclictest11535-21open_files10:35:251
1069992010,7cyclictest2617-21cron12:15:011
106199202,15cyclictest30220-21ssh12:30:190
106199201,13cyclictest1194-21cut08:50:150
1061992010,7cyclictest17058-21sendmail-msp08:00:180
106199200,17cyclictest5047-21timerwakeupswit07:10:270
109099196,10cyclictest0-21swapper/311:25:203
109099193,11cyclictest0-21swapper/312:23:213
109099193,11cyclictest0-21swapper/312:23:203
109099193,11cyclictest0-21swapper/311:38:203
109099192,11cyclictest0-21swapper/309:05:043
1090991911,5cyclictest0-21swapper/312:30:173
109099191,13cyclictest0-21swapper/307:20:143
109099191,12cyclictest0-21swapper/311:30:163
109099191,11cyclictest0-21swapper/307:50:153
1090991910,6cyclictest0-21swapper/311:07:463
106999198,8cyclictest28033-21apt-get11:10:181
106999197,9cyclictest12070-21sendmail-msp11:00:191
106999194,11cyclictest11692-21ssh10:15:161
106999192,11cyclictest24788-21ssh11:48:201
1069991911,6cyclictest5646-21cron08:55:021
106999191,13cyclictest10689-21sendmail-msp07:20:151
106999190,16cyclictest18095-21cut09:55:211
106999190,16cyclictest18095-21cut09:55:201
106199199,7cyclictest27193-21df08:10:150
106199194,3cyclictest131rcu_preempt10:25:120
106199194,12cyclictest9405-21awk10:10:320
106199192,12cyclictest1977-21ssh11:55:130
106199191,15cyclictest28463-21cut07:35:200
1061991910,6cyclictest22541-21ssh10:45:150
1061991910,6cyclictest22541-21ssh10:45:140
109099189,5cyclictest0-21swapper/309:45:133
109099183,11cyclictest0-21swapper/311:45:203
109099182,10cyclictest0-21swapper/308:00:053
109099182,10cyclictest0-21swapper/307:30:143
1090991813,3cyclictest0-21swapper/309:10:243
109099181,11cyclictest0-21swapper/311:00:133
1090991810,6cyclictest0-21swapper/311:21:493
1090991810,6cyclictest0-21swapper/309:54:563
107799181,14cyclictest1153-21latency_hist07:10:022
106999185,10cyclictest4148-21kthreadcore07:10:211
106999182,13cyclictest4131-21kthreadcore11:15:211
106999181,9cyclictest26012-21ntp_192.168.11410:45:251
106999181,9cyclictest26012-21ntp_192.168.11410:45:241
1069991812,4cyclictest0-21swapper/107:50:231
106999181,14cyclictest26581-21kthreadcore07:35:181
106199189,7cyclictest31973-21awk08:15:160
106199188,7cyclictest11024-21kthreadcore08:25:200
106199185,4cyclictest131rcu_preempt09:55:130
106199185,4cyclictest131rcu_preempt09:55:120
106199185,10cyclictest21197-21sh11:45:230
106199181,9cyclictest11668-21/usr/sbin/munin10:15:180
1061991812,4cyclictest7935-21sh10:35:130
1061991812,4cyclictest7935-21sh10:35:120
109099179,6cyclictest0-21swapper/311:15:113
109099179,6cyclictest0-21swapper/311:14:233
109099179,5cyclictest0-21swapper/310:10:243
109099172,10cyclictest0-21swapper/308:55:123
109099172,10cyclictest0-21swapper/308:40:123
109099172,10cyclictest0-21swapper/308:25:123
109099172,10cyclictest0-21swapper/307:45:143
109099172,10cyclictest0-21swapper/307:45:133
109099172,10cyclictest0-21swapper/307:25:133
109099172,10cyclictest0-21swapper/307:15:113
1090991711,4cyclictest0-21swapper/309:25:153
109099171,10cyclictest0-21swapper/307:35:153
106999176,9cyclictest9260-21apt-get08:25:111
106999175,9cyclictest8371-21kthreadcore08:55:201
106999172,12cyclictest9484-21ntp_192.168.11407:15:221
106999171,9cyclictest16074-21switchtime07:55:261
106999171,7cyclictest6540-21timerandwakeup07:45:271
106999171,7cyclictest6540-21timerandwakeup07:45:271
106999171,13cyclictest32130-21cat09:15:301
106999171,13cyclictest32130-21cat09:15:301
106999171,12cyclictest12383-21apt-get09:30:001
106999171,11cyclictest12383-21apt-get09:30:021
1069991710,5cyclictest25273-21sshd07:33:201
106199172,10cyclictest0-21swapper/011:30:140
106199171,5cyclictest0-21swapper/009:08:270
109099168,6cyclictest0-21swapper/310:46:213
109099168,6cyclictest0-21swapper/310:46:213
109099162,10cyclictest0-21swapper/310:00:143
109099162,10cyclictest0-21swapper/310:00:133
109099162,10cyclictest0-21swapper/308:45:123
109099161,9cyclictest0-21swapper/307:40:143
109099161,12cyclictest0-21swapper/308:10:173
109099161,10cyclictest0-21swapper/308:05:153
106999169,4cyclictest27076-21kthreadcore10:25:171
106999168,6cyclictest0-21swapper/108:30:301
106999163,10cyclictest18874-21/usr/sbin/munin08:35:231
106999162,11cyclictest15890-21fschecks_count07:25:151
1069991611,3cyclictest3026-21open_files08:15:281
106999161,13cyclictest26851-21users08:05:331
106999161,13cyclictest0-21swapper/108:10:161
106999161,11cyclictest2624-21irqrtprio09:45:171
106999161,11cyclictest22999-21tr09:10:191
106999161,10cyclictest9583-21cpuspeed09:50:181
106199161,11cyclictest1125-21wc07:10:010
106199160,13cyclictest12399-21awk09:30:000
109099152,9cyclictest0-21swapper/310:40:153
109099152,10cyclictest0-21swapper/309:00:133
109099141,10cyclictest0-21swapper/307:10:143
106999147,4cyclictest1162-21cat07:10:011
109099131,6cyclictest0-21swapper/308:50:133
109099131,3cyclictest0-21swapper/308:30:193
109099121,5cyclictest0-21swapper/308:15:223
109099111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional