You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 08:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri Nov 22, 2024 00:43:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12772900,4chrt0-21swapper/319:21:123
12772900,4chrt0-21swapper/319:21:123
238782810,2sleep30-21swapper/323:11:143
313872800,3chrt0-21swapper/322:06:053
315242730,2sleep331200-21kworker/3:2+events19:16:053
164962730,9sleep016500-21latency_hist00:21:050
164962730,9sleep016500-21latency_hist00:21:050
164962730,9sleep016500-21latency_hist00:21:050
71432670,4sleep17140-21apt-get23:56:141
71432670,4sleep17140-21apt-get23:56:141
315592660,3sleep031537-21ls19:16:070
94622630,2chrt0-21swapper/322:31:263
94622630,2chrt0-21swapper/322:31:263
31772996237,22cyclictest0-21swapper/000:06:110
31772996237,22cyclictest0-21swapper/000:06:110
145832600,3sleep20-21swapper/221:21:182
145832600,3sleep20-21swapper/221:21:182
90862590,3sleep20-21swapper/222:31:212
90862590,3sleep20-21swapper/222:31:212
3177299572,46cyclictest12786-21taskset19:51:180
246602560,3sleep20-21swapper/220:21:252
246602560,3sleep20-21swapper/220:21:252
78392540,1sleep10-21swapper/123:56:231
78392540,1sleep10-21swapper/123:56:231
31772995235,13cyclictest15765-21snmpd00:26:170
31772995235,13cyclictest15765-21snmpd00:26:170
31772995142,4cyclictest0-21swapper/019:56:180
31772995142,4cyclictest0-21swapper/019:56:180
49732500,3sleep10-21swapper/123:46:211
49732500,3sleep10-21swapper/123:46:211
31772994823,17cyclictest171ktimers/019:31:170
31772994823,17cyclictest171ktimers/019:31:170
273212480,2sleep20-21swapper/220:26:312
273212480,2sleep20-21swapper/220:26:312
204612470,2chrt20464-21gltestperf21:36:251
204612470,2chrt20464-21gltestperf21:36:251
31772994619,5cyclictest30624-21timerandwakeup23:26:310
31772994619,5cyclictest30624-21timerandwakeup23:26:310
242632460,2sleep024264-21latency_hist00:41:110
242632460,2sleep024264-21latency_hist00:41:110
243622450,2sleep30-21swapper/320:21:223
243622450,2sleep30-21swapper/320:21:223
225212450,2chrt0-21swapper/100:36:221
225212450,2chrt0-21swapper/100:36:221
241392440,3sleep20-21swapper/223:11:212
241392440,3sleep20-21swapper/223:11:212
31772994318,4cyclictest8129-21grep21:01:280
31772994318,4cyclictest8129-21grep21:01:280
31772994317,5cyclictest31563-21latency_hist22:06:120
269112430,3sleep30-21swapper/320:26:283
269112430,3sleep30-21swapper/320:26:283
3177399425,7cyclictest181rcu_preempt21:06:161
3177399425,7cyclictest181rcu_preempt21:06:161
31773994218,14cyclictest1363-21gdbus00:06:181
31773994218,14cyclictest1363-21gdbus00:06:181
31772994226,5cyclictest0-21swapper/000:06:180
31772994226,5cyclictest0-21swapper/000:06:180
3177299422,33cyclictest1314-21cat19:21:130
31772994220,4cyclictest31771-21cyclictest22:31:210
31772994220,4cyclictest31771-21cyclictest22:31:210
5512410,2chrt552-21awk20:41:301
5512410,2chrt552-21awk20:41:301
31773994132,7cyclictest23688-21cron21:46:071
31773994132,7cyclictest23688-21cron21:46:071
31772994132,6cyclictest18484-21cron00:26:050
31772994132,6cyclictest18484-21cron00:26:050
31772994119,15cyclictest171ktimers/000:11:180
31772994119,15cyclictest171ktimers/000:11:180
31772994115,7cyclictest914-21/usr/sbin/munin23:36:290
31772994115,7cyclictest914-21/usr/sbin/munin23:36:290
31773994032,6cyclictest21294-21needreboot00:31:261
31773994032,6cyclictest21294-21needreboot00:31:261
3177299408,22cyclictest36350irq/47-eth019:16:170
3177299403,32cyclictest0-21swapper/022:41:160
3177299403,32cyclictest0-21swapper/022:41:160
31772994023,9cyclictest201rcuc/020:01:150
31772994023,9cyclictest201rcuc/020:01:150
31772994016,2cyclictest0-21swapper/022:01:130
31772994016,2cyclictest0-21swapper/022:01:130
171822400,3sleep20-21swapper/220:01:232
171822400,3sleep20-21swapper/220:01:232
68502390,4chrt0-21swapper/322:26:193
68502390,4chrt0-21swapper/322:26:193
3177399391,5cyclictest181rcu_preempt22:51:171
3177399391,5cyclictest181rcu_preempt22:51:171
31772993917,12cyclictest171ktimers/023:21:170
31772993917,12cyclictest171ktimers/023:21:170
31772993915,4cyclictest19024-21systemd-journal21:16:050
31772993915,4cyclictest19024-21systemd-journal21:16:050
227422390,3sleep30-21swapper/300:36:243
227422390,3sleep30-21swapper/300:36:243
207112390,3chrt0-21swapper/321:36:273
207112390,3chrt0-21swapper/321:36:273
31773993823,10cyclictest281ktimers/122:36:211
31773993823,10cyclictest281ktimers/122:36:211
31772993831,5cyclictest613-21dbus-daemon20:56:090
31772993831,5cyclictest613-21dbus-daemon20:56:090
221892380,3sleep30-21swapper/321:41:173
221892380,3sleep30-21swapper/321:41:173
203052380,1sleep020460-21/usr/sbin/munin21:36:250
203052380,1sleep020460-21/usr/sbin/munin21:36:250
192792380,2sleep10-21swapper/121:31:301
192792380,2sleep10-21swapper/121:31:301
116412380,3sleep20-21swapper/200:06:242
116412380,3sleep20-21swapper/200:06:242
31773993729,6cyclictest32658-21threads22:06:301
31773993729,6cyclictest32658-21threads22:06:301
3177399372,7cyclictest181rcu_preempt20:46:161
3177399372,7cyclictest181rcu_preempt20:46:161
3177399370,4cyclictest181rcu_preempt23:11:171
3177399370,4cyclictest181rcu_preempt23:11:171
3177299373,33cyclictest201rcuc/020:11:160
3177299373,33cyclictest201rcuc/020:11:160
31772993727,4cyclictest0-21swapper/023:31:160
31772993727,4cyclictest0-21swapper/023:31:160
256122370,3chrt0-21swapper/300:41:303
31773993526,6cyclictest281ktimers/121:11:171
31773993526,6cyclictest281ktimers/121:11:171
3177299353,26cyclictest5089-21taskset20:56:190
31772993527,6cyclictest26638-21netstat21:51:240
31772993527,6cyclictest26638-21netstat21:51:240
31773993428,4cyclictest5690-21gdbus00:06:061
31773993428,4cyclictest5690-21gdbus00:06:061
31773993427,4cyclictest281ktimers/122:16:181
31773993427,4cyclictest281ktimers/122:16:181
3177399342,28cyclictest25913-21/usr/sbin/munin21:51:151
3177399342,28cyclictest25913-21/usr/sbin/munin21:51:151
3177299346,12cyclictest181rcu_preempt21:51:150
239522340,2sleep00-21swapper/023:11:200
239522340,2sleep00-21swapper/023:11:200
3177599331,23cyclictest11076-21chrt00:06:173
31773993328,3cyclictest24719-21cut00:41:211
31772993327,4cyclictest2282-21timerwakeupswit20:46:290
31772993327,4cyclictest2282-21timerwakeupswit20:46:290
31773993226,4cyclictest9372-21kernelversion19:41:211
31773993225,5cyclictest4088-21threads20:51:281
31773993225,5cyclictest4088-21threads20:51:281
31772993226,4cyclictest11181-21ntpd23:56:230
31772993226,4cyclictest11181-21ntpd23:56:230
31772993223,7cyclictest0-21swapper/019:46:210
45402310,3chrt0-21swapper/223:46:102
45402310,3chrt0-21swapper/223:46:102
3177499311,21cyclictest3266-21chrt19:26:182
3177499311,21cyclictest3266-21chrt19:26:182
31773993127,3cyclictest8601-21latency_hist21:06:051
31773993127,3cyclictest8601-21latency_hist21:06:051
31773993125,4cyclictest31771-21cyclictest20:11:211
31773993125,4cyclictest31771-21cyclictest20:11:211
31773993124,5cyclictest22896-21systemctl23:06:261
31773993123,6cyclictest7137-21fschecks_count19:36:191
31773993123,6cyclictest26002-21cut23:16:181
31773993123,6cyclictest26002-21cut23:16:181
3177399312,24cyclictest20134-21taskset23:01:161
3177399312,24cyclictest20134-21taskset23:01:161
3177399312,24cyclictest15765-21snmpd23:26:181
3177399312,24cyclictest15765-21snmpd23:26:181
3177299317,17cyclictest36350irq/47-eth021:26:160
3177299317,17cyclictest36350irq/47-eth021:26:160
31772993125,4cyclictest26047-21grep23:16:180
31772993125,4cyclictest26047-21grep23:16:180
31772993123,6cyclictest20379-21fschecks_time23:01:180
31772993123,6cyclictest20379-21fschecks_time23:01:180
31772993123,6cyclictest0-21swapper/020:26:130
31772993123,6cyclictest0-21swapper/020:26:130
3177599301,22cyclictest31657-21chrt22:06:183
3177599301,22cyclictest31657-21chrt22:06:183
3177499302,20cyclictest20134-21taskset23:01:162
3177499302,20cyclictest20134-21taskset23:01:162
31773993025,3cyclictest29302-21sendmail23:26:061
31773993025,3cyclictest29302-21sendmail23:26:061
31773993025,3cyclictest0-21swapper/122:31:091
31773993025,3cyclictest0-21swapper/122:31:091
31773993023,5cyclictest17543-21netstat00:21:261
31773993023,5cyclictest17543-21netstat00:21:261
31773993023,5cyclictest0-21swapper/122:46:151
31773993023,5cyclictest0-21swapper/122:46:151
31772993023,5cyclictest26337-21cut20:26:210
31772993023,5cyclictest26337-21cut20:26:210
31772993023,5cyclictest2557-21sshd22:14:020
31772993023,5cyclictest2557-21sshd22:14:020
31772993023,5cyclictest23065-21timerandwakeup21:41:270
31772993023,5cyclictest23065-21timerandwakeup21:41:270
31772993023,4cyclictest19024-21systemd-journal19:28:370
31772993023,4cyclictest19024-21systemd-journal19:28:370
3177299302,22cyclictest1365-21taskset19:21:180
3177299302,22cyclictest1365-21taskset19:21:180
205682300,3sleep20-21swapper/220:11:202
205682300,3sleep20-21swapper/220:11:202
166032300,8sleep116604-21fschecks_count21:26:201
166032300,8sleep116604-21fschecks_count21:26:201
31775992922,5cyclictest431rcuc/321:12:443
31775992922,5cyclictest431rcuc/321:12:443
3177399295,16cyclictest281ktimers/100:16:231
3177399295,16cyclictest281ktimers/100:16:231
3177399295,16cyclictest281ktimers/100:16:231
31773992921,6cyclictest4969-21fschecks_count22:21:191
31773992921,6cyclictest4969-21fschecks_count22:21:191
31772992924,3cyclictest32088-21head20:41:190
31772992924,3cyclictest32088-21head20:41:190
31772992924,3cyclictest24707-21head00:41:210
31772992923,4cyclictest798-21systemd-logind21:51:090
31772992923,4cyclictest5687-21polkitd22:36:320
31772992923,4cyclictest5687-21polkitd22:36:320
31772992923,3cyclictest0-21swapper/022:31:160
31772992923,3cyclictest0-21swapper/022:31:160
31772992923,3cyclictest0-21swapper/019:41:050
183342290,3sleep20-21swapper/221:31:192
183342290,3sleep20-21swapper/221:31:192
31775992823,3cyclictest431rcuc/322:49:163
31775992823,3cyclictest431rcuc/322:49:163
3177599282,19cyclictest5089-21chrt20:56:193
3177399283,19cyclictest3266-21runrttasks19:26:181
3177399283,19cyclictest3266-21runrttasks19:26:181
31773992823,3cyclictest0-21swapper/121:01:061
31773992820,6cyclictest12291-21sshd19:48:021
31772992820,6cyclictest19696-21sshd21:33:020
31772992820,6cyclictest19696-21sshd21:33:020
31772992820,5cyclictest0-21swapper/023:51:130
31772992820,5cyclictest0-21swapper/023:51:130
314862285,16sleep20-21swapper/219:15:532
311462286,15sleep10-21swapper/119:11:321
14352280,3chrt0-21swapper/219:21:192
14352280,3chrt0-21swapper/219:21:192
324142270,3chrt0-21swapper/320:41:243
324142270,3chrt0-21swapper/320:41:243
3177399272,20cyclictest26047-21acpi20:26:161
31773992721,3cyclictest0-21swapper/123:41:291
31773992721,3cyclictest0-21swapper/123:41:291
31773992720,5cyclictest0-21swapper/121:51:281
31773992720,5cyclictest0-21swapper/121:51:281
31773992718,6cyclictest20042-21latency_hist23:01:121
31773992718,6cyclictest20042-21latency_hist23:01:121
31772992722,3cyclictest0-21swapper/020:16:300
31772992722,3cyclictest0-21swapper/020:16:300
31772992720,5cyclictest29411-21sshd20:34:020
31772992720,5cyclictest29411-21sshd20:34:020
31775992622,3cyclictest431rcuc/323:21:163
31775992622,3cyclictest431rcuc/323:21:163
31775992621,4cyclictest431rcuc/300:12:413
31775992621,4cyclictest431rcuc/300:12:413
31775992620,4cyclictest431rcuc/321:32:303
31775992620,4cyclictest431rcuc/321:32:303
3177499262,17cyclictest10735-21taskset21:11:172
3177499262,17cyclictest10735-21taskset21:11:172
31773992621,2cyclictest0-21swapper/120:31:291
31773992621,2cyclictest0-21swapper/120:31:291
31773992620,4cyclictest0-21swapper/122:06:061
31773992619,5cyclictest14816-21fschecks_count19:56:201
31773992619,5cyclictest14816-21fschecks_count19:56:201
31773992618,6cyclictest14282-21sshd21:21:021
31773992618,6cyclictest14282-21sshd21:21:021
31773992618,6cyclictest1019-21sort23:36:231
31773992618,6cyclictest1019-21sort23:36:231
31773992617,7cyclictest19020-21cut00:26:211
31773992617,7cyclictest19020-21cut00:26:211
31772992619,5cyclictest5886-21timerandwakeup22:21:310
31772992619,5cyclictest5886-21timerandwakeup22:21:310
31772992619,5cyclictest23510-21sshd23:09:150
31772992617,6cyclictest3069-21df_inode22:16:210
31772992617,6cyclictest3069-21df_inode22:16:210
31772992613,9cyclictest31771-21cyclictest22:46:180
31772992613,9cyclictest31771-21cyclictest22:46:180
3177399254,5cyclictest181rcu_preempt20:06:181
3177399252,19cyclictest24048-21apt-get20:21:181
31773992519,4cyclictest15991-21sshd22:49:131
31773992519,4cyclictest15991-21sshd22:49:131
31773992518,5cyclictest667-21munin-node22:11:341
31773992518,5cyclictest667-21munin-node22:11:341
31773992518,4cyclictest0-21swapper/120:01:221
31773992518,4cyclictest0-21swapper/120:01:221
31773992518,3cyclictest0-21swapper/121:21:161
31773992518,3cyclictest0-21swapper/121:21:161
31773992517,5cyclictest23479-21sshd20:17:021
31773992517,5cyclictest23479-21sshd20:17:021
31773992515,8cyclictest31758-21cut23:31:201
31773992515,8cyclictest31758-21cut23:31:201
31772992519,4cyclictest19024-21systemd-journal20:41:060
31772992519,4cyclictest19024-21systemd-journal20:41:060
31772992517,7cyclictest20077-21awk20:11:060
31772992513,11cyclictest525-21rs:main0
31772992513,11cyclictest525-21rs:main0
31775992419,4cyclictest431rcuc/321:25:513
31775992419,4cyclictest431rcuc/321:25:513
31775992419,3cyclictest431rcuc/323:38:163
31775992419,3cyclictest431rcuc/323:38:163
31773992420,2cyclictest0-21swapper/121:56:221
31773992420,2cyclictest0-21swapper/121:56:221
31773992420,2cyclictest0-21swapper/120:36:261
31773992420,2cyclictest0-21swapper/120:36:261
31773992420,2cyclictest0-21swapper/119:56:061
31773992420,2cyclictest0-21swapper/119:26:051
31773992420,2cyclictest0-21swapper/119:26:051
31773992418,4cyclictest0-21swapper/122:31:281
31773992418,4cyclictest0-21swapper/122:31:281
31772992419,3cyclictest31771-21cyclictest21:21:280
31772992419,3cyclictest31771-21cyclictest21:21:280
31772992419,3cyclictest17085-21ntp_kernel_err22:51:260
31772992419,3cyclictest17085-21ntp_kernel_err22:51:260
31772992419,3cyclictest0-21swapper/022:56:210
31772992419,3cyclictest0-21swapper/022:56:210
31772992419,3cyclictest0-21swapper/000:31:210
31772992419,3cyclictest0-21swapper/000:31:210
31772992418,3cyclictest0-21swapper/023:41:280
31772992418,3cyclictest0-21swapper/023:41:280
31772992417,5cyclictest850-21latency_hist22:11:050
31772992417,5cyclictest850-21latency_hist22:11:050
31775992319,3cyclictest431rcuc/323:46:363
31775992319,3cyclictest431rcuc/323:46:363
31775992319,3cyclictest431rcuc/323:28:593
31775992319,3cyclictest431rcuc/323:28:593
31775992319,3cyclictest431rcuc/322:55:313
31775992319,3cyclictest431rcuc/322:55:313
31775992319,3cyclictest431rcuc/322:38:303
31775992319,3cyclictest431rcuc/322:38:303
31775992319,3cyclictest431rcuc/321:57:463
31775992319,3cyclictest431rcuc/321:57:463
31775992319,3cyclictest431rcuc/321:51:473
31775992319,3cyclictest431rcuc/321:51:473
31775992319,3cyclictest431rcuc/320:11:383
31775992319,3cyclictest431rcuc/320:11:383
31775992318,4cyclictest431rcuc/320:02:173
31775992318,4cyclictest431rcuc/320:02:173
31775992318,4cyclictest431rcuc/300:17:403
31775992318,4cyclictest431rcuc/300:17:403
31775992318,4cyclictest431rcuc/300:17:403
31775992318,3cyclictest431rcuc/320:49:493
31775992318,3cyclictest431rcuc/320:49:493
31775992318,3cyclictest431rcuc/320:36:173
31775992318,3cyclictest431rcuc/320:36:173
31773992317,4cyclictest0-21swapper/119:31:161
31773992317,4cyclictest0-21swapper/119:31:161
31772992317,4cyclictest9751-21sendmail_mailtr21:06:290
31772992317,4cyclictest9751-21sendmail_mailtr21:06:290
31775992218,3cyclictest431rcuc/300:33:473
31775992218,3cyclictest431rcuc/300:33:473
31775992218,3cyclictest431rcuc/300:26:163
31775992218,3cyclictest431rcuc/300:26:163
3177599221,15cyclictest23907-21chrt23:11:183
3177599221,15cyclictest23907-21chrt23:11:183
31773992218,2cyclictest0-21swapper/120:26:261
31773992218,2cyclictest0-21swapper/120:26:261
31773992218,2cyclictest0-21swapper/119:16:161
31773992218,2cyclictest0-21swapper/119:16:161
31773992215,5cyclictest13704-21ntp_kernel_pll_00:11:281
31773992215,5cyclictest13704-21ntp_kernel_pll_00:11:281
3177299221,19cyclictest10869-21date19:46:140
31775992117,3cyclictest431rcuc/323:59:303
31775992117,3cyclictest431rcuc/323:59:303
31775992117,3cyclictest431rcuc/323:53:403
31775992117,3cyclictest431rcuc/323:53:403
31775992117,3cyclictest431rcuc/323:06:043
31775992117,3cyclictest431rcuc/323:06:043
31775992117,3cyclictest431rcuc/322:12:093
31775992117,3cyclictest431rcuc/322:12:093
31775992117,3cyclictest431rcuc/321:08:023
31775992117,3cyclictest431rcuc/321:08:023
31775992117,3cyclictest431rcuc/319:57:473
31775992117,3cyclictest431rcuc/319:57:473
31775992117,3cyclictest431rcuc/319:38:383
31775992116,3cyclictest431rcuc/300:03:283
3177499213,9cyclictest0-21swapper/222:21:182
3177499213,9cyclictest0-21swapper/222:21:182
31772992116,3cyclictest0-21swapper/021:16:290
31772992116,3cyclictest0-21swapper/021:16:290
3177599202,13cyclictest0-21swapper/300:06:293
3177599202,13cyclictest0-21swapper/300:06:293
31775992016,3cyclictest431rcuc/321:19:193
31775992016,3cyclictest431rcuc/321:19:193
31775992016,3cyclictest431rcuc/320:33:533
31775992016,3cyclictest431rcuc/320:33:533
31775992016,3cyclictest431rcuc/319:43:243
31775992016,3cyclictest431rcuc/319:31:213
31775992016,3cyclictest431rcuc/319:31:213
31775992014,4cyclictest431rcuc/321:02:113
31775992014,4cyclictest431rcuc/321:02:113
3177499201,12cyclictest10892-21chrt00:06:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional