You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat Jul 27, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1694023623,0sleep00-21swapper/006:56:040
291350350,0irq/25-eth10-21swapper/306:52:273
1682723424,0sleep10-21swapper/106:54:451
1669623127,0sleep20-21swapper/206:53:102
339960,0migration/32941-21sh10:29:073
339960,0migration/327078-21sh12:26:083
339960,0migration/321625-21sh09:41:473
339960,0migration/321180-21munin-node08:57:413
339960,0migration/319024-21sh11:21:553
339960,0migration/318631-21sh11:00:473
339960,0migration/318472-21sh11:40:123
28415060,0irq/27-eth0-tx-26248-21chrt11:44:162
28415060,0irq/27-eth0-tx-0-21swapper/212:16:512
28415060,0irq/27-eth0-tx-0-21swapper/212:02:422
28415060,0irq/27-eth0-tx-0-21swapper/211:32:372
28415060,0irq/27-eth0-tx-0-21swapper/211:23:412
28415060,0irq/27-eth0-tx-0-21swapper/211:19:002
28415060,0irq/27-eth0-tx-0-21swapper/211:16:502
28415060,0irq/27-eth0-tx-0-21swapper/210:56:372
28415060,0irq/27-eth0-tx-0-21swapper/210:15:592
28415060,0irq/27-eth0-tx-0-21swapper/210:09:012
28415060,0irq/27-eth0-tx-0-21swapper/210:00:502
28415060,0irq/27-eth0-tx-0-21swapper/209:40:222
28415060,0irq/27-eth0-tx-0-21swapper/008:35:500
28405060,0irq/26-eth0-rx-8801-21ntp_kernel_pll_11:12:403
28405060,0irq/26-eth0-rx-29091-21sh10:07:073
28405060,0irq/26-eth0-rx-21151-21iostat_ios10:42:403
249960,0watchdog/226410-21cut07:27:382
249960,0watchdog/212167-21sed09:32:452
249960,0watchdog/20-21swapper/210:07:012
249960,0watchdog/20-21swapper/209:02:412
171249960,0cyclictest0-21swapper/211:11:162
171239960,0cyclictest0-21swapper/111:29:441
171219965,0cyclictest1193-21munin-node08:22:350
8225050,0irq/24-0000:00:18029-21latency_hist07:22:192
35150,0ktimersoftd/30-21swapper/310:40:543
339950,0migration/38722-21kthreadcore08:02:403
339950,0migration/3852-40yum-updatesd08:21:223
339950,0migration/37999-21aten_r1power_fr08:27:323
339950,0migration/36440-21sshd07:59:503
339950,0migration/34875-21kthreadcore12:12:373
339950,0migration/34714-21kthreadcore11:32:363
339950,0migration/331753-21munin-node07:52:453
339950,0migration/330028-21kthreadcore11:07:383
339950,0migration/328178-21missed_timers09:22:373
339950,0migration/322962-21munin-node09:42:353
339950,0migration/320528-21idleruntime12:02:343
339950,0migration/317265-21munin-node07:17:483
339950,0migration/316435-21kthreadcore11:57:393
339950,0migration/314200-21fschecks_time09:57:363
339950,0migration/313787-21cpuspeed_turbos08:07:303
339950,0migration/311449-21kthreadcore09:12:403
28415050,0irq/27-eth0-tx-6372-21iceccd08:43:210
28415050,0irq/27-eth0-tx-3685-21ssh10:31:192
28415050,0irq/27-eth0-tx-27670-21munin-node11:47:442
28415050,0irq/27-eth0-tx-0-21swapper/212:23:292
28415050,0irq/27-eth0-tx-0-21swapper/210:57:272
28415050,0irq/27-eth0-tx-0-21swapper/210:43:592
28415050,0irq/27-eth0-tx-0-21swapper/210:37:232
28415050,0irq/27-eth0-tx-0-21swapper/210:32:322
28415050,0irq/27-eth0-tx-0-21swapper/210:22:282
28415050,0irq/27-eth0-tx-0-21swapper/209:53:392
28415050,0irq/27-eth0-tx-0-21swapper/209:48:362
28415050,0irq/27-eth0-tx-0-21swapper/209:43:142
28415050,0irq/27-eth0-tx-0-21swapper/209:29:342
28415050,0irq/27-eth0-tx-0-21swapper/209:24:502
28415050,0irq/27-eth0-tx-0-21swapper/209:21:342
28415050,0irq/27-eth0-tx-0-21swapper/209:07:442
28415050,0irq/27-eth0-tx-0-21swapper/008:47:360
28415050,0irq/27-eth0-tx-0-21swapper/008:20:360
28405050,0irq/26-eth0-rx-9594-21kthreadcore09:52:383
28405050,0irq/26-eth0-rx-7320-21kthreadcore11:52:373
28405050,0irq/26-eth0-rx-6662-21kthreadcore10:52:393
28405050,0irq/26-eth0-rx-3197-21unixbench_singl10:07:453
28405050,0irq/26-eth0-rx-31274-21kthreadcore11:47:413
28405050,0irq/26-eth0-rx-20040-21perl11:02:313
28405050,0irq/26-eth0-rx-1588-21ssh11:28:343
28405050,0irq/26-eth0-rx-12847-21ps12:17:373
28405050,0irq/26-eth0-rx-0-21swapper/311:42:233
28405050,0irq/26-eth0-rx-0-21swapper/310:48:193
28405050,0irq/26-eth0-rx-0-21swapper/310:22:353
28405050,0irq/26-eth0-rx-0-21swapper/310:12:233
28405050,0irq/26-eth0-rx-0-21swapper/309:33:413
28405050,0irq/26-eth0-rx-0-21swapper/309:17:373
28405050,0irq/26-eth0-rx-0-21swapper/309:11:383
249950,0watchdog/2782-21missed_timers12:07:412
249950,0watchdog/217951-21ssh12:18:132
249950,0watchdog/2150772sleep208:54:092
249950,0watchdog/214684-21cut10:17:372
249950,0watchdog/20-21swapper/211:53:212
249950,0watchdog/20-21swapper/211:40:332
249950,0watchdog/20-21swapper/211:28:052
249950,0watchdog/20-21swapper/210:47:412
249950,0watchdog/20-21swapper/209:15:252
249950,0watchdog/20-21swapper/208:50:052
249950,0watchdog/20-21swapper/208:44:052
249950,0watchdog/20-21swapper/208:31:052
249950,0watchdog/20-21swapper/208:08:052
249950,0watchdog/20-21swapper/207:58:412
249950,0watchdog/20-21swapper/207:54:052
249950,0watchdog/20-21swapper/207:47:492
249950,0watchdog/20-21swapper/207:42:332
249950,0watchdog/20-21swapper/207:39:372
249950,0watchdog/20-21swapper/207:32:212
249950,0watchdog/20-21swapper/206:57:462
19150,0ktimersoftd/10-21swapper/110:35:491
171249955,0cyclictest0-21swapper/209:00:152
171249950,0cyclictest0-21swapper/211:03:322
171249950,0cyclictest0-21swapper/208:13:072
171239955,0cyclictest15366-21diskmemload11:07:021
171239955,0cyclictest0-21swapper/109:46:381
171239955,0cyclictest0-21swapper/109:00:361
171239950,0cyclictest0-21swapper/112:19:341
171219950,0cyclictest0-21swapper/011:51:190
171219950,0cyclictest0-21swapper/010:18:280
171219950,0cyclictest0-21swapper/009:58:450
179950,0migration/16638-21munin-node09:52:361
179950,0migration/132335-21munin-node07:32:351
179950,0migration/130440-21pmu-power07:27:411
179950,0migration/128664-21sh11:47:381
179950,0migration/128482-21kthreadcore07:52:381
179950,0migration/122633-21kthreadcore10:22:361
179950,0migration/113317-21taskset09:35:361
149950,0migration/09031-21proc_pri11:32:410
149950,0migration/06945-21fschecks_time09:32:330
149950,0migration/06321-21taskset07:58:250
149950,0migration/05376-21latency_hist09:52:180
149950,0migration/031673-21aten_r1power_po07:32:310
149950,0migration/031400-21kthreadcore10:47:370
149950,0migration/029583-21kthreadcore11:27:380
149950,0migration/028796-21latency09:02:400
149950,0migration/028525-21taskset09:43:430
149950,0migration/027929-21aten_r1power_vo12:07:300
149950,0migration/027063-21irqcore07:52:370
149950,0migration/026120-21kthreadcore08:37:370
149950,0migration/025996-21irqcore07:27:370
149950,0migration/024420-21kthreadcore10:02:390
149950,0migration/021992-21kthreadcore11:42:370
149950,0migration/021883-21kthreadcore07:22:370
149950,0migration/020365-21irqrtprio07:47:380
149950,0migration/017771-21sh11:57:500
149950,0migration/015873-21kthreadcore09:37:370
149950,0migration/015848-21cpuspeed_turbos08:57:300
149950,0migration/012647-21sh09:55:260
149950,0migration/010995-21sh10:34:280
339940,0migration/330807-21aten_r1power_vo09:27:293
339940,0migration/32929-21kthreadcore07:32:393
339940,0migration/325552-21kthreadcore07:02:403
339940,0migration/324332-21kthreadcore08:37:363
339940,0migration/310720-21kthreadcore07:37:373
28415040,0irq/27-eth0-tx-0-21swapper/008:27:330
28415040,0irq/27-eth0-tx-0-21swapper/006:57:430
28405040,0irq/26-eth0-rx-680-21kthreadcore09:47:393
28405040,0irq/26-eth0-rx-24090-21grep09:02:343
28405040,0irq/26-eth0-rx-0-21swapper/312:07:233
28405040,0irq/26-eth0-rx-0-21swapper/311:23:533
28405040,0irq/26-eth0-rx-0-21swapper/310:36:003
28405040,0irq/26-eth0-rx-0-21swapper/310:17:363
28405040,0irq/26-eth0-rx-0-21swapper/308:26:063
28405040,0irq/26-eth0-rx-0-21swapper/308:12:393
28405040,0irq/26-eth0-rx-0-21swapper/307:27:383
259940,0migration/230807-21kthreadcore08:17:392
259940,0migration/216250-21kthreadcore08:32:412
249940,0watchdog/20-21swapper/208:37:292
249940,0watchdog/20-21swapper/208:04:092
249940,0watchdog/20-21swapper/207:07:342
249940,0watchdog/20-21swapper/207:02:342
171249943,0cyclictest5895-21awk07:12:332
171249943,0cyclictest5895-21awk07:12:332
171249943,0cyclictest3425-21lldpd08:23:062
171249943,0cyclictest3425-21lldpd07:22:342
171249943,0cyclictest12470-21awk11:57:352
171239944,0cyclictest0-21swapper/111:42:311
171239944,0cyclictest0-21swapper/107:17:291
171239943,0cyclictest9849-21awk08:52:371
171239943,0cyclictest8886-21awk08:27:401
171239943,0cyclictest6887-21awk07:37:351
171239943,0cyclictest6445-21snmpd10:37:511
171239943,0cyclictest6445-21snmpd10:27:511
171239943,0cyclictest6445-21snmpd10:06:511
171239943,0cyclictest6445-21snmpd09:15:361
171239943,0cyclictest6445-21snmpd08:08:361
171239943,0cyclictest6445-21snmpd08:03:211
171239943,0cyclictest6445-21snmpd07:11:511
171239943,0cyclictest6445-21snmpd07:04:211
171239943,0cyclictest31521-21awk09:27:331
171239943,0cyclictest30337-21awk10:07:341
171239943,0cyclictest27512-21sadc12:27:191
171239943,0cyclictest13510-21awk07:42:331
171219944,0cyclictest0-21swapper/009:26:070
171219943,0cyclictest6445-21snmpd09:13:360
171219943,0cyclictest5750-21awk07:12:320
171219943,0cyclictest5750-21awk07:12:320
171219943,0cyclictest4392-21awk11:52:340
171219943,0cyclictest4378-21awk10:52:360
171219943,0cyclictest27503-21sadc12:07:190
171219943,0cyclictest27240-21sadc11:07:180
171219943,0cyclictest26696-21sadc08:17:180
171219943,0cyclictest24857-21awk07:02:380
171219943,0cyclictest20648-21awk12:22:330
171219943,0cyclictest14481-21awk08:07:330
171219943,0cyclictest12516-21awk12:17:340
179940,0migration/1964-21kthreadcore09:47:391
179940,0migration/17989-21kthreadcore07:12:361
179940,0migration/17989-21kthreadcore07:12:361
179940,0migration/17931-21kthreadcore10:12:391
179940,0migration/15507-21kthreadcore12:12:371
179940,0migration/128926-21kthreadcore11:07:381
179940,0migration/124408-21kthreadcore09:22:351
179940,0migration/12193-21taskset12:09:401
179940,0migration/120746-21kthreadcore07:22:361
179940,0migration/120261-21aten_r1power_cu08:12:321
179940,0migration/118502-21munin-node10:17:401
179940,0migration/115432-21irqrtprio09:37:361
179940,0migration/113554-21kthreadcore11:57:381
179940,0migration/112822-21kthreadcore10:57:371
169940,0watchdog/129972-21ssh09:04:011
169940,0watchdog/119588-21diskstats07:47:331
169940,0watchdog/119430-21ssh11:22:251
169940,0watchdog/117126-21munin-node11:37:451
169940,0watchdog/115804-21irqrtprio08:32:371
169940,0watchdog/114610-21iostat09:57:411
169940,0watchdog/111065-21ssh11:37:051
169940,0watchdog/110364-21ssh10:55:051
169940,0watchdog/10-21swapper/108:22:331
149940,0migration/025421-21kthreadcore10:42:420
149940,0migration/023725-21kthreadcore10:22:360
149940,0migration/018489-21diskmemload11:20:280
149940,0migration/01783-21missed_timers10:27:390
149940,0migration/017369-21timerandwakeup07:17:500
149940,0migration/015753-21kthreadcore10:57:380
28405030,0irq/26-eth0-rx-0-21swapper/308:32:313
28405030,0irq/26-eth0-rx-0-21swapper/307:42:283
28405030,0irq/26-eth0-rx-0-21swapper/307:22:283
28405030,0irq/26-eth0-rx-0-21swapper/307:12:293
28405030,0irq/26-eth0-rx-0-21swapper/307:12:293
171239933,0cyclictest23654-21kworker/1:011:52:241
171239933,0cyclictest23654-21kworker/1:011:17:201
171239933,0cyclictest23654-21kworker/1:011:12:281
171239933,0cyclictest23654-21kworker/1:010:47:201
171239933,0cyclictest23654-21kworker/1:009:17:251
171239933,0cyclictest23654-21kworker/1:009:07:411
171239933,0cyclictest23654-21kworker/1:008:47:211
171239933,0cyclictest23654-21kworker/1:008:42:251
171239933,0cyclictest23654-21kworker/1:008:37:231
171239933,0cyclictest23654-21kworker/1:008:17:251
171239933,0cyclictest23654-21kworker/1:006:57:231
171239932,0cyclictest6445-21snmpd07:57:211
171219932,0cyclictest9839-21ssh11:13:590
171219932,0cyclictest6134-21cpuspeed_turbos07:37:310
171219932,0cyclictest4889-21kthreadcore12:12:370
171219932,0cyclictest32237-21awk09:07:360
171219932,0cyclictest29586-21aten_r1power_vo10:07:300
171219932,0cyclictest25515-21sh11:22:440
171219932,0cyclictest18107-21ssh11:39:100
171219932,0cyclictest15366-21diskmemload09:19:410
171219932,0cyclictest13054-21awk10:37:350
171219932,0cyclictest12632-21basename07:42:280
171219932,0cyclictest12559-21sensors08:02:480
171219932,0cyclictest10843-21sensors_temp10:12:440
169930,0watchdog/10-21swapper/112:02:331
169930,0watchdog/10-21swapper/110:42:411
329920,0watchdog/33819-21proc_pri07:07:413
329920,0watchdog/30-21swapper/308:52:293
28405020,0irq/26-eth0-rx-0-21swapper/308:47:213
28405020,0irq/26-eth0-rx-0-21swapper/308:42:443
28405020,0irq/26-eth0-rx-0-21swapper/307:47:403
28405020,0irq/26-eth0-rx-0-21swapper/306:57:273
171219921,0cyclictest3067-17auditd08:52:180
171219921,0cyclictest30611-21munin-node07:07:280
171219921,0cyclictest0-21swapper/011:07:250
171219920,0cyclictest0-21swapper/009:27:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional