You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-25 - 14:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2s.osadl.org (updated Mon Nov 25, 2024 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2206224948,0sleep00-21swapper/006:49:550
291350440,0irq/25-eth10-21swapper/306:49:243
2209523524,0sleep10-21swapper/106:50:161
2218123425,0sleep20-21swapper/206:51:172
28415070,0irq/27-eth0-tx-1531-21ssh10:33:082
8225060,0irq/24-0000:00:0-21swapper/209:03:242
4160,0ktimersoftd/00-21swapper/010:56:170
35160,0ktimersoftd/30-21swapper/311:10:283
339960,0migration/38599-21idleruntime11:38:053
339960,0migration/36826-21sh11:15:433
339960,0migration/3369-21munin-node11:33:073
339960,0migration/331766-21sh11:32:503
339960,0migration/330871-21kthreadcore09:23:073
339960,0migration/323271-21sh11:06:163
339960,0migration/322386-21sh11:24:063
339960,0migration/320179-21sh09:38:303
339960,0migration/315430-21sh10:38:493
339960,0migration/314477-21sh11:19:363
339960,0migration/311555-21sh09:55:343
28415060,0irq/27-eth0-tx-15015-21diskmemload09:15:072
28415060,0irq/27-eth0-tx-12418-21munin-node07:58:110
28415060,0irq/27-eth0-tx-0-21swapper/212:20:332
28415060,0irq/27-eth0-tx-0-21swapper/212:08:492
28415060,0irq/27-eth0-tx-0-21swapper/212:03:522
28415060,0irq/27-eth0-tx-0-21swapper/211:58:432
28415060,0irq/27-eth0-tx-0-21swapper/211:54:372
28415060,0irq/27-eth0-tx-0-21swapper/211:49:512
28415060,0irq/27-eth0-tx-0-21swapper/211:47:412
28415060,0irq/27-eth0-tx-0-21swapper/211:33:272
28415060,0irq/27-eth0-tx-0-21swapper/211:29:182
28415060,0irq/27-eth0-tx-0-21swapper/211:26:352
28415060,0irq/27-eth0-tx-0-21swapper/211:20:252
28415060,0irq/27-eth0-tx-0-21swapper/211:11:402
28415060,0irq/27-eth0-tx-0-21swapper/211:06:592
28415060,0irq/27-eth0-tx-0-21swapper/210:38:222
28415060,0irq/27-eth0-tx-0-21swapper/210:28:062
28415060,0irq/27-eth0-tx-0-21swapper/210:23:462
28415060,0irq/27-eth0-tx-0-21swapper/210:09:302
28415060,0irq/27-eth0-tx-0-21swapper/210:02:382
28415060,0irq/27-eth0-tx-0-21swapper/209:52:302
28415060,0irq/27-eth0-tx-0-21swapper/209:43:452
28415060,0irq/27-eth0-tx-0-21swapper/209:24:542
28415060,0irq/27-eth0-tx-0-21swapper/209:12:272
28415060,0irq/27-eth0-tx-0-21swapper/208:58:362
28415060,0irq/27-eth0-tx-0-21swapper/008:09:500
28405060,0irq/26-eth0-rx-30010-21ssh11:48:223
28405060,0irq/26-eth0-rx-28761-40yum-updatesd-he09:21:443
28405060,0irq/26-eth0-rx-15406-21basename08:53:043
249960,0watchdog/213503-21latency10:58:092
249960,0watchdog/20-21swapper/211:13:572
249960,0watchdog/20-21swapper/209:33:132
249960,0watchdog/20-21swapper/208:18:092
223849966,0cyclictest15015-21diskmemload09:02:241
223839960,0cyclictest0-21swapper/011:46:280
149960,0migration/019212-21gltestperf10:23:050
4150,0ktimersoftd/00-21swapper/011:29:230
339950,0migration/3964-21kthreadcore12:13:093
339950,0migration/37520-21irqrtprio09:33:043
339950,0migration/37148-21latency08:43:083
339950,0migration/332760-21kthreadcore08:38:083
339950,0migration/331760-21latency_hist11:52:513
339950,0migration/326983-21kthreadcore07:18:133
339950,0migration/325944-21kthreadcore08:58:093
339950,0migration/320982-21idleruntime08:28:063
339950,0migration/319607-21taskset08:24:023
339950,0migration/318519-21aten_r1power_en10:23:013
339950,0migration/318329-21latency_hist10:22:503
339950,0migration/317568-21kthreadcore12:03:093
339950,0migration/315831-21open_inodes07:08:093
339950,0migration/314560-21runrttasks12:20:583
339950,0migration/312539-21aten_r1power_po07:58:013
339950,0migration/312151-21kthreadcore10:58:083
339950,0migration/31095-21taskset10:32:423
28415050,0irq/27-eth0-tx-6372-21iceccd07:32:180
28415050,0irq/27-eth0-tx-14255-21expr09:38:042
28415050,0irq/27-eth0-tx-0-21swapper/211:38:442
28415050,0irq/27-eth0-tx-0-21swapper/210:48:052
28415050,0irq/27-eth0-tx-0-21swapper/008:53:230
28415050,0irq/27-eth0-tx-0-21swapper/008:16:160
28415050,0irq/27-eth0-tx-0-21swapper/007:38:030
28415050,0irq/27-eth0-tx-0-21swapper/007:13:050
28415050,0irq/27-eth0-tx-0-21swapper/007:02:200
28405050,0irq/26-eth0-rx-9689-21ssh10:14:443
28405050,0irq/26-eth0-rx-6547-21runrttasks12:01:223
28405050,0irq/26-eth0-rx-31816-21irqcore09:03:073
28405050,0irq/26-eth0-rx-12687-21munin-node07:33:073
28405050,0irq/26-eth0-rx-0-21swapper/310:50:063
28405050,0irq/26-eth0-rx-0-21swapper/310:33:193
28405050,0irq/26-eth0-rx-0-21swapper/310:04:513
28405050,0irq/26-eth0-rx-0-21swapper/310:01:183
28405050,0irq/26-eth0-rx-0-21swapper/307:48:023
28405050,0irq/26-eth0-rx-0-21swapper/307:23:123
259950,0migration/24254-21kthreadcore07:48:082
259950,0migration/2346-21cpuspeed_turbos10:53:052
259950,0migration/21285-21kthreadcore07:23:162
249950,0watchdog/2829-21switchtime08:38:132
249950,0watchdog/223683-21munin-node07:18:092
249950,0watchdog/20-21swapper/212:13:412
249950,0watchdog/20-21swapper/210:43:492
249950,0watchdog/20-21swapper/210:17:532
249950,0watchdog/20-21swapper/210:03:212
249950,0watchdog/20-21swapper/209:29:132
249950,0watchdog/20-21swapper/209:19:452
249950,0watchdog/20-21swapper/208:53:052
249950,0watchdog/20-21swapper/208:50:492
249950,0watchdog/20-21swapper/208:46:332
249950,0watchdog/20-21swapper/208:27:572
249950,0watchdog/20-21swapper/208:26:532
249950,0watchdog/20-21swapper/208:03:092
249950,0watchdog/20-21swapper/207:58:052
249950,0watchdog/20-21swapper/207:06:012
223849955,0cyclictest0-21swapper/109:41:211
223849955,0cyclictest0-21swapper/109:22:071
223849954,0cyclictest27872chrt10:33:141
223849953,0cyclictest4401-21kworker/1:210:39:101
223849953,0cyclictest30391-21kworker/1:010:52:401
223839950,0cyclictest0-21swapper/011:14:270
19150,0ktimersoftd/10-21swapper/111:29:231
179950,0migration/19529-21kthreadcore10:58:081
179950,0migration/15247-21fschecks_time07:28:061
179950,0migration/131958-21latency_hist11:12:501
179950,0migration/131006-21kthreadcore10:28:101
179950,0migration/127646-21latency_hist10:07:501
179950,0migration/123013-21sh11:25:521
179950,0migration/113226-21kthreadcore10:18:081
179950,0migration/113038-21taskset08:20:561
169950,0watchdog/18471-21kthreadcore10:13:131
169950,0watchdog/10-21swapper/111:08:051
149950,0migration/09855-21kthreadcore08:18:090
149950,0migration/07151-21kthreadcore10:13:130
149950,0migration/0514-21sh10:30:310
149950,0migration/04825-21kthreadcore08:43:070
149950,0migration/029908-21taskset12:08:300
149950,0migration/027905-21latency06:53:070
149950,0migration/026288-21kthreadcore11:08:080
149950,0migration/025946-21kthreadcore07:18:120
149950,0migration/019996-21latency_hist08:27:500
149950,0migration/016572-21kthreadcore09:58:090
149950,0migration/016212-21latency10:18:100
149950,0migration/015726-21kthreadcore09:38:090
149950,0migration/013235-21taskset09:11:390
149950,0migration/011477-21kthreadcore10:38:090
149950,0migration/010770-21kthreadcore11:38:070
339940,0migration/34796-21diskmemload09:28:353
339940,0migration/31821-21kthreadcore06:58:083
339940,0migration/314884-21taskset08:51:313
339940,0migration/310156-21kthreadcore08:18:093
28415040,0irq/27-eth0-tx-0-21swapper/008:03:240
28415040,0irq/27-eth0-tx-0-21swapper/007:53:010
28405040,0irq/26-eth0-rx-22026-21kthreadcore10:43:073
28405040,0irq/26-eth0-rx-0-21swapper/312:09:383
28405040,0irq/26-eth0-rx-0-21swapper/311:43:043
28405040,0irq/26-eth0-rx-0-21swapper/310:53:103
28405040,0irq/26-eth0-rx-0-21swapper/310:08:593
28405040,0irq/26-eth0-rx-0-21swapper/309:47:593
28405040,0irq/26-eth0-rx-0-21swapper/309:42:573
28405040,0irq/26-eth0-rx-0-21swapper/309:13:123
28405040,0irq/26-eth0-rx-0-21swapper/309:07:533
28405040,0irq/26-eth0-rx-0-21swapper/308:13:043
28405040,0irq/26-eth0-rx-0-21swapper/308:08:093
28405040,0irq/26-eth0-rx-0-21swapper/307:53:083
259940,0migration/27365-21kthreadcore07:53:072
259940,0migration/230095-21kthreadcore08:08:102
259940,0migration/226033-21kthreadcore06:53:072
259940,0migration/211813-21munin-node07:33:042
249940,0watchdog/26277-21kthreadcore10:13:132
249940,0watchdog/20-21swapper/207:13:012
249940,0watchdog/20-21swapper/206:58:012
223859944,0cyclictest22310-21kworker/2:109:52:542
223859944,0cyclictest22310-21kworker/2:108:33:062
223859944,0cyclictest22310-21kworker/2:108:12:562
223859944,0cyclictest22310-21kworker/2:107:43:182
223859944,0cyclictest22310-21kworker/2:107:37:582
223859944,0cyclictest22310-21kworker/2:107:28:202
223859943,0cyclictest10543-21sort07:08:012
223849944,0cyclictest4401-21kworker/1:210:42:541
223849944,0cyclictest4401-21kworker/1:210:01:241
223849944,0cyclictest4401-21kworker/1:209:25:061
223849944,0cyclictest4401-21kworker/1:208:56:301
223849944,0cyclictest4401-21kworker/1:208:09:181
223849944,0cyclictest4401-21kworker/1:208:04:221
223849944,0cyclictest4401-21kworker/1:207:43:521
223849944,0cyclictest4401-21kworker/1:207:18:081
223849944,0cyclictest4401-21kworker/1:207:09:101
223849944,0cyclictest4401-21kworker/1:207:03:281
223849944,0cyclictest394-21kworker/1:112:07:561
223849944,0cyclictest394-21kworker/1:111:43:081
223849944,0cyclictest394-21kworker/1:111:33:401
223849943,0cyclictest8825-21awk11:18:061
223849943,0cyclictest7104-21awk09:08:051
223849943,0cyclictest4401-21kworker/1:207:33:061
223849943,0cyclictest394-21kworker/1:112:17:521
223849943,0cyclictest3425-21lldpd09:57:051
223849943,0cyclictest3425-21lldpd09:51:011
223849943,0cyclictest3425-21lldpd07:55:131
223849943,0cyclictest32594-21awk12:13:071
223849943,0cyclictest30137-21awk06:58:051
223849943,0cyclictest28797-21awk10:08:081
223849943,0cyclictest19410-21awk10:23:061
223849943,0cyclictest17849-21awk07:13:061
223849943,0cyclictest17112-21awk11:03:071
223839944,0cyclictest0-21swapper/012:05:320
223839944,0cyclictest0-21swapper/009:25:590
223839943,0cyclictest6445-21snmpd12:18:190
223839943,0cyclictest6445-21snmpd10:09:040
223839943,0cyclictest6445-21snmpd09:21:190
223839943,0cyclictest634-21irqcore11:33:090
223839943,0cyclictest3425-21lldpd09:42:570
223839943,0cyclictest3110-21irqbalance11:47:550
223839943,0cyclictest3110-21irqbalance10:58:050
223839943,0cyclictest3110-21irqbalance10:48:150
223839943,0cyclictest3110-21irqbalance10:45:350
223839943,0cyclictest3110-21irqbalance09:53:450
223839943,0cyclictest3110-21irqbalance09:13:350
223839943,0cyclictest3110-21irqbalance09:02:550
223839943,0cyclictest3110-21irqbalance08:59:350
223839943,0cyclictest3110-21irqbalance08:48:050
223839943,0cyclictest3110-21irqbalance08:37:550
223839943,0cyclictest3110-21irqbalance08:33:050
223839943,0cyclictest3110-21irqbalance07:47:550
223839943,0cyclictest3110-21irqbalance07:43:250
223839943,0cyclictest3110-21irqbalance07:33:350
223839943,0cyclictest23720-21munin-run11:27:500
179940,0migration/16135-21sh11:54:281
179940,0migration/1542-21kthreadcore09:03:081
179940,0migration/15080-21runrttasks07:48:341
179940,0migration/132371-21kthreadcore07:23:161
179940,0migration/118840-21gltestperf07:38:061
179940,0migration/118650-21kthreadcore08:23:091
179940,0migration/11849-21aten_r1power_en08:43:011
179940,0migration/118372-21unixbench_singl07:58:141
179940,0migration/116622-21if_eth1.212:03:071
179940,0migration/113456-21kthreadcore08:48:071
169940,0watchdog/14591-21ssh09:27:531
169940,0watchdog/130600-21ssh11:50:051
169940,0watchdog/114723-21ssh11:40:331
169940,0watchdog/110749-21kthreadcore11:58:091
169940,0watchdog/10-21swapper/109:33:011
169940,0watchdog/10-21swapper/108:38:051
169940,0watchdog/10-21swapper/108:28:051
169940,0watchdog/10-21swapper/108:13:091
149940,0migration/0944-21kthreadcore11:53:100
149940,0migration/05758-21sh12:13:540
149940,0migration/030332-21irqcore09:48:070
149940,0migration/025487-21mdstat08:28:100
149940,0migration/010654-21kthreadcore09:33:050
149940,0migration/010041-21kthreadcore11:18:070
28415030,0irq/27-eth0-tx-0-21swapper/007:08:000
28405030,0irq/26-eth0-rx-0-21swapper/307:38:033
28405030,0irq/26-eth0-rx-0-21swapper/307:28:023
28405030,0irq/26-eth0-rx-0-21swapper/307:16:253
28405030,0irq/26-eth0-rx-0-21swapper/307:03:013
223849933,0cyclictest4401-21kworker/1:209:12:521
223849933,0cyclictest13722-21kworker/1:006:53:061
223849933,0cyclictest1003-21awk10:53:081
223849932,0cyclictest3425-21lldpd08:32:511
223849932,0cyclictest21529-21basename09:43:001
223839932,0cyclictest4315-21tr10:33:150
223839932,0cyclictest14311-21sh11:59:520
223839932,0cyclictest0-21swapper/007:03:010
223839931,0cyclictest0-21swapper/010:03:090
223839930,0cyclictest17577-21grep11:03:080
223839930,0cyclictest0-21swapper/007:23:120
149930,0migration/05090-21runrttasks09:29:460
28405020,0irq/26-eth0-rx-24239-21kthreadcore06:53:063
28405020,0irq/26-eth0-rx-0-21swapper/308:32:563
28405020,0irq/26-eth0-rx-0-21swapper/308:03:123
28405020,0irq/26-eth0-rx-0-21swapper/307:43:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional