You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 08:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Nov 22, 2024 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1471425756,0sleep20-21swapper/218:48:162
291350480,0irq/25-eth10-21swapper/318:50:243
1566523726,0sleep00-21swapper/018:51:070
1568723026,0sleep10-21swapper/118:51:201
339970,0migration/318490-21munin-node23:18:093
339970,0migration/318490-21munin-node23:18:093
249970,0watchdog/20-21swapper/221:28:452
339960,0migration/38732-21sh00:17:533
28415060,0irq/27-eth0-tx-0-21swapper/223:32:542
28415060,0irq/27-eth0-tx-0-21swapper/222:04:382
28415060,0irq/27-eth0-tx-0-21swapper/221:03:592
28415060,0irq/27-eth0-tx-0-21swapper/019:23:150
28415060,0irq/27-eth0-tx-0-21swapper/019:18:380
28405060,0irq/26-eth0-rx-633-21kthreadcore21:03:163
28405060,0irq/26-eth0-rx-2299-21seq23:52:463
249960,0watchdog/20-21swapper/221:18:372
149960,0migration/018068-21sh23:17:440
339950,0migration/39828-21idleruntime00:18:123
339950,0migration/38428-21idleruntime20:48:133
339950,0migration/37361-21kthreadcore23:33:123
339950,0migration/33540-21hwlatdetect23:53:183
339950,0migration/331052-21fschecks_time19:28:123
339950,0migration/330983-21kthreadcore23:03:163
339950,0migration/329629-21munin-node19:03:103
339950,0migration/329184-21kthreadcore19:48:153
339950,0migration/328849-21kthreadcore20:38:123
339950,0migration/328815-21sh20:59:343
339950,0migration/328805-21idleruntime22:43:113
339950,0migration/326709-21irqcore20:13:143
339950,0migration/326404-21munin-node22:18:163
339950,0migration/323935-21irqrtprio18:58:153
339950,0migration/323765-21munin-node19:43:223
339950,0migration/323378-21taskset19:21:453
339950,0migration/322326-21aten_r1power_po21:18:073
339950,0migration/321767-21latency_hist21:37:583
339950,0migration/320435-21kthreadcore20:08:133
339950,0migration/320009-21munin-node23:43:103
339950,0migration/315457-21kthreadcore20:28:143
339950,0migration/314493-21kthreadcore19:38:143
339950,0migration/312083-21munin-node23:38:113
339950,0migration/310891-21kthreadcore20:23:153
28415050,0irq/27-eth0-tx-6685-21munin-node20:23:150
28415050,0irq/27-eth0-tx-20867-21ssh21:56:492
28415050,0irq/27-eth0-tx-0-21swapper/222:33:022
28415050,0irq/27-eth0-tx-0-21swapper/221:58:112
28415050,0irq/27-eth0-tx-0-21swapper/220:58:202
28415050,0irq/27-eth0-tx-0-21swapper/200:09:112
28415050,0irq/27-eth0-tx-0-21swapper/019:41:370
28415050,0irq/27-eth0-tx-0-21swapper/019:30:120
28415050,0irq/27-eth0-tx-0-21swapper/019:17:450
28405050,0irq/26-eth0-rx-8927-21ssh23:08:423
28405050,0irq/26-eth0-rx-417-21kthreadcore21:43:153
28405050,0irq/26-eth0-rx-31963-21kthreadcore21:23:143
28405050,0irq/26-eth0-rx-31468-21kthreadcore22:03:153
28405050,0irq/26-eth0-rx-18828-21ssh22:34:393
28405050,0irq/26-eth0-rx-11645-21irqrtprio19:13:133
28405050,0irq/26-eth0-rx-0-21swapper/323:28:463
28405050,0irq/26-eth0-rx-0-21swapper/322:52:383
28405050,0irq/26-eth0-rx-0-21swapper/322:40:573
28405050,0irq/26-eth0-rx-0-21swapper/322:29:343
28405050,0irq/26-eth0-rx-0-21swapper/322:23:103
28405050,0irq/26-eth0-rx-0-21swapper/322:08:543
28405050,0irq/26-eth0-rx-0-21swapper/321:58:093
28405050,0irq/26-eth0-rx-0-21swapper/321:14:113
28405050,0irq/26-eth0-rx-0-21swapper/321:11:023
28405050,0irq/26-eth0-rx-0-21swapper/300:08:143
28405050,0irq/26-eth0-rx-0-21swapper/300:01:143
27150,0ktimersoftd/20-21swapper/200:17:112
249950,0watchdog/22682-21kthreadcore19:53:172
249950,0watchdog/220545-21kthreadcore19:43:172
249950,0watchdog/2120322sleep220:01:412
249950,0watchdog/20-21swapper/223:53:082
249950,0watchdog/20-21swapper/223:49:322
249950,0watchdog/20-21swapper/223:43:202
249950,0watchdog/20-21swapper/223:40:362
249950,0watchdog/20-21swapper/223:19:042
249950,0watchdog/20-21swapper/223:19:042
249950,0watchdog/20-21swapper/223:14:002
249950,0watchdog/20-21swapper/223:09:362
249950,0watchdog/20-21swapper/222:59:002
249950,0watchdog/20-21swapper/222:53:562
249950,0watchdog/20-21swapper/222:38:442
249950,0watchdog/20-21swapper/222:28:162
249950,0watchdog/20-21swapper/222:15:532
249950,0watchdog/20-21swapper/221:49:092
249950,0watchdog/20-21swapper/221:43:452
249950,0watchdog/20-21swapper/221:25:172
249950,0watchdog/20-21swapper/221:14:372
249950,0watchdog/20-21swapper/221:08:452
249950,0watchdog/20-21swapper/220:53:172
249950,0watchdog/20-21swapper/220:41:212
249950,0watchdog/20-21swapper/220:28:412
249950,0watchdog/20-21swapper/220:03:172
249950,0watchdog/20-21swapper/219:40:212
249950,0watchdog/20-21swapper/219:27:012
249950,0watchdog/20-21swapper/219:17:372
249950,0watchdog/20-21swapper/200:07:202
179950,0migration/128876-21sh21:19:471
179950,0migration/127759-21latency_hist22:42:581
179950,0migration/124849-21kthreadcore21:58:151
179950,0migration/122425-21kthreadcore23:43:141
179950,0migration/113289-21taskset21:11:221
179950,0migration/111434-21aten_r1power_en22:53:061
169950,0watchdog/131979-21kthreadcore23:03:161
169950,0watchdog/110888-21kthreadcore20:48:171
158979954,0cyclictest9543-21kworker/1:120:23:181
158979954,0cyclictest4314-21kworker/1:023:56:221
158979954,0cyclictest4314-21kworker/1:023:26:321
158979953,0cyclictest4314-21kworker/1:023:39:001
158979950,0cyclictest0-21swapper/123:52:171
158979950,0cyclictest0-21swapper/123:02:281
158979950,0cyclictest0-21swapper/122:24:471
158979950,0cyclictest0-21swapper/121:44:261
158979950,0cyclictest0-21swapper/121:29:461
158969955,0cyclictest0-21swapper/022:33:070
158969955,0cyclictest0-21swapper/021:12:430
158969950,0cyclictest0-21swapper/022:50:180
158969950,0cyclictest0-21swapper/021:49:080
158969950,0cyclictest0-21swapper/021:19:490
158969950,0cyclictest0-21swapper/000:04:530
149950,0migration/09441-21taskset23:55:010
149950,0migration/07879-21kthreadcore21:28:140
149950,0migration/07178-21taskset20:45:400
149950,0migration/05559-21kthreadcore23:33:120
149950,0migration/03020-21sh22:47:190
149950,0migration/030154-21kthreadcore23:03:160
149950,0migration/029829-21taskset21:02:210
149950,0migration/027667-21kthreadcore23:23:150
149950,0migration/026377-21taskset22:38:400
149950,0migration/023519-21kthreadcore23:43:140
149950,0migration/022510-21kthreadcore19:43:170
149950,0migration/022014-21kthreadcore23:18:150
149950,0migration/022014-21kthreadcore23:18:150
149950,0migration/021697-21fschecks_time22:18:110
149950,0migration/020386-21taskset21:55:230
149950,0migration/01898-21kthreadcore19:53:160
149950,0migration/017197-21switchtime22:53:180
149950,0migration/010333-21munin-node22:28:230
339940,0migration/38687-21kthreadcore21:28:143
339940,0migration/36344-21kthreadcore19:08:153
339940,0migration/35398-21taskset19:57:023
339940,0migration/330561-21kthreadcore23:23:163
339940,0migration/321324-21kthreadcore22:58:153
339940,0migration/320841-21sh21:56:443
28415040,0irq/27-eth0-tx-0-21swapper/223:58:022
28415040,0irq/27-eth0-tx-0-21swapper/223:34:272
28415040,0irq/27-eth0-tx-0-21swapper/222:43:072
28415040,0irq/27-eth0-tx-0-21swapper/222:23:222
28415040,0irq/27-eth0-tx-0-21swapper/019:58:100
28415040,0irq/27-eth0-tx-0-21swapper/019:03:170
28405040,0irq/26-eth0-rx-0-21swapper/323:13:533
28405040,0irq/26-eth0-rx-0-21swapper/322:53:063
28405040,0irq/26-eth0-rx-0-21swapper/322:13:413
28405040,0irq/26-eth0-rx-0-21swapper/321:48:143
28405040,0irq/26-eth0-rx-0-21swapper/321:34:043
28405040,0irq/26-eth0-rx-0-21swapper/320:57:173
28405040,0irq/26-eth0-rx-0-21swapper/320:43:213
28405040,0irq/26-eth0-rx-0-21swapper/320:03:153
28405040,0irq/26-eth0-rx-0-21swapper/300:03:193
259940,0migration/24265-21fschecks_count19:08:112
259940,0migration/225340-21munin-node20:33:172
259940,0migration/223195-21kthreadcore22:18:142
249940,0watchdog/20-21swapper/223:03:002
249940,0watchdog/20-21swapper/222:48:162
249940,0watchdog/20-21swapper/220:45:492
249940,0watchdog/20-21swapper/220:23:012
249940,0watchdog/20-21swapper/220:18:252
249940,0watchdog/20-21swapper/220:08:412
249940,0watchdog/20-21swapper/219:51:212
249940,0watchdog/20-21swapper/219:33:212
249940,0watchdog/20-21swapper/219:28:172
249940,0watchdog/20-21swapper/219:18:412
249940,0watchdog/20-21swapper/219:03:172
249940,0watchdog/20-21swapper/218:53:052
249940,0watchdog/20-21swapper/200:18:362
179940,0migration/19674-21kthreadcore22:28:181
179940,0migration/18713-21kthreadcore19:58:161
179940,0migration/128868-21kthreadcore20:38:121
179940,0migration/125771-21kthreadcore18:58:161
179940,0migration/125665-21latency_hist20:12:581
179940,0migration/123232-21kthreadcore20:33:161
179940,0migration/122846-21fschecks_time20:58:111
179940,0migration/118581-21munin-node22:13:201
179940,0migration/117006-21latency_hist19:17:581
179940,0migration/114677-21kthreadcore20:03:151
179940,0migration/113647-21aten_r1power_po21:53:111
179940,0migration/113605-21kthreadcore19:13:141
169940,0watchdog/19068-21kthreadcore21:48:171
169940,0watchdog/18089-21cut22:48:161
169940,0watchdog/126666-21fschecks_count00:08:121
169940,0watchdog/11699-21awk20:43:131
169940,0watchdog/10-21swapper/119:33:211
169940,0watchdog/10-21swapper/119:28:171
158989943,0cyclictest8523-21awk20:48:142
158989943,0cyclictest6445-21snmpd21:33:102
158989943,0cyclictest5985-21awk22:08:122
158989943,0cyclictest3425-21lldpd18:59:302
158989943,0cyclictest27287-21awk23:23:142
158989943,0cyclictest26531-21awk20:13:122
158989943,0cyclictest22744-21awk21:38:112
158979944,0cyclictest9543-21kworker/1:122:35:341
158979944,0cyclictest9543-21kworker/1:122:18:161
158979944,0cyclictest9543-21kworker/1:122:08:081
158979944,0cyclictest9543-21kworker/1:122:04:301
158979944,0cyclictest9543-21kworker/1:121:38:081
158979944,0cyclictest9543-21kworker/1:121:33:201
158979944,0cyclictest9543-21kworker/1:121:24:501
158979944,0cyclictest9543-21kworker/1:121:13:221
158979944,0cyclictest9543-21kworker/1:121:03:001
158979944,0cyclictest9543-21kworker/1:120:53:101
158979944,0cyclictest9543-21kworker/1:120:28:081
158979944,0cyclictest9543-21kworker/1:120:18:141
158979944,0cyclictest9543-21kworker/1:119:53:221
158979944,0cyclictest9543-21kworker/1:119:48:521
158979944,0cyclictest9543-21kworker/1:119:23:201
158979944,0cyclictest9543-21kworker/1:119:08:121
158979944,0cyclictest9543-21kworker/1:119:03:121
158979944,0cyclictest4314-21kworker/1:023:36:581
158979944,0cyclictest4314-21kworker/1:023:18:081
158979944,0cyclictest4314-21kworker/1:023:18:081
158979944,0cyclictest4314-21kworker/1:023:13:021
158979944,0cyclictest4314-21kworker/1:023:08:101
158979944,0cyclictest4314-21kworker/1:000:17:521
158979944,0cyclictest4314-21kworker/1:000:03:061
158979943,0cyclictest4314-21kworker/1:023:58:081
158979943,0cyclictest4314-21kworker/1:023:28:281
158979943,0cyclictest4314-21kworker/1:000:19:101
158979943,0cyclictest3425-21lldpd22:43:391
158979943,0cyclictest20043-21awk20:08:121
158969944,0cyclictest0-21swapper/020:03:020
158969943,0cyclictest877-21awk20:18:120
158969943,0cyclictest6445-21snmpd22:00:550
158969943,0cyclictest6445-21snmpd20:39:550
158969943,0cyclictest5615-21basename21:27:580
158969943,0cyclictest25463-21sadc20:12:580
158969943,0cyclictest23715-21awk18:58:140
158969943,0cyclictest2229-21awk00:13:160
158969943,0cyclictest20955-21awk20:33:140
158969943,0cyclictest20266-21awk22:58:140
158969943,0cyclictest16959-21awk18:53:130
149940,0migration/08802-21uptime23:08:210
149940,0migration/02468-21ksmtuned23:30:200
149940,0migration/022451-21fschecks_time21:38:090
149940,0migration/020875-21latency_hist22:17:580
149940,0migration/015645-21kthreadcore20:28:140
149940,0migration/010167-21nfs_client19:33:170
339930,0migration/310125-21kthreadcore19:58:163
28415030,0irq/27-eth0-tx-0-21swapper/020:53:010
28405030,0irq/26-eth0-rx-0-21swapper/320:33:093
28405030,0irq/26-eth0-rx-0-21swapper/319:37:173
28405030,0irq/26-eth0-rx-0-21swapper/319:23:123
28405030,0irq/26-eth0-rx-0-21swapper/318:53:093
158979933,0cyclictest9543-21kworker/1:119:43:161
158979933,0cyclictest9543-21kworker/1:119:38:001
158979933,0cyclictest9543-21kworker/1:118:53:241
158969933,0cyclictest28155-21awk23:48:140
158969933,0cyclictest0-21swapper/021:05:000
158969932,0cyclictest4261-21sh22:06:160
158969932,0cyclictest30795-21awk21:43:130
158969932,0cyclictest29895-21awk22:23:130
158969932,0cyclictest28499-21grep00:08:160
158969932,0cyclictest16774-21grep21:33:170
158969932,0cyclictest10494-21kthreadcore20:48:160
158969930,0cyclictest9597-21perl00:18:120
158969930,0cyclictest0-21swapper/021:13:190
149930,0migration/011977-21runrttasks22:10:180
28405020,0irq/26-eth0-rx-0-21swapper/320:18:073
158969921,0cyclictest3411-21lldpd20:13:000
158969921,0cyclictest10582-21ssh23:58:050
158969920,0cyclictest3650-21latency_hist19:07:580
158969920,0cyclictest0-21swapper/023:38:010
158969920,0cyclictest0-21swapper/019:47:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional