You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-21 - 21:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Nov 21, 2024 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1884325958,0sleep10-21swapper/106:50:121
1884725857,0sleep00-21swapper/006:50:150
291350450,0irq/25-eth10-21swapper/306:50:433
1874223425,0sleep20-21swapper/206:49:052
19147991111,0cyclictest0-21swapper/008:55:240
19147991010,0cyclictest25360-21sensors11:08:200
28415070,0irq/27-eth0-tx-0-21swapper/210:28:572
191479970,0cyclictest0-21swapper/011:34:440
35160,0ktimersoftd/30-21swapper/308:53:103
339960,0migration/37455-21sh09:55:453
339960,0migration/3555-21sh11:55:443
339960,0migration/323107-21sh10:06:583
339960,0migration/319308-21latency_hist08:57:593
28415060,0irq/27-eth0-tx-20835-21ssh10:24:452
28415060,0irq/27-eth0-tx-11605-21ssh10:40:022
28415060,0irq/27-eth0-tx-0-21swapper/212:03:002
28415060,0irq/27-eth0-tx-0-21swapper/211:51:302
28415060,0irq/27-eth0-tx-0-21swapper/211:40:212
28415060,0irq/27-eth0-tx-0-21swapper/211:26:232
28415060,0irq/27-eth0-tx-0-21swapper/210:44:222
28415060,0irq/27-eth0-tx-0-21swapper/210:00:222
28415060,0irq/27-eth0-tx-0-21swapper/007:39:400
28415060,0irq/27-eth0-tx-0-21swapper/007:05:550
28405060,0irq/26-eth0-rx-26356-21fschecks_count12:13:163
28405060,0irq/26-eth0-rx-0-21swapper/310:58:113
249960,0watchdog/211387-21diskmemload11:34:412
249960,0watchdog/20-21swapper/209:15:222
249960,0watchdog/20-21swapper/207:38:142
249960,0watchdog/20-21swapper/207:13:102
191489960,0cyclictest0-21swapper/110:42:221
149960,0migration/08294-21missed_timers11:18:200
149960,0migration/024704-21taskset11:49:440
8225050,0irq/24-0000:00:19407-21latency_hist09:22:592
4150,0ktimersoftd/00-21swapper/011:31:460
339950,0migration/39218-21sh11:41:053
339950,0migration/35715-21taskset10:15:363
339950,0migration/35692-21kthreadcore08:43:143
339950,0migration/33491-21gltestperf09:33:163
339950,0migration/331764-21timerandwakeup06:58:203
339950,0migration/329607-21taskset08:37:153
339950,0migration/328883-21kthreadcore11:33:233
339950,0migration/328729-21kthreadcore10:08:163
339950,0migration/327203-21cpuspeed_turbos07:23:113
339950,0migration/325198-21kthreadcore10:48:183
339950,0migration/323657-21kthreadcore09:23:163
339950,0migration/322541-21kthreadcore11:08:153
339950,0migration/321916-21taskset08:06:413
339950,0migration/32171-21fschecks_count11:58:133
339950,0migration/320503-21kthreadcore12:08:173
339950,0migration/319303-21sh10:44:363
339950,0migration/318600-21taskset09:20:153
28415050,0irq/27-eth0-tx-2621-21ssh09:08:322
28415050,0irq/27-eth0-tx-13636-21munin-node07:13:170
28415050,0irq/27-eth0-tx-0-21swapper/212:18:582
28415050,0irq/27-eth0-tx-0-21swapper/212:10:232
28415050,0irq/27-eth0-tx-0-21swapper/211:32:552
28415050,0irq/27-eth0-tx-0-21swapper/211:18:362
28415050,0irq/27-eth0-tx-0-21swapper/211:13:412
28415050,0irq/27-eth0-tx-0-21swapper/211:08:382
28415050,0irq/27-eth0-tx-0-21swapper/210:22:232
28415050,0irq/27-eth0-tx-0-21swapper/209:40:282
28415050,0irq/27-eth0-tx-0-21swapper/209:20:332
28415050,0irq/27-eth0-tx-0-21swapper/209:03:002
28415050,0irq/27-eth0-tx-0-21swapper/208:58:112
28415050,0irq/27-eth0-tx-0-21swapper/006:55:520
28405050,0irq/26-eth0-rx-9224-21if_err_eth1.209:58:203
28405050,0irq/26-eth0-rx-345-21cut10:33:163
28405050,0irq/26-eth0-rx-27377-21ssh09:28:073
28405050,0irq/26-eth0-rx-26021-21diskstats09:48:123
28405050,0irq/26-eth0-rx-18964-21grep10:23:183
28405050,0irq/26-eth0-rx-14645-21chrt07:13:153
28405050,0irq/26-eth0-rx-11875-21aten_r1power_vo11:03:113
28405050,0irq/26-eth0-rx-0-21swapper/312:18:143
28405050,0irq/26-eth0-rx-0-21swapper/312:07:403
28405050,0irq/26-eth0-rx-0-21swapper/311:48:153
28405050,0irq/26-eth0-rx-0-21swapper/311:25:493
28405050,0irq/26-eth0-rx-0-21swapper/311:13:263
28405050,0irq/26-eth0-rx-0-21swapper/310:38:173
28405050,0irq/26-eth0-rx-0-21swapper/310:19:443
28405050,0irq/26-eth0-rx-0-21swapper/309:44:003
28405050,0irq/26-eth0-rx-0-21swapper/309:04:453
249950,0watchdog/20-21swapper/212:15:172
249950,0watchdog/20-21swapper/211:54:292
249950,0watchdog/20-21swapper/211:04:262
249950,0watchdog/20-21swapper/210:58:582
249950,0watchdog/20-21swapper/210:48:302
249950,0watchdog/20-21swapper/210:14:262
249950,0watchdog/20-21swapper/210:09:582
249950,0watchdog/20-21swapper/210:03:142
249950,0watchdog/20-21swapper/209:53:302
249950,0watchdog/20-21swapper/209:52:142
249950,0watchdog/20-21swapper/209:45:062
249950,0watchdog/20-21swapper/209:33:142
249950,0watchdog/20-21swapper/209:28:502
249950,0watchdog/20-21swapper/208:48:022
249950,0watchdog/20-21swapper/208:38:462
249950,0watchdog/20-21swapper/208:28:182
249950,0watchdog/20-21swapper/208:23:022
249950,0watchdog/20-21swapper/208:12:022
249950,0watchdog/20-21swapper/207:59:542
249950,0watchdog/20-21swapper/207:53:182
249950,0watchdog/20-21swapper/207:52:382
249950,0watchdog/20-21swapper/207:44:142
249950,0watchdog/20-21swapper/207:29:022
249950,0watchdog/20-21swapper/207:22:542
249950,0watchdog/20-21swapper/207:03:222
249950,0watchdog/20-21swapper/206:58:022
249950,0watchdog/20-21swapper/206:57:502
191499955,0cyclictest0-21swapper/212:00:012
191499955,0cyclictest0-21swapper/211:45:592
191499955,0cyclictest0-21swapper/208:53:022
191489955,0cyclictest0-21swapper/111:41:201
191489955,0cyclictest0-21swapper/109:23:231
191489953,0cyclictest17562-21kworker/1:107:38:191
191489950,0cyclictest0-21swapper/112:08:271
191489950,0cyclictest0-21swapper/111:55:031
191489950,0cyclictest0-21swapper/109:40:151
191489950,0cyclictest0-21swapper/109:15:201
191489950,0cyclictest0-21swapper/108:56:061
191489950,0cyclictest0-21swapper/108:33:241
191479950,0cyclictest0-21swapper/010:25:030
191479950,0cyclictest0-21swapper/009:28:370
191479950,0cyclictest0-21swapper/009:13:100
179950,0migration/129373-21kthreadcore11:13:171
179950,0migration/12788-21taskset10:55:381
179950,0migration/126970-21aten_r1power_en07:23:101
179950,0migration/125128-21munin-node08:58:211
179950,0migration/111894-21sh09:18:051
149950,0migration/09995-21kthreadcore07:33:150
149950,0migration/08864-21taskset12:21:240
149950,0migration/04154-21kthreadcore11:38:160
149950,0migration/03713-21sh10:34:120
149950,0migration/03280-21irqcore07:53:160
149950,0migration/029511-21taskset08:36:120
149950,0migration/028910-21taskset10:31:190
149950,0migration/027451-21cpuspeed_turbos09:03:110
149950,0migration/027365-21irqrtprio11:53:190
149950,0migration/026526-21fschecks_time12:13:170
149950,0migration/025845-21sh08:59:180
149950,0migration/024319-21sh12:09:290
149950,0migration/021355-21kthreadcore09:23:150
149950,0migration/019927-21kthreadcore10:03:170
149950,0migration/01834-21kthreadcore10:13:150
149950,0migration/018088-21aten_r1power_fr09:43:100
149950,0migration/017630-21switchtime11:03:220
149950,0migration/017204-21kthreadcore08:03:190
149950,0migration/012882-21sh10:18:360
149950,0migration/011754-21if_eth1.211:23:140
339940,0migration/38893-21kthreadcore07:08:163
339940,0migration/38259-21kthreadcore08:18:183
339940,0migration/37820-21kthreadcore11:18:193
339940,0migration/34573-21kthreadcore07:03:163
339940,0migration/328521-21taskset08:11:093
339940,0migration/326918-21sendmail_mailtr07:43:243
339940,0migration/323834-21kthreadcore10:28:163
339940,0migration/322698-21kthreadcore11:28:173
339940,0migration/311756-21kthreadcore08:23:153
339940,0migration/311294-21irqcore11:43:153
28415040,0irq/27-eth0-tx-1967-21kthreadcore10:33:162
28415040,0irq/27-eth0-tx-0-21swapper/210:53:052
28415040,0irq/27-eth0-tx-0-21swapper/008:28:140
28405040,0irq/26-eth0-rx-0-21swapper/310:53:103
28405040,0irq/26-eth0-rx-0-21swapper/309:38:113
28405040,0irq/26-eth0-rx-0-21swapper/309:13:453
28405040,0irq/26-eth0-rx-0-21swapper/309:11:033
28405040,0irq/26-eth0-rx-0-21swapper/307:53:113
28405040,0irq/26-eth0-rx-0-21swapper/307:28:153
249940,0watchdog/20-21swapper/208:43:102
249940,0watchdog/20-21swapper/208:33:182
249940,0watchdog/20-21swapper/208:05:222
249940,0watchdog/20-21swapper/207:08:302
2295240,0sleep00-21swapper/010:54:050
191499943,0cyclictest8800-21awk07:33:132
191499943,0cyclictest6445-21snmpd08:20:052
191499943,0cyclictest6445-21snmpd08:13:352
191499943,0cyclictest27792-21awk07:23:132
191489944,0cyclictest17720-21kworker/1:212:13:171
191489944,0cyclictest17720-21kworker/1:212:06:451
191489944,0cyclictest17720-21kworker/1:211:58:291
191489944,0cyclictest17720-21kworker/1:211:23:111
191489944,0cyclictest17720-21kworker/1:210:33:091
191489944,0cyclictest17720-21kworker/1:209:28:291
191489944,0cyclictest17562-21kworker/1:108:52:191
191489944,0cyclictest17562-21kworker/1:108:46:031
191489944,0cyclictest17562-21kworker/1:108:29:551
191489944,0cyclictest17562-21kworker/1:108:19:271
191489944,0cyclictest17562-21kworker/1:108:13:071
191489944,0cyclictest17562-21kworker/1:107:43:191
191489944,0cyclictest17562-21kworker/1:107:18:291
191489944,0cyclictest17562-21kworker/1:107:13:571
191489944,0cyclictest17562-21kworker/1:107:09:391
191489944,0cyclictest17562-21kworker/1:107:07:451
191489944,0cyclictest17562-21kworker/1:106:58:111
191489944,0cyclictest17562-21kworker/1:106:54:031
191489943,0cyclictest9676-21kthreadcore09:58:231
191489943,0cyclictest8865-21awk07:33:141
191489943,0cyclictest6445-21snmpd11:48:211
191489943,0cyclictest6445-21snmpd11:19:061
191489943,0cyclictest6445-21snmpd10:20:501
191489943,0cyclictest6445-21snmpd10:13:051
191489943,0cyclictest6445-21snmpd09:53:051
191489943,0cyclictest6445-21snmpd09:51:051
191489943,0cyclictest30779-21awk08:38:131
191489943,0cyclictest27573-21awk11:33:221
191489943,0cyclictest2480-21awk12:18:131
191489943,0cyclictest2056-21awk07:28:131
191489943,0cyclictest17720-21kworker/1:210:48:571
191489943,0cyclictest17562-21kworker/1:108:08:231
191489943,0cyclictest17562-21kworker/1:107:57:591
191489940,0cyclictest0-21swapper/108:24:151
191479943,0cyclictest6445-21snmpd08:40:200
191479943,0cyclictest6445-21snmpd07:49:490
191479943,0cyclictest4186-21awk08:18:150
191479943,0cyclictest23138-21awk08:08:130
191479943,0cyclictest21506-21awk10:48:150
191479943,0cyclictest15368-21sadc08:02:590
191479943,0cyclictest13684-21awk10:43:130
191479943,0cyclictest12799-21awk09:18:150
191479943,0cyclictest11283-21awk09:38:160
191479943,0cyclictest11088-21awk11:43:140
191479943,0cyclictest10694-21perf08:47:590
191479940,0cyclictest28922-21munin-node08:13:120
191479940,0cyclictest10114-21munin-run09:37:590
19140,0ktimersoftd/122306-21sh10:03:581
179940,0migration/16571-21kthreadcore07:53:171
179940,0migration/13741-21if_eth1.209:33:181
179940,0migration/13190-21sh09:10:391
179940,0migration/123513-21ntp_kernel_pll_09:43:191
179940,0migration/122689-21fschecks_time10:28:121
179940,0migration/120998-21kthreadcore11:08:151
179940,0migration/12079-21taskset09:07:281
179940,0migration/114938-21kthreadcore11:43:171
179940,0migration/114882-21fschecks_count10:23:131
179940,0migration/114781-21kthreadcore11:03:161
169940,0watchdog/14423-21fschecks_time10:58:141
169940,0watchdog/13622-21ksmtuned11:28:131
169940,0watchdog/132373-21kthreadcore07:48:181
169940,0watchdog/118268-21needreboot10:43:181
149940,0migration/09232-21kthreadcore07:08:160
149940,0migration/06162-21kthreadcore11:58:180
149940,0migration/028910-21kthreadcore06:58:140
149940,0migration/022026-21kthreadcore07:18:150
28405030,0irq/26-eth0-rx-0-21swapper/308:52:553
28405030,0irq/26-eth0-rx-0-21swapper/308:28:103
28405030,0irq/26-eth0-rx-0-21swapper/307:59:253
28405030,0irq/26-eth0-rx-0-21swapper/307:48:243
28405030,0irq/26-eth0-rx-0-21swapper/307:33:203
28405030,0irq/26-eth0-rx-0-21swapper/307:18:123
28405030,0irq/26-eth0-rx-0-21swapper/306:53:123
191489933,0cyclictest17720-21kworker/1:210:08:031
191479933,0cyclictest27847-21awk07:23:140
191479932,0cyclictest6445-21snmpd07:31:000
191479932,0cyclictest3878-21basename09:12:590
191479932,0cyclictest26526-21awk09:48:140
191479932,0cyclictest23697-21grep07:43:190
191479932,0cyclictest1562-21awk09:53:150
191479932,0cyclictest15369-21ntpq08:23:180
191479932,0cyclictest13915-21ssh09:58:260
191479931,0cyclictest8517-21kthreadcore10:58:180
191479930,0cyclictest28114-21perl11:13:130
149930,0migration/030732-21sh10:11:590
28405020,0irq/26-eth0-rx-0-21swapper/308:38:103
28405020,0irq/26-eth0-rx-0-21swapper/308:13:053
28405020,0irq/26-eth0-rx-0-21swapper/307:39:183
191489920,0cyclictest15576-21sed08:02:591
191479921,0cyclictest110272sleep008:49:070
191479920,0cyclictest0-21swapper/012:02:590
191479920,0cyclictest0-21swapper/010:38:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional