You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-25 - 14:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot6s.osadl.org (updated Mon Nov 25, 2024 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20638993838,0cyclictest8570-21fwupd07:57:090
20646992117,3cyclictest61550irq/125-lan11:49:132
20646992018,2cyclictest61550irq/125-lan12:10:222
20646992018,2cyclictest61550irq/125-lan09:20:112
20646992017,2cyclictest61550irq/125-lan11:09:062
20646992017,2cyclictest61550irq/125-lan11:04:372
20646992017,2cyclictest61550irq/125-lan09:50:132
20646992017,2cyclictest61550irq/125-lan09:47:342
20646991918,1cyclictest61550irq/125-lan12:05:172
20646991918,1cyclictest61550irq/125-lan11:17:212
20646991918,1cyclictest61550irq/125-lan08:00:122
20646991917,2cyclictest61550irq/125-lan11:51:572
20646991917,2cyclictest61550irq/125-lan10:08:432
20646991917,2cyclictest61550irq/125-lan10:03:452
20646991916,2cyclictest61550irq/125-lan10:50:482
20646991818,0cyclictest61550irq/125-lan09:33:432
20646991818,0cyclictest61550irq/125-lan07:35:442
20646991817,1cyclictest61550irq/125-lan11:55:242
20646991817,1cyclictest61550irq/125-lan11:34:182
20646991817,1cyclictest61550irq/125-lan10:49:212
20646991817,1cyclictest61550irq/125-lan10:25:142
20646991817,1cyclictest61550irq/125-lan09:40:112
20646991817,1cyclictest61550irq/125-lan09:25:132
20646991816,2cyclictest61550irq/125-lan12:15:172
20646991816,2cyclictest61550irq/125-lan11:25:122
20646991816,2cyclictest61550irq/125-lan11:20:012
20646991816,2cyclictest61550irq/125-lan08:50:142
20646991816,2cyclictest61550irq/125-lan07:15:182
20646991815,2cyclictest61550irq/125-lan12:35:102
20646991815,2cyclictest61550irq/125-lan11:41:012
20646991815,2cyclictest61550irq/125-lan10:31:542
20646991814,3cyclictest61550irq/125-lan11:11:222
20649991717,0cyclictest716-21snmpd11:35:213
20649991717,0cyclictest716-21snmpd08:55:303
20646991717,0cyclictest61550irq/125-lan10:55:132
20646991717,0cyclictest61550irq/125-lan10:40:542
20646991717,0cyclictest61550irq/125-lan08:20:022
20646991716,1cyclictest61550irq/125-lan12:25:092
20646991716,1cyclictest61550irq/125-lan12:20:042
20646991716,1cyclictest61550irq/125-lan11:35:582
20646991716,1cyclictest61550irq/125-lan10:39:582
20646991716,1cyclictest61550irq/125-lan10:10:202
20646991716,1cyclictest61550irq/125-lan09:58:502
20646991716,1cyclictest61550irq/125-lan07:45:202
20646991716,0cyclictest61550irq/125-lan09:16:102
20646991715,2cyclictest61550irq/125-lan12:30:122
20646991715,2cyclictest61550irq/125-lan12:01:522
20646991715,2cyclictest61550irq/125-lan09:35:152
20649991614,0cyclictest331-21hwrng11:29:133
20646991616,0cyclictest61550irq/125-lan08:40:512
20646991615,1cyclictest61550irq/125-lan10:20:102
20646991615,1cyclictest61550irq/125-lan09:10:142
20646991615,1cyclictest61550irq/125-lan09:05:172
20646991615,1cyclictest61550irq/125-lan08:10:122
20646991613,2cyclictest61550irq/125-lan08:05:192
20649991515,0cyclictest331-21hwrng08:01:213
20649991515,0cyclictest331-21hwrng07:23:283
20649991515,0cyclictest331-21hwrng07:11:113
20646991514,1cyclictest61550irq/125-lan09:00:112
20646991514,1cyclictest61550irq/125-lan07:30:152
20649991414,0cyclictest331-21hwrng07:19:223
20646991414,0cyclictest61550irq/125-lan10:19:072
20646991413,1cyclictest61550irq/125-lan08:45:152
20646991413,1cyclictest61550irq/125-lan08:35:492
20646991413,1cyclictest61550irq/125-lan08:25:132
20646991413,1cyclictest61550irq/125-lan08:15:282
20646991413,1cyclictest61550irq/125-lan07:50:142
20643991414,0cyclictest331-21hwrng12:34:461
20643991414,0cyclictest331-21hwrng10:37:001
20643991414,0cyclictest331-21hwrng10:33:571
20643991414,0cyclictest331-21hwrng10:12:261
20643991414,0cyclictest331-21hwrng07:38:501
20649991313,0cyclictest716-21snmpd12:32:113
20649991313,0cyclictest716-21snmpd12:22:193
20649991313,0cyclictest716-21snmpd11:45:333
20649991313,0cyclictest716-21snmpd11:04:143
20649991313,0cyclictest716-21snmpd10:11:093
20649991313,0cyclictest716-21snmpd07:42:073
20649991313,0cyclictest331-21hwrng11:13:523
20649991313,0cyclictest331-21hwrng10:50:193
20643991313,0cyclictest331-21hwrng09:54:001
20649991212,0cyclictest716-21snmpd10:25:493
20649991212,0cyclictest716-21snmpd10:18:233
20649991212,0cyclictest716-21snmpd09:23:493
20649991212,0cyclictest716-21snmpd07:55:023
20649991212,0cyclictest331-21hwrng11:15:553
20649991212,0cyclictest331-21hwrng09:25:193
20649991212,0cyclictest30864-21cat11:05:143
20646991211,1cyclictest61550irq/125-lan07:25:162
20646991211,1cyclictest61550irq/125-lan07:10:202
20646991210,2cyclictest61550irq/125-lan07:20:162
20638991212,0cyclictest331-21hwrng08:28:000
20649991111,0cyclictest716-21snmpd09:37:313
20649991111,0cyclictest716-21snmpd08:20:023
20649991111,0cyclictest716-21snmpd07:50:013
20649991111,0cyclictest331-21hwrng11:32:183
20649991111,0cyclictest331-21hwrng10:22:403
20649991111,0cyclictest331-21hwrng10:04:143
20649991111,0cyclictest331-21hwrng09:32:293
20649991111,0cyclictest331-21hwrng08:51:323
20649991111,0cyclictest331-21hwrng08:34:073
20649991111,0cyclictest331-21hwrng07:25:313
20646991110,1cyclictest61550irq/125-lan08:55:192
20646991110,1cyclictest61550irq/125-lan07:55:172
20643991111,0cyclictest331-21hwrng12:37:501
2064999109,0cyclictest331-21hwrng09:45:483
20649991010,0cyclictest716-21snmpd07:31:443
20649991010,0cyclictest331-21hwrng12:15:183
20649991010,0cyclictest331-21hwrng11:59:583
20649991010,0cyclictest331-21hwrng11:54:503
20649991010,0cyclictest331-21hwrng11:20:013
20649991010,0cyclictest331-21hwrng10:05:153
20649991010,0cyclictest331-21hwrng08:06:283
2064699109,1cyclictest61550irq/125-lan07:40:132
20646991010,0cyclictest61550irq/125-lan08:30:442
20643991010,0cyclictest331-21hwrng07:41:541
20638991010,0cyclictest331-21hwrng10:16:310
206499999,0cyclictest331-21hwrng12:10:123
206499999,0cyclictest331-21hwrng12:06:053
206499999,0cyclictest331-21hwrng12:00:583
206499999,0cyclictest331-21hwrng10:40:043
206439998,1cyclictest716-21snmpd08:47:511
206439998,1cyclictest716-21snmpd08:29:191
206439998,1cyclictest716-21snmpd07:30:071
206439998,1cyclictest716-21snmpd07:29:401
206439996,3cyclictest716-21snmpd10:08:291
206389998,1cyclictest716-21snmpd07:27:430
206499988,0cyclictest716-21snmpd12:27:103
206499988,0cyclictest716-21snmpd09:43:013
206499988,0cyclictest331-21hwrng09:57:043
206499988,0cyclictest331-21hwrng07:46:013
206439988,0cyclictest716-21snmpd11:16:081
206439988,0cyclictest716-21snmpd09:58:531
206439988,0cyclictest716-21snmpd07:20:181
206439986,2cyclictest716-21snmpd12:19:201
206439986,2cyclictest716-21snmpd11:55:341
206439986,2cyclictest716-21snmpd08:22:081
206439986,2cyclictest716-21snmpd07:11:091
206439985,3cyclictest716-21snmpd10:58:161
206439984,4cyclictest716-21snmpd11:33:531
206439982,2cyclictest13444-21ssh11:20:291
206389988,0cyclictest716-21snmpd10:56:430
206389988,0cyclictest716-21snmpd09:21:020
206389988,0cyclictest716-21snmpd08:04:290
206389988,0cyclictest331-21hwrng11:12:500
206389987,1cyclictest716-21snmpd08:30:210
206389987,1cyclictest716-21snmpd08:20:380
206389987,1cyclictest716-21snmpd07:10:000
206389986,2cyclictest716-21snmpd09:50:290
206499977,0cyclictest716-21snmpd10:39:033
206499977,0cyclictest331-21hwrng12:36:483
206499977,0cyclictest331-21hwrng10:56:273
206499977,0cyclictest331-21hwrng09:12:013
206499977,0cyclictest331-21hwrng09:04:503
206499977,0cyclictest331-21hwrng08:46:253
206499977,0cyclictest331-21hwrng08:43:213
206499977,0cyclictest331-21hwrng08:15:413
206439977,0cyclictest716-21snmpd12:11:561
206439977,0cyclictest716-21snmpd10:02:171
206439977,0cyclictest331-21hwrng09:25:191
206439976,1cyclictest716-21snmpd11:50:481
206439976,1cyclictest716-21snmpd09:02:101
206439976,1cyclictest716-21snmpd08:15:411
206439976,1cyclictest716-21snmpd07:53:521
206439976,1cyclictest716-21snmpd07:18:401
206439975,2cyclictest716-21snmpd10:29:361
206439974,3cyclictest716-21snmpd10:41:301
206439974,3cyclictest716-21snmpd10:21:241
206439973,4cyclictest716-21snmpd09:15:401
206439973,1cyclictest716-21snmpd11:03:491
206439973,1cyclictest716-21snmpd07:45:371
206439972,2cyclictest716-21snmpd09:10:011
206439971,2cyclictest31360-21ssh11:38:431
206389977,0cyclictest716-21snmpd12:13:490
206389977,0cyclictest716-21snmpd12:04:330
206389977,0cyclictest716-21snmpd07:53:590
206389976,1cyclictest716-21snmpd11:37:140
206389976,1cyclictest716-21snmpd11:23:330
206389976,1cyclictest716-21snmpd10:32:400
206389976,1cyclictest716-21snmpd08:41:420
206389976,1cyclictest716-21snmpd08:35:060
206389975,2cyclictest716-21snmpd12:05:090
206389975,2cyclictest716-21snmpd10:26:540
206499966,0cyclictest331-21hwrng09:05:523
206499966,0cyclictest331-21hwrng08:10:343
206499966,0cyclictest331-21hwrng07:35:453
206439966,0cyclictest716-21snmpd09:20:311
206439966,0cyclictest716-21snmpd08:06:531
206439965,1cyclictest716-21snmpd08:32:391
206439965,1cyclictest716-21snmpd07:55:291
206439963,3cyclictest29595-21gltestperf08:50:131
206439962,2cyclictest716-21snmpd11:42:361
206439962,1cyclictest22598-21ssh09:31:321
206439962,1cyclictest0-21swapper/112:25:441
206439961,2cyclictest9277-21fschecks_count08:00:131
206439961,2cyclictest716-21snmpd09:38:351
206439961,2cyclictest5397-21ssh11:12:281
206439961,1cyclictest27606-21ssh12:05:101
206439961,1cyclictest1183-21ssh11:08:151
206389966,0cyclictest716-21snmpd12:38:360
206389966,0cyclictest716-21snmpd12:33:590
206389966,0cyclictest716-21snmpd10:07:500
206389966,0cyclictest331-21hwrng09:42:440
206389965,1cyclictest716-21snmpd09:59:000
206389965,1cyclictest716-21snmpd09:15:370
206439955,0cyclictest331-21hwrng11:49:421
206439951,1cyclictest23103-21ntpq12:00:181
206439950,2cyclictest19378-21/usr/sbin/munin10:50:241
206439950,1cyclictest171rcu_preempt10:49:471
206389955,0cyclictest716-21snmpd11:58:460
206389955,0cyclictest716-21snmpd11:54:330
206389955,0cyclictest716-21snmpd10:51:370
206389954,1cyclictest716-21snmpd12:20:460
206389954,1cyclictest716-21snmpd10:45:250
206389954,1cyclictest716-21snmpd10:41:570
206389954,1cyclictest716-21snmpd08:13:020
206389954,1cyclictest716-21snmpd07:46:070
206389954,1cyclictest4495-21diskmemload12:25:200
206389950,2cyclictest244522ssh09:35:050
206499943,1cyclictest716-21snmpd09:51:453
206499943,1cyclictest17976-21dump-pmu-power10:50:013
206499940,4cyclictest9678-21ntpq09:15:203
206439944,0cyclictest331-21hwrng08:40:161
206439944,0cyclictest331-21hwrng08:10:341
206439943,1cyclictest271ktimers/112:24:051
206439942,2cyclictest26541-21cron10:20:001
206439940,1cyclictest0-21swapper/111:26:571
206389944,0cyclictest331-21hwrng11:40:300
206389944,0cyclictest331-21hwrng11:33:190
206389944,0cyclictest331-21hwrng10:21:380
206389944,0cyclictest331-21hwrng09:49:540
206389944,0cyclictest331-21hwrng07:20:240
206389944,0cyclictest331-21hwrng07:16:180
206389943,1cyclictest716-21snmpd11:07:260
206389943,1cyclictest716-21snmpd10:00:590
206389943,1cyclictest3282-21kworker/u8:2+nfsiod11:45:180
206389943,1cyclictest151ktimers/011:27:480
206389942,2cyclictest9324-21ssh11:16:210
206389942,2cyclictest9246-21ssh12:17:310
206389942,2cyclictest3526-21ntpq09:05:150
206389942,2cyclictest32154-21ps08:55:190
206389942,2cyclictest21977-21awk09:30:240
206389942,2cyclictest2042-21ntpq07:40:190
206389942,2cyclictest19846-21ntpq10:10:230
206499932,1cyclictest23654-21ntpq08:35:183
206499930,3cyclictest431ktimers/310:31:553
206499930,2cyclictest721-21runrttasks11:40:453
206439933,0cyclictest331-21hwrng08:55:371
206439933,0cyclictest331-21hwrng08:35:091
206439932,1cyclictest1-21systemd09:48:301
206439930,2cyclictest30472-21ssh09:42:001
206389932,1cyclictest0-21swapper/008:45:170
206389931,2cyclictest26449-21grep11:00:120
206389930,3cyclictest0-21swapper/007:30:160
206389930,2cyclictest539-21sendmail-msp09:00:010
206389930,1cyclictest4495-21diskmemload09:10:010
206389930,0cyclictest0-21swapper/009:25:120
206499921,1cyclictest431ktimers/308:25:013
206439920,1cyclictest2874-21grep09:05:111
206389920,2cyclictest31708-21df07:35:170
206389920,2cyclictest29125-21gpgv08:50:060
206389920,2cyclictest14970-21munin-node08:15:100
206389920,2cyclictest11039-21/usr/sbin/munin08:05:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional