You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 06:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot6.osadl.org (updated Wed Apr 02, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1619621780,6sleep02506899cyclictest23:09:420
1538321760,5sleep32508699cyclictest23:48:523
2942721430,4sleep32508699cyclictest01:19:453
1518911000,3ptp4l401ktimersoftd/321:04:213
19812880,3sleep00-21swapper/000:49:230
25086997663,6cyclictest41-21ksoftirqd/300:14:343
25086997258,6cyclictest41-21ksoftirqd/300:29:303
25086997157,7cyclictest41-21ksoftirqd/322:34:313
157492680,2sleep10-21swapper/102:20:301
25086996755,4cyclictest41-21ksoftirqd/300:19:343
230032660,3sleep00-21swapper/002:30:530
307922620,2sleep00-21swapper/000:43:490
88462610,2sleep20-21swapper/200:18:222
66742610,5sleep26675-21latency_hist00:54:232
132952610,4sleep013272-21munin-run02:19:250
25086996049,6cyclictest41-21ksoftirqd/300:39:353
249572604,10sleep20-21swapper/221:09:132
25086995947,6cyclictest41-21ksoftirqd/300:59:353
93522580,4sleep02506899cyclictest21:39:450
26392580,1sleep10-21swapper/123:32:321
151891580,1ptp4l401ktimersoftd/322:05:143
151891580,1ptp4l401ktimersoftd/301:27:293
151891570,1ptp4l401ktimersoftd/302:15:173
2507499567,5cyclictest25-21ksoftirqd/101:11:271
151891560,1ptp4l401ktimersoftd/322:28:573
25086995545,5cyclictest41-21ksoftirqd/323:39:333
25086995542,5cyclictest41-21ksoftirqd/321:09:323
25074995528,3cyclictest25-21ksoftirqd/100:19:221
25068995526,5cyclictest9-21ksoftirqd/001:24:230
25068995525,4cyclictest9-21ksoftirqd/000:19:220
25086995443,5cyclictest41-21ksoftirqd/321:29:333
25086995442,6cyclictest41-21ksoftirqd/301:09:313
25068995427,4cyclictest9-21ksoftirqd/023:54:210
243172540,1sleep30-21swapper/322:14:493
151891540,1ptp4l401ktimersoftd/300:54:393
25074995326,4cyclictest25-21ksoftirqd/123:34:451
25074995324,3cyclictest25-21ksoftirqd/123:59:211
25068995326,4cyclictest9-21ksoftirqd/001:14:530
25068995325,4cyclictest9-21ksoftirqd/001:59:370
246652534,38sleep00-21swapper/021:05:240
151891530,1ptp4l401ktimersoftd/322:55:383
25074995226,4cyclictest25-21ksoftirqd/121:49:391
25074995225,4cyclictest25-21ksoftirqd/100:39:451
25074995211,9cyclictest25-21ksoftirqd/123:50:551
2506899526,7cyclictest9-21ksoftirqd/002:19:370
25068995226,4cyclictest9-21ksoftirqd/000:09:400
25068995225,4cyclictest9-21ksoftirqd/000:59:520
25068995225,4cyclictest9-21ksoftirqd/000:54:420
25068995225,10cyclictest9-21ksoftirqd/000:29:360
25068995224,4cyclictest9-21ksoftirqd/022:29:200
25068995224,4cyclictest9-21ksoftirqd/002:09:410
25068995224,4cyclictest9-21ksoftirqd/001:34:450
2508699513,10cyclictest41-21ksoftirqd/322:54:203
25074995125,4cyclictest25-21ksoftirqd/123:59:391
25074995125,4cyclictest25-21ksoftirqd/123:59:391
25074995125,4cyclictest25-21ksoftirqd/121:14:351
25074995125,3cyclictest25-21ksoftirqd/100:49:481
2506899513,7cyclictest9-21ksoftirqd/002:29:250
25068995125,5cyclictest9-21ksoftirqd/023:39:420
25068995125,4cyclictest9-21ksoftirqd/023:54:380
25068995125,4cyclictest9-21ksoftirqd/022:19:480
25068995125,3cyclictest9-21ksoftirqd/000:34:430
25068995124,4cyclictest9-21ksoftirqd/023:14:450
25068995124,4cyclictest9-21ksoftirqd/022:39:370
25068995124,4cyclictest9-21ksoftirqd/002:34:580
25068995122,4cyclictest9-21ksoftirqd/000:49:320
25068995121,7cyclictest9-21ksoftirqd/023:49:210
247492514,33sleep10-21swapper/121:06:301
25086995015,3cyclictest41-21ksoftirqd/323:09:393
25074995026,4cyclictest25-21ksoftirqd/101:54:471
25074995024,4cyclictest25-21ksoftirqd/121:29:391
25074995024,4cyclictest25-21ksoftirqd/102:14:481
25074995024,4cyclictest25-21ksoftirqd/100:38:091
25074995024,3cyclictest25-21ksoftirqd/122:29:201
25074995024,3cyclictest25-21ksoftirqd/102:09:421
25074995022,4cyclictest25-21ksoftirqd/101:19:451
25068995024,4cyclictest9-21ksoftirqd/022:34:330
25068995024,13cyclictest9-21ksoftirqd/021:59:430
25068995023,4cyclictest9-21ksoftirqd/023:19:440
25068995023,4cyclictest9-21ksoftirqd/021:54:180
25068995023,4cyclictest9-21ksoftirqd/001:54:480
25068995023,4cyclictest9-21ksoftirqd/001:47:050
25068995023,4cyclictest9-21ksoftirqd/001:47:050
25068995022,4cyclictest9-21ksoftirqd/022:59:430
25068995022,4cyclictest9-21ksoftirqd/022:14:440
25068995021,4cyclictest9-21ksoftirqd/001:34:060
25068995021,4cyclictest9-21ksoftirqd/001:09:350
25068995020,5cyclictest9-21ksoftirqd/001:39:380
25086994919,5cyclictest41-21ksoftirqd/302:36:053
25074994924,4cyclictest25-21ksoftirqd/122:34:481
25074994924,4cyclictest25-21ksoftirqd/102:34:411
25074994924,3cyclictest25-21ksoftirqd/101:14:551
25074994923,3cyclictest25-21ksoftirqd/123:04:401
25074994923,3cyclictest25-21ksoftirqd/121:14:181
25074994920,4cyclictest25-21ksoftirqd/123:19:451
25074994918,4cyclictest25-21ksoftirqd/123:19:211
25068994924,4cyclictest9-21ksoftirqd/021:09:350
25068994923,7cyclictest9-21ksoftirqd/001:51:290
25068994923,4cyclictest9-21ksoftirqd/001:29:230
25068994922,5cyclictest9-21ksoftirqd/022:09:360
25068994922,5cyclictest9-21ksoftirqd/022:09:360
25068994922,4cyclictest9-21ksoftirqd/023:29:350
25068994922,4cyclictest9-21ksoftirqd/021:19:470
25068994922,4cyclictest9-21ksoftirqd/000:19:360
25068994920,7cyclictest9-21ksoftirqd/023:38:530
25068994919,5cyclictest9-21ksoftirqd/021:24:320
25074994824,4cyclictest25-21ksoftirqd/100:55:121
25074994824,3cyclictest25-21ksoftirqd/121:44:381
25074994823,4cyclictest25-21ksoftirqd/121:54:431
25074994823,4cyclictest25-21ksoftirqd/102:27:381
25074994823,4cyclictest25-21ksoftirqd/101:06:541
25074994822,3cyclictest25-21ksoftirqd/122:14:341
25074994820,4cyclictest25-21ksoftirqd/101:59:551
25074994818,9cyclictest25-21ksoftirqd/101:49:241
25074994818,9cyclictest25-21ksoftirqd/101:49:241
25068994823,4cyclictest9-21ksoftirqd/021:59:190
25068994822,4cyclictest9-21ksoftirqd/021:34:480
25068994821,4cyclictest9-21ksoftirqd/022:54:360
151891480,1ptp4l401ktimersoftd/321:39:403
25074994723,3cyclictest25-21ksoftirqd/123:27:201
25074994723,3cyclictest25-21ksoftirqd/100:30:021
25074994722,3cyclictest25-21ksoftirqd/122:04:191
25074994721,4cyclictest25-21ksoftirqd/101:34:451
25074994721,4cyclictest25-21ksoftirqd/100:59:361
25074994721,4cyclictest25-21ksoftirqd/100:19:391
25074994710,7cyclictest25-21ksoftirqd/122:05:181
25068994722,4cyclictest9-21ksoftirqd/021:29:440
25068994722,4cyclictest9-21ksoftirqd/021:14:340
25068994722,4cyclictest9-21ksoftirqd/001:04:340
25068994721,4cyclictest9-21ksoftirqd/023:04:380
25068994721,4cyclictest9-21ksoftirqd/022:29:310
25068994721,4cyclictest9-21ksoftirqd/021:44:370
25068994720,4cyclictest9-21ksoftirqd/002:04:350
25068994717,4cyclictest9-21ksoftirqd/000:29:220
25086994614,9cyclictest41-21ksoftirqd/323:34:413
2507499468,9cyclictest25-21ksoftirqd/101:29:471
25074994622,3cyclictest25-21ksoftirqd/101:39:331
25074994621,4cyclictest25-21ksoftirqd/121:34:321
25074994621,3cyclictest25-21ksoftirqd/123:46:011
25074994621,3cyclictest25-21ksoftirqd/102:32:061
25074994620,4cyclictest25-21ksoftirqd/123:39:371
25074994620,4cyclictest25-21ksoftirqd/122:59:381
25074994620,4cyclictest25-21ksoftirqd/122:49:481
25074994620,4cyclictest25-21ksoftirqd/100:29:221
25074994620,3cyclictest25-21ksoftirqd/101:54:041
25074994619,4cyclictest25-21ksoftirqd/123:09:371
25074994611,3cyclictest121rcu_preempt01:25:551
25068994621,4cyclictest9-21ksoftirqd/023:24:500
25068994620,4cyclictest9-21ksoftirqd/023:59:320
25068994620,4cyclictest9-21ksoftirqd/023:59:320
25068994620,3cyclictest9-21ksoftirqd/022:44:430
241602460,2sleep20-21swapper/222:14:442
2508699454,10cyclictest41-21ksoftirqd/302:34:253
25074994521,3cyclictest25-21ksoftirqd/122:09:361
25074994521,3cyclictest25-21ksoftirqd/122:09:361
25074994520,3cyclictest25-21ksoftirqd/100:12:011
25074994518,4cyclictest25-21ksoftirqd/121:24:471
25068994519,4cyclictest9-21ksoftirqd/022:49:430
25068994519,4cyclictest9-21ksoftirqd/000:06:270
25086994412,3cyclictest41-21ksoftirqd/321:34:363
25074994420,3cyclictest25-21ksoftirqd/122:39:421
25086994336,3cyclictest41-21ksoftirqd/301:29:283
25074994320,3cyclictest25-21ksoftirqd/100:45:581
25074994319,4cyclictest25-21ksoftirqd/122:45:431
25074994319,4cyclictest25-21ksoftirqd/121:19:471
25074994317,4cyclictest25-21ksoftirqd/122:29:371
25086994226,4cyclictest41-21ksoftirqd/323:19:413
25086994117,4cyclictest41-21ksoftirqd/300:09:463
25086994114,7cyclictest41-21ksoftirqd/302:12:003
2508699411,11cyclictest41-21ksoftirqd/321:49:443
25074994118,3cyclictest25-21ksoftirqd/102:04:551
25086993932,3cyclictest41-21ksoftirqd/323:54:353
25086993926,10cyclictest41-21ksoftirqd/323:14:393
25086993923,5cyclictest41-21ksoftirqd/301:05:353
25068993925,3cyclictest9-21ksoftirqd/022:04:490
151891380,1ptp4l401ktimersoftd/302:26:573
151891380,1ptp4l401ktimersoftd/301:50:193
25074993723,4cyclictest25-21ksoftirqd/122:54:331
2507499371,7cyclictest25-21ksoftirqd/121:39:411
151891370,1ptp4l401ktimersoftd/321:54:393
151891370,1ptp4l401ktimersoftd/321:15:473
2508699367,11cyclictest41-21ksoftirqd/323:54:213
2507499362,3cyclictest25-21ksoftirqd/122:19:381
151891360,1ptp4l401ktimersoftd/301:48:063
151891360,1ptp4l401ktimersoftd/301:48:063
25086993530,3cyclictest41-21ksoftirqd/301:54:433
2508699350,3cyclictest121rcu_preempt22:29:363
25078993522,9cyclictest47550irq/101-ens3-Tx01:59:372
2507499351,2cyclictest121rcu_preempt00:09:211
151891350,1ptp4l401ktimersoftd/300:02:573
151891350,1ptp4l401ktimersoftd/300:02:573
25086993428,3cyclictest41-21ksoftirqd/323:09:213
2508699341,4cyclictest32548-21grep23:29:403
25086993413,10cyclictest41-21ksoftirqd/302:05:013
151891340,2ptp4l20812-21kworker/3:001:35:453
151891340,1ptp4l401ktimersoftd/321:45:403
316002330,2sleep30-21swapper/321:19:373
25086993328,3cyclictest41-21ksoftirqd/301:59:493
25086993327,3cyclictest41-21ksoftirqd/323:29:213
25086993327,3cyclictest41-21ksoftirqd/300:49:243
25086993327,3cyclictest41-21ksoftirqd/300:09:213
25086993316,3cyclictest41-21ksoftirqd/322:59:503
151891330,1ptp4l401ktimersoftd/322:39:433
151891330,11ptp4l401ktimersoftd/300:37:343
151891330,0ptp4l401ktimersoftd/301:17:283
25086993226,3cyclictest41-21ksoftirqd/301:39:343
151891320,1ptp4l401ktimersoftd/302:22:013
2508699316,4cyclictest41-21ksoftirqd/321:24:423
25078993115,7cyclictest30636-21sendmail21:19:182
151891310,1ptp4l401ktimersoftd/300:47:433
25086993025,3cyclictest41-21ksoftirqd/322:19:373
25086993024,3cyclictest41-21ksoftirqd/322:44:343
25086993024,3cyclictest41-21ksoftirqd/322:09:333
25086993024,3cyclictest41-21ksoftirqd/322:09:333
25086992923,3cyclictest41-21ksoftirqd/322:04:193
151891290,1ptp4l401ktimersoftd/300:27:093
25078992823,3cyclictest33-21ksoftirqd/222:39:192
25078992821,4cyclictest33-21ksoftirqd/223:43:452
25078992821,4cyclictest33-21ksoftirqd/222:20:302
25078992821,4cyclictest33-21ksoftirqd/221:12:012
2507899281,24cyclictest1152-21diskstats21:24:332
25078992720,4cyclictest33-21ksoftirqd/222:33:432
25078992720,4cyclictest33-21ksoftirqd/221:46:062
25078992720,4cyclictest33-21ksoftirqd/221:41:122
25078992720,4cyclictest33-21ksoftirqd/200:45:402
2507899264,13cyclictest23818-21rm01:14:362
25078992620,4cyclictest25830-21sort01:54:402
25078992620,3cyclictest33-21ksoftirqd/200:57:562
25078992620,3cyclictest33-21ksoftirqd/200:43:552
25078992619,4cyclictest33-21ksoftirqd/222:46:562
25078992619,4cyclictest33-21ksoftirqd/222:03:442
25078992619,4cyclictest33-21ksoftirqd/202:04:452
25078992619,4cyclictest33-21ksoftirqd/201:05:232
25078992618,4cyclictest13343-21cat02:19:252
25078992519,4cyclictest33-21ksoftirqd/223:11:532
25078992519,4cyclictest33-21ksoftirqd/221:35:502
25078992519,3cyclictest33-21ksoftirqd/201:10:422
25078992518,4cyclictest33-21ksoftirqd/222:59:242
2507899251,21cyclictest26789-21ssh23:20:482
2507899251,21cyclictest22231-21ssh01:49:442
2507899250,22cyclictest30344-21ssh00:05:412
25078992418,3cyclictest33-21ksoftirqd/223:48:452
25078992417,4cyclictest33-21ksoftirqd/221:57:582
25078992417,4cyclictest33-21ksoftirqd/221:53:042
25078992417,4cyclictest33-21ksoftirqd/221:21:572
25078992417,4cyclictest33-21ksoftirqd/202:36:252
2507899241,20cyclictest11977-21diskstats00:59:382
2513622316,3sleep233-21ksoftirqd/223:59:322
2513622316,3sleep233-21ksoftirqd/223:59:322
2507899234,17cyclictest47550irq/101-ens3-Tx00:36:392
2507899233,3cyclictest121rcu_preempt23:34:422
25078992318,3cyclictest33-21ksoftirqd/223:14:342
25078992317,4cyclictest33-21ksoftirqd/222:40:502
25078992316,4cyclictest33-21ksoftirqd/223:31:122
25078992316,4cyclictest33-21ksoftirqd/223:08:322
25078992316,4cyclictest33-21ksoftirqd/200:25:282
2507899231,20cyclictest14845-21diskmemload02:29:402
2507899231,20cyclictest14845-21diskmemload00:31:082
2507899231,20cyclictest13188-21apt-get01:39:352
2507899231,19cyclictest14845-21diskmemload01:21:532
2507899230,21cyclictest0-21swapper/202:29:012
2507899230,21cyclictest0-21swapper/200:09:292
2507899229,10cyclictest4466-21irqrtprio01:29:372
25078992217,3cyclictest33-21ksoftirqd/201:38:522
25078992216,3cyclictest33-21ksoftirqd/222:10:392
25078992216,3cyclictest33-21ksoftirqd/222:10:392
25078992216,3cyclictest33-21ksoftirqd/222:06:042
25078992216,3cyclictest33-21ksoftirqd/202:12:522
25078992215,4cyclictest33-21ksoftirqd/221:34:092
25078992215,4cyclictest27613-21chrt22:25:522
2507899221,18cyclictest17452-21snmp_o5powerctl23:49:422
2507899220,20cyclictest0-21swapper/200:19:492
25078992115,4cyclictest33-21ksoftirqd/201:48:432
25078992115,4cyclictest33-21ksoftirqd/201:48:432
25078992114,4cyclictest14845-21diskmemload23:27:222
25078992114,4cyclictest14160-21gltestperf02:19:422
2507899211,3cyclictest237362sleep223:57:182
25078992110,9cyclictest6801-21/usr/sbin/munin22:49:502
25078992013,4cyclictest2729-21sh01:28:322
2507899200,17cyclictest8077-21df_inode22:54:352
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional