You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Thu Apr 03, 2025 00:45:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137391230163,22phc2sys0-21swapper/319:06:183
189392207174,22sleep20-21swapper/219:08:032
189382204171,21sleep10-21swapper/119:08:031
188182202169,22sleep00-21swapper/019:06:280
2005921190,5sleep21938299cyclictest19:10:202
19383996231,7cyclictest41-21ksoftirqd/319:39:593
106322600,2sleep10-21swapper/120:00:011
137291580,1ptp4l401ktimersoftd/319:29:333
19383995622,8cyclictest41-21ksoftirqd/320:20:003
137291560,1ptp4l401ktimersoftd/300:09:323
214132550,4sleep321071-21/usr/sbin/munin22:55:143
279812530,2sleep20-21swapper/223:10:212
19383995325,7cyclictest41-21ksoftirqd/321:40:153
1938399518,5cyclictest41-21ksoftirqd/321:50:333
19383995122,8cyclictest41-21ksoftirqd/322:40:203
19383995021,4cyclictest41-21ksoftirqd/319:20:163
19383994917,6cyclictest41-21ksoftirqd/322:00:193
19383994916,8cyclictest41-21ksoftirqd/300:25:003
19383994916,8cyclictest41-21ksoftirqd/300:25:003
1938399487,4cyclictest41-21ksoftirqd/322:53:563
19383994830,9cyclictest41-21ksoftirqd/320:45:293
19383994830,9cyclictest41-21ksoftirqd/320:45:283
19383994822,7cyclictest41-21ksoftirqd/323:50:113
19383994820,4cyclictest41-21ksoftirqd/320:10:193
19383994818,7cyclictest41-21ksoftirqd/321:20:003
19383994818,7cyclictest41-21ksoftirqd/321:20:003
19383994818,4cyclictest41-21ksoftirqd/320:43:563
1938399475,4cyclictest41-21ksoftirqd/322:29:213
19383994722,9cyclictest41-21ksoftirqd/319:30:163
19383994722,3cyclictest41-21ksoftirqd/321:35:273
19383994720,7cyclictest41-21ksoftirqd/320:25:143
19383994720,7cyclictest41-21ksoftirqd/320:05:013
19383994720,6cyclictest41-21ksoftirqd/322:10:013
19383994719,7cyclictest41-21ksoftirqd/321:30:133
19383994713,9cyclictest41-21ksoftirqd/319:50:263
1938399466,4cyclictest41-21ksoftirqd/321:04:453
19383994621,7cyclictest41-21ksoftirqd/320:55:013
19383994620,6cyclictest41-21ksoftirqd/319:55:153
19383994619,7cyclictest41-21ksoftirqd/323:40:173
19383994618,4cyclictest41-21ksoftirqd/323:20:193
19383994618,4cyclictest41-21ksoftirqd/322:10:083
19383994616,9cyclictest41-21ksoftirqd/323:10:103
19383994614,4cyclictest41-21ksoftirqd/319:15:283
19383994611,4cyclictest41-21ksoftirqd/319:10:153
19381994640,4cyclictest8656-21cron23:45:001
1938399454,4cyclictest41-21ksoftirqd/321:25:143
19383994520,7cyclictest41-21ksoftirqd/320:05:163
19383994517,7cyclictest41-21ksoftirqd/321:25:003
19383994517,7cyclictest41-21ksoftirqd/300:15:153
19383994517,6cyclictest41-21ksoftirqd/300:15:013
19383994517,4cyclictest41-21ksoftirqd/323:55:173
19383994515,4cyclictest41-21ksoftirqd/320:37:023
1938199452,39cyclictest11561-21idleruntime-cro21:10:001
1938399446,4cyclictest41-21ksoftirqd/320:59:363
1938399443,2cyclictest121rcu_preempt22:20:003
19383994420,6cyclictest41-21ksoftirqd/319:45:283
19383994419,6cyclictest41-21ksoftirqd/321:55:163
19383994418,7cyclictest41-21ksoftirqd/322:20:233
19383994413,3cyclictest41-21ksoftirqd/320:20:263
19383994311,8cyclictest41-21ksoftirqd/322:30:093
1938399428,4cyclictest41-21ksoftirqd/323:37:243
1938399426,4cyclictest41-21ksoftirqd/323:48:483
1938399422,5cyclictest121rcu_preempt23:15:173
19383994216,6cyclictest41-21ksoftirqd/321:10:193
137291420,0ptp4l401ktimersoftd/320:30:263
1938399418,4cyclictest41-21ksoftirqd/300:04:403
1938399414,4cyclictest41-21ksoftirqd/323:34:483
19383994110,7cyclictest41-21ksoftirqd/300:35:293
1938099416,4cyclictest9-21ksoftirqd/022:50:000
19380994111,7cyclictest9-21ksoftirqd/000:14:380
1938399401,2cyclictest121rcu_preempt19:45:003
19382994012,26cyclictest12373-21kworker/2:220:05:292
1938199401,36cyclictest1989-21cron20:50:001
1938199401,36cyclictest1989-21cron20:50:001
19383993934,3cyclictest41-21ksoftirqd/323:30:003
19383993933,3cyclictest41-21ksoftirqd/322:50:003
19383993933,3cyclictest41-21ksoftirqd/322:39:593
1938399390,7cyclictest41-21ksoftirqd/323:00:253
19380993935,2cyclictest9-21ksoftirqd/022:39:590
19380993934,3cyclictest9-21ksoftirqd/022:00:000
19380993932,5cyclictest9-21ksoftirqd/022:12:060
19380993910,4cyclictest9-21ksoftirqd/021:30:280
274172380,1sleep20-21swapper/220:35:002
19383993832,3cyclictest41-21ksoftirqd/300:35:003
1938099381,6cyclictest9-21ksoftirqd/021:00:000
19383993732,3cyclictest41-21ksoftirqd/323:10:003
19383993732,2cyclictest41-21ksoftirqd/300:30:003
19383993731,3cyclictest41-21ksoftirqd/321:10:003
1938299370,35cyclictest31919-21kworker/2:223:50:192
19381993729,6cyclictest25-21ksoftirqd/119:15:001
1938099373,3cyclictest9-21ksoftirqd/021:10:170
19380993733,3cyclictest9-21ksoftirqd/019:55:000
1938099370,4cyclictest18659-21cron21:25:010
1938399360,2cyclictest121rcu_preempt21:50:003
19381993630,4cyclictest25-21ksoftirqd/123:02:371
19381993630,4cyclictest25-21ksoftirqd/121:19:111
19381993630,4cyclictest25-21ksoftirqd/121:19:111
19381993628,5cyclictest25-21ksoftirqd/122:07:461
19380993632,2cyclictest9-21ksoftirqd/023:30:000
19380993632,2cyclictest9-21ksoftirqd/020:35:000
19380993631,3cyclictest9-21ksoftirqd/023:55:140
19380993631,3cyclictest9-21ksoftirqd/023:35:010
19380993631,3cyclictest9-21ksoftirqd/023:15:140
19380993631,3cyclictest9-21ksoftirqd/020:30:010
19380993630,4cyclictest9-21ksoftirqd/023:04:210
19380993630,4cyclictest9-21ksoftirqd/021:37:000
19380993630,4cyclictest9-21ksoftirqd/019:16:520
19380993630,3cyclictest9-21ksoftirqd/023:50:010
19380993622,5cyclictest9-21ksoftirqd/022:24:580
19382993528,4cyclictest13103-21cron20:04:592
19381993529,4cyclictest25-21ksoftirqd/123:32:381
19381993529,4cyclictest25-21ksoftirqd/123:23:111
19381993529,4cyclictest25-21ksoftirqd/120:50:241
19381993529,4cyclictest25-21ksoftirqd/120:02:381
1938099356,3cyclictest9-21ksoftirqd/019:40:260
1938099353,3cyclictest9-21ksoftirqd/019:25:470
19380993532,2cyclictest9-21ksoftirqd/022:45:000
19380993531,3cyclictest9-21ksoftirqd/020:00:180
19380993531,2cyclictest9-21ksoftirqd/019:25:010
19380993530,3cyclictest9-21ksoftirqd/021:50:000
19380993529,4cyclictest9-21ksoftirqd/023:52:310
19380993529,4cyclictest9-21ksoftirqd/022:55:000
19380993529,4cyclictest9-21ksoftirqd/000:22:480
19380993529,4cyclictest9-21ksoftirqd/000:22:480
19380993529,3cyclictest9-21ksoftirqd/020:50:220
19382993428,4cyclictest33-21ksoftirqd/223:30:432
19382993428,4cyclictest33-21ksoftirqd/223:17:472
19382993428,4cyclictest33-21ksoftirqd/222:19:222
19382993428,4cyclictest33-21ksoftirqd/219:28:492
19382993428,4cyclictest33-21ksoftirqd/200:18:322
19382993414,17cyclictest27795-21latency_hist00:35:002
19381993428,4cyclictest25-21ksoftirqd/123:51:151
19381993428,4cyclictest25-21ksoftirqd/123:09:121
19381993428,4cyclictest25-21ksoftirqd/120:14:381
19381993428,4cyclictest25-21ksoftirqd/120:09:371
1938099346,2cyclictest121rcu_preempt23:10:000
1938099344,2cyclictest121rcu_preempt22:55:170
1938099343,3cyclictest9-21ksoftirqd/022:00:270
19380993430,2cyclictest9-21ksoftirqd/000:29:430
19380993430,2cyclictest9-21ksoftirqd/000:05:000
19380993429,3cyclictest9-21ksoftirqd/023:45:010
19380993428,4cyclictest9-21ksoftirqd/020:11:190
1938099342,6cyclictest9-21ksoftirqd/021:50:130
1938099342,2cyclictest9-21ksoftirqd/022:09:590
1938099342,2cyclictest121rcu_preempt21:00:260
19380993416,5cyclictest9-21ksoftirqd/020:20:140
19382993328,3cyclictest33-21ksoftirqd/200:04:542
19382993327,4cyclictest33-21ksoftirqd/222:13:162
19382993327,4cyclictest33-21ksoftirqd/221:18:092
19382993327,4cyclictest33-21ksoftirqd/221:18:082
19382993327,4cyclictest33-21ksoftirqd/220:51:362
19382993327,4cyclictest33-21ksoftirqd/220:16:272
19381993327,4cyclictest25-21ksoftirqd/122:55:071
19381993327,4cyclictest25-21ksoftirqd/122:48:411
19381993327,4cyclictest25-21ksoftirqd/121:56:511
19381993327,4cyclictest25-21ksoftirqd/119:49:571
19381993326,5cyclictest25-21ksoftirqd/119:33:121
19381993325,5cyclictest25-21ksoftirqd/100:05:161
1938199330,2cyclictest121rcu_preempt21:11:531
1938099334,3cyclictest9-21ksoftirqd/000:15:220
1938099334,3cyclictest121rcu_preempt23:10:170
19380993328,3cyclictest9-21ksoftirqd/023:35:140
19380993328,3cyclictest9-21ksoftirqd/023:20:130
19380993328,3cyclictest9-21ksoftirqd/021:40:220
19380993328,3cyclictest9-21ksoftirqd/020:00:000
19380993328,3cyclictest9-21ksoftirqd/019:34:410
19380993328,3cyclictest9-21ksoftirqd/019:15:010
19380993327,4cyclictest9-21ksoftirqd/019:35:410
1938099332,6cyclictest9-21ksoftirqd/022:30:160
19380993320,4cyclictest9-21ksoftirqd/020:07:310
19380993316,8cyclictest9-21ksoftirqd/021:05:280
1938099331,5cyclictest9-21ksoftirqd/020:45:180
1938099331,5cyclictest9-21ksoftirqd/020:45:170
1938299325,25cyclictest31919-21kworker/2:223:40:302
19382993226,4cyclictest33-21ksoftirqd/222:46:342
19382993226,4cyclictest33-21ksoftirqd/222:34:512
19382993226,4cyclictest33-21ksoftirqd/220:26:462
19382993226,4cyclictest33-21ksoftirqd/219:55:082
19382993226,4cyclictest33-21ksoftirqd/219:50:022
19382993219,11cyclictest33-21ksoftirqd/219:45:002
19381993226,4cyclictest25-21ksoftirqd/122:54:551
19381993226,4cyclictest25-21ksoftirqd/120:38:521
19381993226,4cyclictest25-21ksoftirqd/119:52:511
19381993226,4cyclictest25-21ksoftirqd/119:43:271
19381993226,4cyclictest25-21ksoftirqd/119:39:381
19381993226,4cyclictest25-21ksoftirqd/100:34:441
19381993226,4cyclictest25-21ksoftirqd/100:21:561
19381993226,4cyclictest25-21ksoftirqd/100:21:561
19381993225,5cyclictest25-21ksoftirqd/123:26:321
1938099325,6cyclictest9-21ksoftirqd/019:45:210
1938099325,3cyclictest121rcu_preempt20:40:210
19380993228,2cyclictest9-21ksoftirqd/000:35:010
19380993227,3cyclictest9-21ksoftirqd/022:18:270
19380993227,3cyclictest9-21ksoftirqd/021:15:140
19380993227,3cyclictest9-21ksoftirqd/021:15:130
19380993227,3cyclictest9-21ksoftirqd/000:40:000
19382993127,3cyclictest33-21ksoftirqd/220:58:382
19382993126,3cyclictest33-21ksoftirqd/223:30:002
19382993126,3cyclictest33-21ksoftirqd/221:59:462
19382993125,4cyclictest33-21ksoftirqd/221:43:072
19382993125,4cyclictest33-21ksoftirqd/220:14:502
1938299312,27cyclictest25494-21kworker/2:119:30:262
1938299310,3cyclictest121rcu_preempt22:25:542
19381993126,3cyclictest25-21ksoftirqd/123:12:321
19381993126,3cyclictest25-21ksoftirqd/120:44:521
19381993125,4cyclictest25-21ksoftirqd/121:53:331
19381993125,4cyclictest25-21ksoftirqd/121:45:591
19381993125,4cyclictest25-21ksoftirqd/121:44:551
19381993125,4cyclictest25-21ksoftirqd/120:30:531
19381993125,4cyclictest25-21ksoftirqd/120:16:101
19381993125,4cyclictest25-21ksoftirqd/100:11:221
19380993127,2cyclictest9-21ksoftirqd/022:30:020
19380993127,2cyclictest9-21ksoftirqd/021:30:010
19382993025,3cyclictest33-21ksoftirqd/219:21:312
19382993024,4cyclictest33-21ksoftirqd/222:38:272
19382993024,4cyclictest33-21ksoftirqd/222:20:382
19382993024,4cyclictest33-21ksoftirqd/221:22:102
19382993024,4cyclictest33-21ksoftirqd/221:07:132
19382993024,4cyclictest33-21ksoftirqd/219:38:322
19381993025,3cyclictest25-21ksoftirqd/123:49:581
19381993025,3cyclictest25-21ksoftirqd/119:26:061
19381993024,4cyclictest25-21ksoftirqd/121:21:001
19381993024,4cyclictest25-21ksoftirqd/121:01:261
19381993024,4cyclictest25-21ksoftirqd/120:25:401
19381993024,4cyclictest25-21ksoftirqd/120:21:591
19381993024,4cyclictest25-21ksoftirqd/119:21:211
19381993024,4cyclictest25-21ksoftirqd/119:17:451
19381993024,4cyclictest25-21ksoftirqd/100:01:551
19381993023,5cyclictest25-21ksoftirqd/121:28:381
19382992925,3cyclictest33-21ksoftirqd/220:23:382
19382992924,3cyclictest33-21ksoftirqd/223:25:002
19382992924,3cyclictest33-21ksoftirqd/222:41:302
19382992924,3cyclictest33-21ksoftirqd/200:36:052
19382992923,4cyclictest33-21ksoftirqd/222:58:092
19382992923,4cyclictest33-21ksoftirqd/221:54:422
19382992923,4cyclictest33-21ksoftirqd/221:14:072
19382992923,4cyclictest33-21ksoftirqd/221:01:232
19382992923,4cyclictest33-21ksoftirqd/220:45:032
19382992923,4cyclictest33-21ksoftirqd/220:45:022
19382992923,4cyclictest33-21ksoftirqd/219:46:262
1938299291,27cyclictest4628-21kworker/2:121:33:472
19381992923,4cyclictest25-21ksoftirqd/121:38:411
19381992923,4cyclictest25-21ksoftirqd/100:19:441
19381992922,5cyclictest25-21ksoftirqd/122:02:251
1938099293,8cyclictest121rcu_preempt20:20:000
1938299287,20cyclictest25494-21kworker/2:119:18:462
19382992823,3cyclictest33-21ksoftirqd/222:51:032
1938299282,24cyclictest4628-21kworker/2:123:07:142
1938299282,24cyclictest4628-21kworker/2:123:00:102
19382992822,4cyclictest33-21ksoftirqd/221:27:442
19381992822,4cyclictest25-21ksoftirqd/123:59:141
19381992822,4cyclictest25-21ksoftirqd/123:35:591
19381992821,5cyclictest25-21ksoftirqd/120:55:531
1938099281,2cyclictest121rcu_preempt20:35:150
1938299272,23cyclictest19624-21kworker/2:300:23:142
1938299272,23cyclictest19624-21kworker/2:300:23:142
19382992720,5cyclictest33-21ksoftirqd/222:01:412
1938299270,26cyclictest4628-21kworker/2:122:08:222
1938299270,25cyclictest12373-21kworker/2:220:35:022
19381992722,3cyclictest25-21ksoftirqd/123:17:491
19381992722,3cyclictest25-21ksoftirqd/122:45:001
19381992722,3cyclictest25-21ksoftirqd/122:35:001
19381992721,4cyclictest25-21ksoftirqd/121:33:311
19380992722,3cyclictest9-21ksoftirqd/000:05:470
19382992620,4cyclictest33-21ksoftirqd/223:37:492
19382992620,4cyclictest33-21ksoftirqd/200:05:572
19381992620,4cyclictest25-21ksoftirqd/122:36:021
1938299254,19cyclictest4628-21kworker/2:121:46:022
19382992520,3cyclictest33-21ksoftirqd/200:14:212
19381992519,4cyclictest25-21ksoftirqd/122:21:581
19381992519,4cyclictest25-21ksoftirqd/122:18:371
19381992519,4cyclictest25-21ksoftirqd/100:27:181
1938299244,18cyclictest12373-21kworker/2:220:43:502
19381992418,4cyclictest25-21ksoftirqd/122:13:151
19381992318,3cyclictest25-21ksoftirqd/100:37:531
19382992217,3cyclictest33-21ksoftirqd/223:57:462
19382992217,3cyclictest33-21ksoftirqd/223:46:322
1938299202,17cyclictest19624-21kworker/2:300:26:452
19382992015,3cyclictest33-21ksoftirqd/221:36:102
19381991714,2cyclictest0-21swapper/122:28:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional