You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-31 - 14:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot3.osadl.org (updated Mon Mar 31, 2025 00:45:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137191237170,23phc2sys0-21swapper/319:06:513
256142208161,21sleep10-21swapper/119:09:001
255432205172,22sleep20-21swapper/219:08:032
255162202169,22sleep00-21swapper/019:07:410
291912660,2sleep10-21swapper/123:10:001
25990996228,4cyclictest41-21ksoftirqd/323:45:013
25990996128,4cyclictest41-21ksoftirqd/320:50:003
25990996128,4cyclictest41-21ksoftirqd/320:25:013
25990996028,10cyclictest41-21ksoftirqd/320:15:253
25990996027,4cyclictest41-21ksoftirqd/319:25:013
25990996026,9cyclictest41-21ksoftirqd/300:35:293
25990996026,7cyclictest41-21ksoftirqd/322:00:203
25990996026,5cyclictest41-21ksoftirqd/319:45:003
25990996021,8cyclictest41-21ksoftirqd/322:55:013
25990995928,7cyclictest41-21ksoftirqd/322:10:143
25990995925,5cyclictest41-21ksoftirqd/321:15:163
25990995925,5cyclictest41-21ksoftirqd/321:15:163
25990995924,8cyclictest41-21ksoftirqd/320:25:183
25990995831,7cyclictest41-21ksoftirqd/322:15:163
25990995824,8cyclictest41-21ksoftirqd/322:30:273
25990995729,7cyclictest41-21ksoftirqd/320:05:123
25990995725,8cyclictest41-21ksoftirqd/320:55:283
25990995725,8cyclictest41-21ksoftirqd/300:00:153
25990995725,7cyclictest41-21ksoftirqd/322:35:143
25990995724,7cyclictest41-21ksoftirqd/320:40:163
25990995724,10cyclictest41-21ksoftirqd/320:50:103
25990995724,10cyclictest41-21ksoftirqd/319:35:003
25990995722,10cyclictest41-21ksoftirqd/322:40:193
25989995727,20cyclictest33-21ksoftirqd/222:00:122
25990995626,7cyclictest41-21ksoftirqd/319:50:003
25990995624,9cyclictest41-21ksoftirqd/319:15:003
25990995623,8cyclictest41-21ksoftirqd/321:25:013
25990995623,8cyclictest41-21ksoftirqd/300:15:173
25990995621,7cyclictest41-21ksoftirqd/319:25:163
25990995528,7cyclictest41-21ksoftirqd/321:05:233
25990995528,3cyclictest41-21ksoftirqd/321:55:283
25990995527,8cyclictest41-21ksoftirqd/319:50:223
25990995526,8cyclictest41-21ksoftirqd/320:30:183
25990995525,8cyclictest41-21ksoftirqd/323:00:223
25990995523,7cyclictest41-21ksoftirqd/322:20:133
25990995523,10cyclictest41-21ksoftirqd/323:10:243
25990995522,8cyclictest41-21ksoftirqd/323:30:203
25990995519,4cyclictest41-21ksoftirqd/323:35:193
2598899550,49cyclictest307-21kworker/1:022:30:261
25990995427,8cyclictest41-21ksoftirqd/322:55:173
25990995424,8cyclictest41-21ksoftirqd/321:20:193
25990995420,8cyclictest41-21ksoftirqd/323:55:233
25990995325,8cyclictest41-21ksoftirqd/323:20:153
25990995323,10cyclictest41-21ksoftirqd/321:30:113
25990995322,8cyclictest41-21ksoftirqd/320:35:253
25990995321,8cyclictest41-21ksoftirqd/320:00:143
25990995321,3cyclictest41-21ksoftirqd/321:48:243
25990995316,4cyclictest41-21ksoftirqd/321:40:153
25990995224,8cyclictest41-21ksoftirqd/300:30:133
25990995222,4cyclictest41-21ksoftirqd/319:35:263
25990995120,7cyclictest41-21ksoftirqd/300:20:173
25990995120,4cyclictest41-21ksoftirqd/300:10:093
25990995119,7cyclictest41-21ksoftirqd/321:00:143
106552510,2sleep00-21swapper/021:00:170
2599099506,2cyclictest41-21ksoftirqd/319:55:243
2599099505,2cyclictest41-21ksoftirqd/300:30:003
25990995017,4cyclictest41-21ksoftirqd/319:15:163
25990994920,3cyclictest41-21ksoftirqd/321:52:233
25990994911,7cyclictest41-21ksoftirqd/322:30:003
2599099485,6cyclictest41-21ksoftirqd/320:10:233
2599099483,10cyclictest41-21ksoftirqd/321:10:223
25989994842,4cyclictest33-21ksoftirqd/220:30:002
25990994716,7cyclictest41-21ksoftirqd/323:15:223
2599099470,6cyclictest41-21ksoftirqd/300:07:233
2599099465,2cyclictest41-21ksoftirqd/321:35:263
2599099463,7cyclictest41-21ksoftirqd/322:50:013
2599099463,6cyclictest41-21ksoftirqd/323:10:003
2598899460,44cyclictest0-21swapper/122:20:261
2599099454,6cyclictest41-21ksoftirqd/323:30:003
2599099451,7cyclictest41-21ksoftirqd/323:54:593
25989994427,5cyclictest33-21ksoftirqd/219:25:282
2599099431,8cyclictest41-21ksoftirqd/323:50:013
2599099420,7cyclictest41-21ksoftirqd/322:09:593
2598999387,8cyclictest20132-21kworker/2:100:10:072
2598899381,2cyclictest121rcu_preempt19:25:281
2598999376,8cyclictest28337-21kworker/2:200:33:592
25989993731,4cyclictest33-21ksoftirqd/223:44:182
25988993711,3cyclictest10524-21df21:00:161
25989993631,3cyclictest33-21ksoftirqd/223:05:522
25989993630,4cyclictest33-21ksoftirqd/221:57:312
25989993630,4cyclictest33-21ksoftirqd/221:41:352
2598999354,9cyclictest32265-21kworker/2:023:59:082
2598999354,8cyclictest32265-21kworker/2:023:45:232
2598999354,10cyclictest20541-21kworker/2:221:47:592
2598999353,20cyclictest28337-21kworker/2:200:37:232
25989993529,4cyclictest33-21ksoftirqd/223:30:352
25989993529,4cyclictest33-21ksoftirqd/220:17:392
25988993530,3cyclictest11787-21cron23:45:011
25988993529,4cyclictest25-21ksoftirqd/123:19:391
25988993529,4cyclictest25-21ksoftirqd/122:15:121
25987993529,4cyclictest9-21ksoftirqd/022:57:480
25987993529,4cyclictest9-21ksoftirqd/022:41:070
25987993513,19cyclictest26402-21sed20:20:200
2598999346,8cyclictest29101-21kworker/2:119:44:112
2598999346,8cyclictest29101-21kworker/2:119:37:192
2598999343,8cyclictest32265-21kworker/2:000:06:112
25989993429,3cyclictest33-21ksoftirqd/222:10:442
25989993428,4cyclictest33-21ksoftirqd/223:23:212
25989993428,4cyclictest33-21ksoftirqd/221:09:582
25989993428,4cyclictest33-21ksoftirqd/220:24:492
25989993428,4cyclictest33-21ksoftirqd/220:06:242
2598999341,21cyclictest32265-21kworker/2:023:38:102
2598899345,28cyclictest0-21swapper/120:15:261
25988993429,3cyclictest25-21ksoftirqd/120:24:221
25988993428,4cyclictest25-21ksoftirqd/123:57:371
25988993428,4cyclictest25-21ksoftirqd/123:29:251
25988993428,4cyclictest25-21ksoftirqd/122:28:361
25988993428,4cyclictest25-21ksoftirqd/120:05:001
25988993418,14cyclictest307-21kworker/1:022:55:571
25987993428,4cyclictest9-21ksoftirqd/021:32:270
25987993428,4cyclictest9-21ksoftirqd/021:14:130
25987993428,4cyclictest9-21ksoftirqd/019:58:270
25987993428,4cyclictest9-21ksoftirqd/000:32:300
25987993428,4cyclictest9-21ksoftirqd/000:24:170
25987993428,4cyclictest9-21ksoftirqd/000:01:260
25987993415,17cyclictest891-21/usr/sbin/munin19:25:220
2598999335,8cyclictest20541-21kworker/2:221:16:592
2598999335,8cyclictest20541-21kworker/2:221:16:592
2598999334,9cyclictest29101-21kworker/2:119:30:112
2598999333,3cyclictest20132-21kworker/2:100:27:042
2598999332,9cyclictest32265-21kworker/2:000:05:002
2598999332,8cyclictest32265-21kworker/2:023:52:282
25989993327,4cyclictest33-21ksoftirqd/222:39:322
25989993327,4cyclictest33-21ksoftirqd/222:27:252
25989993327,4cyclictest33-21ksoftirqd/222:15:092
25989993327,4cyclictest33-21ksoftirqd/221:31:522
25989993327,4cyclictest33-21ksoftirqd/219:13:442
2598999331,9cyclictest20132-21kworker/2:100:23:242
25989993311,20cyclictest1507-21wc20:35:302
25989993310,20cyclictest20146-21date22:40:252
25988993327,4cyclictest25-21ksoftirqd/121:21:321
25988993327,4cyclictest25-21ksoftirqd/119:41:271
25988993327,4cyclictest25-21ksoftirqd/100:08:451
25987993327,4cyclictest9-21ksoftirqd/023:31:540
25987993327,4cyclictest9-21ksoftirqd/023:09:230
25987993327,4cyclictest9-21ksoftirqd/022:54:500
25987993327,4cyclictest9-21ksoftirqd/022:46:030
25987993327,4cyclictest9-21ksoftirqd/021:16:460
25987993327,4cyclictest9-21ksoftirqd/021:16:460
25987993327,4cyclictest9-21ksoftirqd/020:15:040
25987993327,4cyclictest9-21ksoftirqd/000:25:530
25987993313,18cyclictest2419-21/usr/sbin/munin20:40:270
25987993313,17cyclictest9005-21tune2fs22:15:170
25987993311,3cyclictest7824-21latency_hist19:40:000
2598999325,8cyclictest29101-21kworker/2:119:19:552
2598999324,8cyclictest20541-21kworker/2:221:23:552
2598999324,26cyclictest20541-21kworker/2:221:54:352
25989993227,3cyclictest33-21ksoftirqd/223:00:362
25989993227,3cyclictest33-21ksoftirqd/221:36:022
25989993226,4cyclictest33-21ksoftirqd/222:49:342
25989993226,4cyclictest33-21ksoftirqd/219:22:482
25989993226,4cyclictest33-21ksoftirqd/200:19:092
25989993210,20cyclictest9140-21grep20:55:262
25988993226,4cyclictest25-21ksoftirqd/122:52:351
25988993226,4cyclictest25-21ksoftirqd/122:08:221
25988993226,4cyclictest25-21ksoftirqd/120:59:421
2598799329,3cyclictest5543-21cpuspeed_turbos20:50:100
25987993227,3cyclictest9-21ksoftirqd/020:10:180
25987993226,4cyclictest9-21ksoftirqd/023:23:190
25987993226,4cyclictest9-21ksoftirqd/023:14:370
25987993226,4cyclictest9-21ksoftirqd/022:36:090
25987993226,4cyclictest9-21ksoftirqd/020:59:010
25987993218,6cyclictest9-21ksoftirqd/023:48:490
25987993213,17cyclictest12490-21/usr/sbin/munin21:05:260
25987993212,3cyclictest29375-21sed19:15:170
2598999313,8cyclictest29101-21kworker/2:119:45:322
2598999312,9cyclictest20541-21kworker/2:221:27:272
25989993126,3cyclictest33-21ksoftirqd/222:06:332
25989993126,3cyclictest33-21ksoftirqd/220:45:002
25989993125,4cyclictest33-21ksoftirqd/220:14:182
25989993111,17cyclictest7085-21sed20:50:232
25988993126,3cyclictest25-21ksoftirqd/123:01:011
25988993126,3cyclictest25-21ksoftirqd/122:02:221
25988993126,3cyclictest25-21ksoftirqd/119:54:501
25988993125,4cyclictest25-21ksoftirqd/120:32:501
25988993125,4cyclictest25-21ksoftirqd/119:47:331
25988993125,4cyclictest25-21ksoftirqd/119:23:501
25988993124,4cyclictest25-21ksoftirqd/100:20:361
2598799319,3cyclictest28098-21df_abs21:40:150
25987993125,4cyclictest9-21ksoftirqd/021:52:530
25987993125,4cyclictest9-21ksoftirqd/020:09:180
25987993125,4cyclictest9-21ksoftirqd/000:12:120
25989993025,3cyclictest33-21ksoftirqd/223:25:262
25989993025,3cyclictest33-21ksoftirqd/220:48:582
2598999301,27cyclictest14680-21kworker/2:223:12:422
25989993010,18cyclictest12083-21wc19:50:172
2598999300,9cyclictest32265-21kworker/2:023:19:112
25988993025,3cyclictest25-21ksoftirqd/121:49:211
25988993024,4cyclictest25-21ksoftirqd/119:37:351
2598799308,3cyclictest14768-21sed22:30:190
25987993025,3cyclictest9-21ksoftirqd/000:15:330
25987993024,4cyclictest9-21ksoftirqd/023:57:090
25987993024,4cyclictest9-21ksoftirqd/022:21:410
25987993024,4cyclictest9-21ksoftirqd/019:31:470
2598799302,10cyclictest27667-21grep19:10:280
25987993010,3cyclictest620-21find19:25:020
2598999299,17cyclictest12203-21grep22:20:222
2598999297,19cyclictest105182chrt21:00:152
25989992924,3cyclictest33-21ksoftirqd/221:10:292
25989992923,4cyclictest33-21ksoftirqd/222:32:422
25989992923,4cyclictest33-21ksoftirqd/220:01:422
25989992923,4cyclictest33-21ksoftirqd/219:55:132
25988992924,3cyclictest25-21ksoftirqd/123:12:061
25988992923,4cyclictest25-21ksoftirqd/122:47:061
25988992923,4cyclictest25-21ksoftirqd/100:35:411
25988992922,5cyclictest25-21ksoftirqd/121:56:391
25988992922,5cyclictest25-21ksoftirqd/120:44:501
2598799295,5cyclictest5160-21latency_hist20:50:000
2598799294,5cyclictest29797-21latency_hist20:30:010
25987992924,3cyclictest9-21ksoftirqd/021:49:520
25987992924,3cyclictest9-21ksoftirqd/020:03:030
25987992923,4cyclictest9-21ksoftirqd/023:40:350
25987992923,4cyclictest9-21ksoftirqd/023:03:090
25987992923,4cyclictest9-21ksoftirqd/000:38:360
25987992923,4cyclictest9-21ksoftirqd/000:08:240
25987992911,3cyclictest1297-21runrttasks22:02:330
25988992823,3cyclictest25-21ksoftirqd/122:40:111
25988992823,3cyclictest25-21ksoftirqd/121:54:451
25988992823,3cyclictest25-21ksoftirqd/121:40:381
25988992823,3cyclictest25-21ksoftirqd/120:46:551
25988992823,3cyclictest25-21ksoftirqd/120:11:231
25988992822,4cyclictest25-21ksoftirqd/123:51:441
2598899281,22cyclictest307-21kworker/1:020:50:031
2598899281,16cyclictest3464-21kworker/1:219:56:471
2598799288,3cyclictest9310-21proc_pri19:40:230
25987992822,4cyclictest9-21ksoftirqd/021:25:490
2598899273,23cyclictest3464-21kworker/1:220:05:311
25988992722,3cyclictest25-21ksoftirqd/120:35:471
25988992721,4cyclictest25-21ksoftirqd/120:25:441
25988992721,4cyclictest25-21ksoftirqd/119:32:291
2598799277,3cyclictest436-21phc2sys23:15:220
25987992722,3cyclictest9-21ksoftirqd/020:36:300
25987992721,3cyclictest9-21ksoftirqd/020:30:020
2598999268,15cyclictest24027-21cpu22:55:132
25989992621,3cyclictest33-21ksoftirqd/220:31:382
2598999260,23cyclictest23878-21cat22:55:012
25988992621,3cyclictest25-21ksoftirqd/119:16:081
25988992620,4cyclictest25-21ksoftirqd/121:06:521
2598899261,2cyclictest121rcu_preempt23:48:501
25987992621,3cyclictest9-21ksoftirqd/023:51:200
25987992621,3cyclictest9-21ksoftirqd/019:50:040
25987992620,4cyclictest9-21ksoftirqd/023:37:190
25988992520,3cyclictest25-21ksoftirqd/121:30:281
25988992520,3cyclictest25-21ksoftirqd/121:26:261
25988992519,4cyclictest25-21ksoftirqd/122:11:591
25988992519,4cyclictest25-21ksoftirqd/100:27:021
25987992520,3cyclictest9-21ksoftirqd/023:26:430
25987992520,3cyclictest9-21ksoftirqd/022:07:570
25987992519,4cyclictest9-21ksoftirqd/022:29:070
25988992419,3cyclictest25-21ksoftirqd/121:17:591
25988992419,3cyclictest25-21ksoftirqd/121:17:591
25988992419,3cyclictest25-21ksoftirqd/121:10:051
25988992418,4cyclictest25-21ksoftirqd/100:32:161
25987992418,4cyclictest9-21ksoftirqd/019:45:430
25988992318,3cyclictest25-21ksoftirqd/123:32:381
25988992317,4cyclictest25-21ksoftirqd/121:35:171
25987992318,3cyclictest9-21ksoftirqd/022:14:330
25987992318,3cyclictest9-21ksoftirqd/021:57:350
25988992217,3cyclictest25-21ksoftirqd/123:21:141
25988992217,3cyclictest25-21ksoftirqd/100:17:441
25988992216,4cyclictest25-21ksoftirqd/123:37:561
25987992217,3cyclictest9-21ksoftirqd/021:39:020
25988992115,4cyclictest25-21ksoftirqd/100:13:591
25988992014,4cyclictest17622-21sendmail_mailqu22:35:231
25987992015,3cyclictest20285-21if_err_enp4s021:20:200
25988991914,3cyclictest26209-21grep19:10:151
25988991813,3cyclictest18850-21sensors_temp00:00:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional