You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 07:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Wed Apr 02, 2025 00:50:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3091729186,2sleep60-21swapper/619:05:156
3091729186,2sleep60-21swapper/619:05:156
30828450,29sleep30-21swapper/319:06:163
30828450,29sleep30-21swapper/319:06:163
57728248,30sleep50-21swapper/519:09:555
57728248,30sleep50-21swapper/519:09:555
40728146,30sleep40-21swapper/419:07:374
40728146,30sleep40-21swapper/419:07:374
3043828147,30sleep20-21swapper/219:05:032
3043828147,30sleep20-21swapper/219:05:032
34428046,29sleep70-21swapper/719:06:447
34428046,29sleep70-21swapper/719:06:447
3134228075,2sleep00-21swapper/019:05:170
3134228075,2sleep00-21swapper/019:05:170
39527944,30sleep10-21swapper/119:07:261
39527944,30sleep10-21swapper/119:07:261
265822480,0sleep20-21swapper/223:48:462
88492470,0sleep70-21swapper/723:05:597
138092470,0sleep40-21swapper/422:52:194
61342440,0sleep00-21swapper/000:27:420
11152440,0sleep30-21swapper/323:01:233
83612430,0sleep20-21swapper/223:21:312
1256993232,0cyclictest0-21swapper/022:54:380
1263993131,0cyclictest0-21swapper/723:50:527
126199290,29cyclictest0-21swapper/522:00:295
1258992929,0cyclictest0-21swapper/223:50:352
126199270,26cyclictest0-21swapper/521:30:195
1257992610,16cyclictest0-21swapper/122:19:401
1260992424,0cyclictest69450irq/124-eth022:59:434
1262992321,2cyclictest1559-21snmpd21:47:546
1257992321,1cyclictest241ktimersoftd/122:40:191
117022310,0sleep50-21swapper/523:51:385
1258992212,8cyclictest0-21swapper/223:42:032
1258992210,9cyclictest0-21swapper/223:08:302
1760622110,0sleep00-21swapper/000:33:280
1260992121,0cyclictest0-21swapper/422:47:164
1259992121,0cyclictest0-21swapper/323:30:363
1258992110,9cyclictest0-21swapper/223:29:082
1258992110,9cyclictest0-21swapper/222:24:262
1258992110,9cyclictest0-21swapper/221:51:342
1258992110,9cyclictest0-21swapper/221:38:082
1258992110,9cyclictest0-21swapper/200:06:302
1258992110,8cyclictest0-21swapper/221:01:232
1258992110,8cyclictest0-21swapper/200:17:262
1256992121,0cyclictest0-21swapper/000:35:110
1256992110,11cyclictest0-21swapper/022:23:390
1258992010,9cyclictest0-21swapper/223:57:242
1258992010,8cyclictest0-21swapper/223:39:162
1258992010,8cyclictest0-21swapper/223:17:352
1258992010,8cyclictest0-21swapper/223:13:492
1258992010,8cyclictest0-21swapper/222:40:262
1258992010,8cyclictest0-21swapper/222:36:482
1258992010,8cyclictest0-21swapper/222:25:022
1258992010,8cyclictest0-21swapper/222:14:442
1258992010,8cyclictest0-21swapper/222:06:052
1258992010,8cyclictest0-21swapper/221:42:382
1258992010,8cyclictest0-21swapper/221:18:042
1258992010,8cyclictest0-21swapper/220:57:312
1258992010,8cyclictest0-21swapper/220:07:502
1258992010,8cyclictest0-21swapper/220:01:572
1258992010,8cyclictest0-21swapper/200:31:222
1258992010,8cyclictest0-21swapper/200:25:132
1258992010,8cyclictest0-21swapper/200:24:032
1258992010,7cyclictest0-21swapper/222:50:102
1258992010,7cyclictest0-21swapper/221:49:462
1258992010,7cyclictest0-21swapper/221:25:252
1258992010,7cyclictest0-21swapper/200:37:492
1258992010,7cyclictest0-21swapper/200:10:162
1256992019,1cyclictest1642-21ntpd23:42:410
1263991919,0cyclictest0-21swapper/720:45:147
1259991919,0cyclictest0-21swapper/300:17:353
1259991915,4cyclictest649-21nscd21:46:373
1258991910,8cyclictest0-21swapper/222:55:042
1258991910,7cyclictest0-21swapper/223:32:112
1258991910,7cyclictest0-21swapper/222:01:462
1258991910,7cyclictest0-21swapper/221:57:172
1258991910,7cyclictest0-21swapper/221:11:322
1258991910,7cyclictest0-21swapper/220:50:272
1258991910,7cyclictest0-21swapper/220:45:502
1258991910,7cyclictest0-21swapper/220:20:192
1258991910,7cyclictest0-21swapper/219:45:352
1258991910,7cyclictest0-21swapper/200:00:132
1258991910,5cyclictest0-21swapper/223:00:102
1258991910,5cyclictest0-21swapper/222:34:342
1258991910,5cyclictest0-21swapper/222:15:272
1258991910,5cyclictest0-21swapper/221:31:322
125799195,1cyclictest121rcu_preempt00:36:391
1257991913,2cyclictest25849-21sh21:35:441
1262991810,0cyclictest0-21swapper/623:25:146
1260991815,3cyclictest19802-21nscd23:28:324
1258991810,7cyclictest0-21swapper/219:40:012
1258991810,6cyclictest0-21swapper/221:22:352
1258991810,6cyclictest0-21swapper/220:13:422
1258991810,6cyclictest0-21swapper/219:56:542
1258991810,5cyclictest0-21swapper/220:15:102
1258991810,5cyclictest0-21swapper/219:37:112
1258991810,5cyclictest0-21swapper/219:37:112
1258991810,5cyclictest0-21swapper/219:22:382
1258991810,5cyclictest0-21swapper/219:15:172
1258991810,5cyclictest0-21swapper/219:13:432
1258991810,4cyclictest0-21swapper/222:47:172
1258991810,4cyclictest0-21swapper/219:30:182
1257991814,3cyclictest231rcuc/121:32:101
1263991716,1cyclictest641-21nscd23:45:537
126399170,0cyclictest0-21swapper/700:08:507
1262991714,3cyclictest11478-21ssh22:50:486
1262991714,2cyclictest1560-21runrttasks22:57:356
1262991714,2cyclictest1559-21snmpd22:06:256
1262991710,4cyclictest0-21swapper/620:45:006
1260991717,0cyclictest69450irq/124-eth000:29:114
1260991710,3cyclictest0-21swapper/423:23:104
1260991710,1cyclictest21605-21ls21:50:154
1260991710,0cyclictest0-21swapper/423:31:344
1259991713,3cyclictest1254-21cyclictest23:46:193
1259991710,0cyclictest0-21swapper/322:28:063
1259991710,0cyclictest0-21swapper/319:40:133
1258991710,6cyclictest0-21swapper/220:25:422
1258991710,6cyclictest0-21swapper/219:25:152
1258991710,5cyclictest0-21swapper/220:30:122
1258991710,4cyclictest0-21swapper/221:09:152
1258991710,4cyclictest0-21swapper/220:40:142
1258991710,4cyclictest0-21swapper/219:50:192
1257991715,1cyclictest241ktimersoftd/122:50:071
1257991715,1cyclictest241ktimersoftd/100:00:551
1257991710,4cyclictest25-21ksoftirqd/122:49:411
1256991710,7cyclictest0-21swapper/023:14:000
1263991610,6cyclictest0-21swapper/700:00:227
1263991610,0cyclictest0-21swapper/723:26:067
1263991610,0cyclictest0-21swapper/700:28:217
1262991611,1cyclictest28498-21sh22:42:496
1262991610,6cyclictest0-21swapper/600:04:396
1262991610,3cyclictest0-21swapper/622:18:556
1262991610,0cyclictest0-21swapper/623:38:216
1262991610,0cyclictest0-21swapper/600:28:476
1261991616,0cyclictest25863-21perf19:50:005
1261991615,1cyclictest7881-21ssh23:05:215
1261991615,1cyclictest12173-21ssh23:57:025
1260991616,0cyclictest0-21swapper/423:16:184
1260991614,2cyclictest30105-21nscd00:24:464
1260991613,3cyclictest0-21swapper/400:07:214
1260991612,4cyclictest1559-21snmpd23:46:204
1260991610,6cyclictest0-21swapper/423:40:284
1260991610,0cyclictest0-21swapper/423:52:314
1260991610,0cyclictest0-21swapper/422:40:384
1260991610,0cyclictest0-21swapper/422:14:334
1260991610,0cyclictest0-21swapper/421:43:594
1259991615,1cyclictest649-21nscd23:38:283
1259991612,2cyclictest1254-21cyclictest00:08:533
1259991610,0cyclictest0-21swapper/323:07:153
1258991610,5cyclictest0-21swapper/220:35:032
125799169,4cyclictest1537-21munin-plugin-st21:05:001
125799169,0cyclictest121rcu_preempt23:55:121
125799168,2cyclictest131rcu_sched23:29:001
125799166,9cyclictest25-21ksoftirqd/120:45:121
125799166,8cyclictest121rcu_preempt00:24:341
125799166,6cyclictest121rcu_preempt22:08:491
1257991615,1cyclictest241ktimersoftd/122:35:391
1257991615,1cyclictest241ktimersoftd/121:47:111
1257991610,6cyclictest29323-21kworker/1:200:08:051
1257991610,0cyclictest0-21swapper/123:20:391
1257991610,0cyclictest0-21swapper/100:26:021
1256991610,6cyclictest0-21swapper/023:46:300
1256991610,6cyclictest0-21swapper/023:28:030
1256991610,3cyclictest0-21swapper/023:38:320
1256991610,2cyclictest0-21swapper/022:47:110
1256991610,0cyclictest0-21swapper/022:59:200
1256991610,0cyclictest0-21swapper/021:18:480
1256991610,0cyclictest0-21swapper/000:16:150
1263991515,0cyclictest4993-21kworker/u16:222:22:457
1263991515,0cyclictest0-21swapper/722:55:017
1263991514,1cyclictest4529-21diskmemload22:28:117
1263991514,0cyclictest24755-21rm23:14:357
1263991514,0cyclictest21064-21sh00:18:277
1263991513,1cyclictest4529-21diskmemload23:02:317
1263991511,3cyclictest641-21nscd23:24:397
1263991510,5cyclictest0-21swapper/700:20:067
1263991510,0cyclictest0-21swapper/722:45:317
1263991510,0cyclictest0-21swapper/721:57:307
1263991510,0cyclictest0-21swapper/721:34:517
1263991510,0cyclictest0-21swapper/721:29:177
1263991510,0cyclictest0-21swapper/700:36:567
1262991515,0cyclictest6379-21ls23:20:236
1262991514,1cyclictest641-21nscd21:43:276
1262991514,0cyclictest8113-1kworker/u17:222:33:216
1262991510,5cyclictest0-21swapper/623:17:536
1262991510,3cyclictest0-21swapper/619:25:186
1262991510,0cyclictest0-21swapper/623:58:466
1262991510,0cyclictest0-21swapper/623:53:246
1262991510,0cyclictest0-21swapper/623:03:356
1262991510,0cyclictest0-21swapper/622:48:506
1262991510,0cyclictest0-21swapper/622:00:156
1262991510,0cyclictest0-21swapper/600:36:126
1261991514,1cyclictest3344-21ssh23:35:395
1261991510,5cyclictest0-21swapper/523:46:415
1261991510,5cyclictest0-21swapper/522:53:385
1261991510,0cyclictest0-21swapper/523:27:285
1261991510,0cyclictest0-21swapper/523:20:235
1261991510,0cyclictest0-21swapper/523:10:165
1260991515,0cyclictest69450irq/124-eth023:36:074
1260991515,0cyclictest27562-21ls21:20:174
1260991515,0cyclictest0-21swapper/419:18:334
1260991515,0cyclictest0-21swapper/400:14:214
1260991512,1cyclictest69450irq/124-eth023:55:534
1260991511,2cyclictest0-21swapper/400:30:144
1260991510,5cyclictest0-21swapper/400:38:184
1260991510,5cyclictest0-21swapper/400:02:524
1260991510,0cyclictest0-21swapper/423:04:474
1260991510,0cyclictest0-21swapper/422:26:104
1260991510,0cyclictest0-21swapper/422:05:474
1260991510,0cyclictest0-21swapper/421:57:264
1260991510,0cyclictest0-21swapper/421:47:494
1260991510,0cyclictest0-21swapper/421:10:564
1260991510,0cyclictest0-21swapper/420:30:084
1259991512,2cyclictest734-21gmain21:16:423
1259991510,5cyclictest0-21swapper/323:55:213
1259991510,5cyclictest0-21swapper/322:34:323
1259991510,5cyclictest0-21swapper/321:51:183
1259991510,5cyclictest0-21swapper/321:29:163
1259991510,5cyclictest0-21swapper/300:22:233
1259991510,0cyclictest0-21swapper/323:50:083
1259991510,0cyclictest0-21swapper/323:26:403
1259991510,0cyclictest0-21swapper/323:16:123
1259991510,0cyclictest0-21swapper/322:55:523
1259991510,0cyclictest0-21swapper/322:53:153
1259991510,0cyclictest0-21swapper/322:45:183
1259991510,0cyclictest0-21swapper/321:57:403
1259991510,0cyclictest0-21swapper/321:12:023
1257991515,0cyclictest0-21swapper/100:33:001
1257991514,1cyclictest241ktimersoftd/121:11:551
1257991514,1cyclictest0-21swapper/120:00:171
1257991514,1cyclictest0-21swapper/100:17:011
1257991513,1cyclictest641-21nscd21:22:001
1257991513,1cyclictest241ktimersoftd/122:01:431
1257991513,1cyclictest241ktimersoftd/119:25:151
1257991511,1cyclictest25-21ksoftirqd/122:15:001
1257991510,5cyclictest0-21swapper/123:03:191
1257991510,4cyclictest0-21swapper/123:38:591
1257991510,3cyclictest121rcu_preempt22:56:131
1257991510,2cyclictest0-21swapper/122:31:181
1257991510,2cyclictest0-21swapper/121:58:561
1257991510,1cyclictest241ktimersoftd/119:20:171
1257991510,1cyclictest10991-21kworker/1:123:40:481
1257991510,1cyclictest0-21swapper/100:10:161
1257991510,0cyclictest0-21swapper/123:50:061
1257991510,0cyclictest0-21swapper/123:19:181
1256991514,1cyclictest1642-21ntpd23:22:320
1256991510,5cyclictest0-21swapper/023:52:570
1256991510,5cyclictest0-21swapper/021:25:540
1256991510,0cyclictest0-21swapper/023:57:480
1256991510,0cyclictest0-21swapper/023:34:120
1256991510,0cyclictest0-21swapper/023:04:570
1256991510,0cyclictest0-21swapper/022:35:120
1256991510,0cyclictest0-21swapper/000:22:050
1256991510,0cyclictest0-21swapper/000:07:400
34802140,2sleep773-21ksoftirqd/720:10:017
1263991414,0cyclictest570-21fschecks_count20:00:157
1263991414,0cyclictest0-21swapper/721:51:077
1263991410,4cyclictest734-21gmain22:19:457
1263991410,4cyclictest0-21swapper/723:37:467
1263991410,4cyclictest0-21swapper/722:35:217
1263991410,4cyclictest0-21swapper/721:45:427
1263991410,4cyclictest0-21swapper/721:21:447
1263991410,4cyclictest0-21swapper/721:11:067
1263991410,0cyclictest0-21swapper/723:56:527
1263991410,0cyclictest0-21swapper/723:42:527
1263991410,0cyclictest0-21swapper/723:15:297
1263991410,0cyclictest0-21swapper/722:10:097
1263991410,0cyclictest0-21swapper/700:30:267
1263991410,0cyclictest0-21swapper/700:14:257
1262991414,0cyclictest7847-21ls20:15:176
1262991414,0cyclictest7626-1kworker/u17:223:30:046
1262991414,0cyclictest4529-21diskmemload22:21:166
1262991412,0cyclictest30042-21ls20:55:206
1262991410,4cyclictest0-21swapper/622:37:366
1262991410,4cyclictest0-21swapper/619:20:206
1262991410,3cyclictest0-21swapper/600:22:246
1262991410,2cyclictest0-21swapper/621:50:206
1262991410,0cyclictest0-21swapper/623:46:056
1262991410,0cyclictest0-21swapper/623:40:196
1262991410,0cyclictest0-21swapper/623:14:126
1262991410,0cyclictest0-21swapper/623:09:476
1262991410,0cyclictest0-21swapper/622:25:496
1262991410,0cyclictest0-21swapper/622:10:036
1262991410,0cyclictest0-21swapper/621:56:296
1262991410,0cyclictest0-21swapper/619:47:236
1262991410,0cyclictest0-21swapper/600:30:126
1262991410,0cyclictest0-21swapper/600:12:576
1262991410,0cyclictest0-21swapper/600:05:146
1261991413,1cyclictest28053-21sh00:05:235
1261991412,1cyclictest4529-21diskmemload21:50:095
1261991412,1cyclictest4529-21diskmemload21:49:095
1261991412,1cyclictest25612-21sh00:04:015
1261991412,1cyclictest15196-21idleruntime19:30:185
1261991410,4cyclictest0-21swapper/500:15:395
1261991410,1cyclictest4529-21diskmemload23:31:565
1261991410,0cyclictest0-21swapper/523:19:585
1261991410,0cyclictest0-21swapper/522:58:135
1261991410,0cyclictest0-21swapper/522:40:535
1261991410,0cyclictest0-21swapper/522:22:565
1261991410,0cyclictest0-21swapper/522:13:145
1261991410,0cyclictest0-21swapper/522:05:235
1261991410,0cyclictest0-21swapper/500:33:285
1261991410,0cyclictest0-21swapper/500:21:385
1261991410,0cyclictest0-21swapper/500:14:385
1260991414,0cyclictest69450irq/124-eth022:36:044
1260991413,1cyclictest69450irq/124-eth023:10:084
1260991413,1cyclictest69450irq/124-eth021:35:564
1260991413,1cyclictest69450irq/124-eth021:29:234
1260991410,4cyclictest0-21swapper/400:15:104
1260991410,0cyclictest0-21swapper/423:09:454
1260991410,0cyclictest0-21swapper/422:33:114
1260991410,0cyclictest0-21swapper/422:20:214
1260991410,0cyclictest0-21swapper/422:18:344
1260991410,0cyclictest0-21swapper/422:02:504
1260991410,0cyclictest0-21swapper/421:33:314
1259991414,0cyclictest27942-21ls22:10:123
1259991414,0cyclictest0-21swapper/300:11:003
1259991413,1cyclictest12285-1kworker/u17:223:11:193
1259991410,4cyclictest0-21swapper/322:36:123
1259991410,4cyclictest0-21swapper/322:15:193
1259991410,4cyclictest0-21swapper/321:36:483
1259991410,3cyclictest0-21swapper/322:00:153
1259991410,1cyclictest0-21swapper/320:50:273
1259991410,1cyclictest0-21swapper/300:36:593
1259991410,1cyclictest0-21swapper/300:32:033
1259991410,0cyclictest0-21swapper/323:42:153
1259991410,0cyclictest0-21swapper/323:20:233
1259991410,0cyclictest0-21swapper/322:42:343
1259991410,0cyclictest0-21swapper/322:23:053
1259991410,0cyclictest0-21swapper/321:40:443
1259991410,0cyclictest0-21swapper/321:32:443
1259991410,0cyclictest0-21swapper/321:21:473
1259991410,0cyclictest0-21swapper/320:58:203
1259991410,0cyclictest0-21swapper/320:41:063
1259991410,0cyclictest0-21swapper/300:29:413
1259991410,0cyclictest0-21swapper/300:00:593
1257991413,1cyclictest6468-21ls19:15:161
1257991413,1cyclictest241ktimersoftd/123:49:311
1257991413,1cyclictest241ktimersoftd/121:40:471
1257991413,1cyclictest241ktimersoftd/121:26:511
1257991412,1cyclictest25-21ksoftirqd/121:00:011
1257991412,1cyclictest241ktimersoftd/121:05:151
1257991412,1cyclictest241ktimersoftd/120:16:231
1257991412,1cyclictest241ktimersoftd/119:50:191
1257991412,1cyclictest241ktimersoftd/119:35:391
1257991412,1cyclictest241ktimersoftd/119:35:391
1257991412,1cyclictest22154-21rm21:17:561
1257991410,4cyclictest0-21swapper/123:11:261
1257991410,4cyclictest0-21swapper/122:27:311
1257991410,4cyclictest0-21swapper/119:40:191
1257991410,0cyclictest0-21swapper/123:31:061
1257991410,0cyclictest0-21swapper/123:08:461
1257991410,0cyclictest0-21swapper/122:23:391
1257991410,0cyclictest0-21swapper/121:50:191
1257991410,0cyclictest0-21swapper/120:10:141
1256991414,0cyclictest0-21swapper/022:18:300
1256991414,0cyclictest0-21swapper/021:53:210
1256991411,1cyclictest0-21swapper/019:44:560
1256991410,0cyclictest0-21swapper/023:17:000
1256991410,0cyclictest0-21swapper/023:07:090
1256991410,0cyclictest0-21swapper/022:32:350
1256991410,0cyclictest0-21swapper/022:25:500
1256991410,0cyclictest0-21swapper/022:10:580
1256991410,0cyclictest0-21swapper/022:01:500
1256991410,0cyclictest0-21swapper/021:55:240
1256991410,0cyclictest0-21swapper/021:42:220
1256991410,0cyclictest0-21swapper/021:10:550
1256991410,0cyclictest0-21swapper/000:01:080
327221310,0sleep70-21swapper/721:05:187
1345421310,2sleep30-21swapper/319:26:343
1267921310,2sleep40-21swapper/420:28:004
1263991312,1cyclictest17592-21python21:15:207
1263991310,3cyclictest0-21swapper/722:05:217
1263991310,3cyclictest0-21swapper/721:35:127
1263991310,3cyclictest0-21swapper/720:50:197
1263991310,2cyclictest0-21swapper/720:00:007
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional