You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sat Jul 27, 2024 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20157521070,98pltrace0-21swapper/207:07:092
27144521040,3pltrace0-21swapper/210:09:452
22954521010,93pltrace0-21swapper/307:09:473
2015752930,86pltrace0-21swapper/607:05:226
2266127461,9sleep50-21swapper/507:05:595
2284927143,11sleep00-21swapper/007:08:340
2265027142,10sleep40-21swapper/407:05:504
2295452670,59pltrace0-21swapper/707:09:447
2279426147,10sleep10-21swapper/107:07:471
867352300,0pltrace0-21swapper/710:35:137
417352280,0pltrace0-21swapper/711:25:237
310652260,4pltrace0-21swapper/710:25:127
2714452250,0pltrace0-21swapper/710:10:257
1981852230,0pltrace0-21swapper/710:55:147
1596752210,7pltrace0-21swapper/709:51:257
137152200,0pltrace0-21swapper/711:20:267
2886352190,4pltrace0-21swapper/709:14:577
203952180,0pltrace0-21swapper/709:25:157
524052170,8pltrace0-21swapper/712:26:327
3224952170,0pltrace0-21swapper/707:25:207
2945252170,0pltrace0-21swapper/707:20:317
2649052170,0pltrace0-21swapper/712:05:447
2320799170,16cyclictest20249-21grep10:55:136
23204991712,2cyclictest0-21swapper/508:08:455
1981852170,11pltrace54-21ksoftirqd/410:55:164
823752160,0pltrace0-21swapper/707:43:177
652452160,10pltrace78-21ksoftirqd/708:35:167
245452160,13pltrace54-21ksoftirqd/412:21:224
245452160,13pltrace54-21ksoftirqd/412:21:224
2321099161,8cyclictest0-21swapper/712:31:157
2321099161,8cyclictest0-21swapper/711:17:197
2321099160,15cyclictest0-21swapper/710:52:247
23207991611,3cyclictest0-21swapper/611:46:426
23207991611,2cyclictest0-21swapper/607:31:216
2320799160,15cyclictest0-21swapper/612:15:136
23204991610,5cyclictest0-21swapper/508:04:135
2320499161,7cyclictest0-21swapper/510:13:085
2320499161,14cyclictest0-21swapper/509:25:465
2320499160,15cyclictest0-21swapper/509:06:195
2320499160,15cyclictest0-21swapper/508:46:245
2320499160,14cyclictest441-21processes11:15:195
23195991612,2cyclictest0-21swapper/311:53:193
2319599161,8cyclictest0-21swapper/311:59:503
2319599160,15cyclictest0-21swapper/312:30:123
23190991611,3cyclictest0-21swapper/211:48:172
2319099160,4cyclictest0-21swapper/209:06:402
2319099160,4cyclictest0-21swapper/207:23:012
2319099160,15cyclictest0-21swapper/209:27:092
23187991614,2cyclictest0-21swapper/112:30:001
23187991611,3cyclictest0-21swapper/109:10:451
2318799160,15cyclictest0-21swapper/107:10:421
2318299168,7cyclictest0-21swapper/011:30:200
23182991611,3cyclictest0-21swapper/012:34:060
2318299161,7cyclictest0-21swapper/007:16:110
2318299160,2cyclictest0-21swapper/012:00:180
2295452160,2pltrace0-21swapper/707:10:217
974652150,9pltrace54-21ksoftirqd/411:35:214
652452150,10pltrace0-21swapper/508:35:355
417352150,2pltrace0-21swapper/311:26:193
2321099150,14cyclictest0-21swapper/710:34:547
2321099150,14cyclictest0-21swapper/708:52:477
2321099150,14cyclictest0-21swapper/708:40:367
2321099150,14cyclictest0-21swapper/707:35:127
23207991510,3cyclictest0-21swapper/610:52:096
2320799150,7cyclictest0-21swapper/611:27:346
2320799150,7cyclictest0-21swapper/607:13:536
2320799150,4cyclictest0-21swapper/609:28:406
2320799150,14cyclictest0-21swapper/610:00:566
2320799150,14cyclictest0-21swapper/609:33:136
2320799150,14cyclictest0-21swapper/609:33:136
2320799150,13cyclictest0-21swapper/609:17:246
23204991510,3cyclictest0-21swapper/508:33:235
2320499151,2cyclictest0-21swapper/508:56:275
2319999159,5cyclictest0-21swapper/411:26:584
23199991512,2cyclictest0-21swapper/407:36:574
23199991510,3cyclictest0-21swapper/411:01:344
23199991510,3cyclictest0-21swapper/408:58:314
2319999150,2cyclictest0-21swapper/412:00:254
2319999150,14cyclictest0-21swapper/411:17:194
23195991511,2cyclictest0-21swapper/312:09:503
23195991510,4cyclictest0-21swapper/310:26:093
2319599150,8cyclictest0-21swapper/311:00:063
2319599150,7cyclictest0-21swapper/311:08:073
2319599150,7cyclictest0-21swapper/308:24:103
2319599150,4cyclictest0-21swapper/308:25:403
2319599150,3cyclictest0-21swapper/310:10:093
23190991511,3cyclictest0-21swapper/211:40:262
2319099150,8cyclictest0-21swapper/212:25:572
2319099150,8cyclictest0-21swapper/211:31:262
2319099150,8cyclictest0-21swapper/208:35:532
2319099150,7cyclictest0-21swapper/208:27:312
2319099150,3cyclictest0-21swapper/211:51:192
2319099150,2cyclictest0-21swapper/212:00:282
2319099150,14cyclictest0-21swapper/209:57:052
2319099150,14cyclictest0-21swapper/207:39:312
2318799150,7cyclictest0-21swapper/110:07:171
2318799150,7cyclictest0-21swapper/107:56:041
23182991510,4cyclictest0-21swapper/009:45:190
2318299150,7cyclictest0-21swapper/012:07:120
2318299150,7cyclictest0-21swapper/010:20:230
2318299150,7cyclictest0-21swapper/007:59:200
2318299150,14cyclictest0-21swapper/011:08:050
2318299150,14cyclictest0-21swapper/009:14:030
2318299150,14cyclictest0-21swapper/008:27:490
203952150,8pltrace0-21swapper/309:26:293
1981852150,6pltrace0-21swapper/010:55:270
1981852150,10pltrace0-21swapper/510:55:255
1487752150,3pltrace54-21ksoftirqd/408:50:054
2327052140,4pltrace54-21ksoftirqd/409:08:044
2321099140,7cyclictest0-21swapper/712:37:467
2321099140,7cyclictest0-21swapper/709:18:207
2320799149,3cyclictest0-21swapper/611:59:126
23207991410,3cyclictest0-21swapper/609:50:166
2320799140,13cyclictest0-21swapper/612:33:016
2320499149,2cyclictest0-21swapper/509:35:175
2319999140,13cyclictest0-21swapper/411:20:004
2319599149,3cyclictest0-21swapper/309:36:433
23190991412,1cyclictest0-21swapper/207:40:222
23190991410,2cyclictest0-21swapper/212:18:302
2319099140,13cyclictest0-21swapper/211:35:222
2318799149,3cyclictest0-21swapper/107:41:131
2318799140,8cyclictest0-21swapper/110:49:451
2318799140,7cyclictest0-21swapper/109:54:331
2318299148,5cyclictest0-21swapper/009:21:480
2318299143,9cyclictest0-21swapper/008:42:590
23182991413,1cyclictest0-21swapper/010:44:460
2318299140,7cyclictest0-21swapper/012:22:510
2318299140,7cyclictest0-21swapper/012:22:510
2318299140,7cyclictest0-21swapper/008:07:500
2318299140,3cyclictest0-21swapper/011:01:470
2318299140,3cyclictest0-21swapper/007:33:450
1766452140,2pltrace0-21swapper/708:59:137
1487752140,6pltrace17446-21runrttasks08:51:545
1381252140,6pltrace0-21swapper/507:49:535
1146252140,10pltrace54-21ksoftirqd/410:41:054
89452130,0pltrace0-21swapper/708:29:227
3206452130,4pltrace0-21swapper/512:18:215
3056052130,5pltrace0-21swapper/708:22:037
2945252130,8pltrace54-21ksoftirqd/407:20:244
2368252130,3pltrace26180-21chrt12:00:526
2321099130,12cyclictest0-21swapper/707:53:457
2321099130,1cyclictest0-21swapper/708:04:437
2320799130,13cyclictest0-21swapper/607:40:266
2320799130,12cyclictest0-21swapper/611:33:446
2320499130,6cyclictest0-21swapper/510:35:415
2320499130,13cyclictest0-21swapper/507:40:175
2319999138,3cyclictest0-21swapper/409:58:224
2319999138,3cyclictest0-21swapper/408:45:294
2319999137,5cyclictest0-21swapper/411:13:554
2319999130,13cyclictest0-21swapper/411:41:324
2319999130,12cyclictest0-21swapper/410:20:234
2319999130,12cyclictest0-21swapper/409:52:264
2319999130,12cyclictest0-21swapper/408:40:124
2319999130,12cyclictest0-21swapper/408:04:454
2319599138,3cyclictest0-21swapper/312:10:093
2319599138,3cyclictest0-21swapper/312:02:123
2319599130,12cyclictest0-21swapper/309:58:503
2319599130,1cyclictest0-21swapper/311:10:233
2319099138,4cyclictest25853-21chrt07:15:002
2319099133,8cyclictest0-21swapper/212:13:542
2319099133,8cyclictest0-21swapper/209:23:092
2319099132,4cyclictest23663-21logger11:59:442
2319099130,6cyclictest0-21swapper/210:47:042
2319099130,13cyclictest0-21swapper/208:02:332
2319099130,12cyclictest0-21swapper/209:14:242
2319099130,12cyclictest0-21swapper/207:54:542
2318799139,3cyclictest2192-21flock10:20:201
2318799138,3cyclictest0-21swapper/109:44:131
23187991310,2cyclictest0-21swapper/111:25:111
2318299136,6cyclictest0-21swapper/007:38:390
23182991312,1cyclictest0-21swapper/007:40:220
2318299130,12cyclictest0-21swapper/008:18:440
2318299130,1cyclictest0-21swapper/008:53:320
1703252130,8pltrace54-21ksoftirqd/410:50:144
1531552130,6pltrace0-21swapper/011:48:170
1487752130,5pltrace0-21swapper/108:51:521
524052120,4pltrace0-21swapper/512:26:035
2927252120,5pltrace0-21swapper/512:13:245
2714452120,5pltrace0-21swapper/610:13:406
2542252120,3pltrace28161-21runrttasks11:09:206
2321099128,2cyclictest0-21swapper/711:02:467
2321099127,3cyclictest0-21swapper/710:19:147
2321099126,5cyclictest0-21swapper/709:46:437
23210991210,1cyclictest0-21swapper/711:40:047
2321099120,11cyclictest0-21swapper/710:01:597
2321099120,11cyclictest0-21swapper/708:45:027
2321099120,1cyclictest0-21swapper/707:15:267
2320799129,2cyclictest1090-21dbus-daemon12:30:016
2320799129,2cyclictest0-21swapper/612:11:046
23207991211,1cyclictest0-21swapper/612:35:266
2320499126,5cyclictest0-21swapper/510:20:005
2320499120,11cyclictest0-21swapper/511:06:475
2319999127,4cyclictest27839-21tail09:10:204
2319999123,8cyclictest0-21swapper/408:07:054
23199991211,1cyclictest0-21swapper/411:45:234
2319999120,12cyclictest0-21swapper/408:11:334
2319999120,11cyclictest0-21swapper/411:30:284
2319999120,11cyclictest0-21swapper/410:33:114
2319999120,11cyclictest0-21swapper/410:18:114
2319999120,11cyclictest0-21swapper/409:20:214
2319999120,11cyclictest0-21swapper/408:30:294
2319599128,3cyclictest3561-21timerandwakeup11:20:243
2319599120,5cyclictest0-21swapper/308:09:553
2319599120,11cyclictest0-21swapper/308:37:263
2319599120,11cyclictest0-21swapper/308:01:013
2319099129,2cyclictest31068-21grep10:15:152
2319099128,2cyclictest0-21swapper/212:25:002
2319099128,2cyclictest0-21swapper/212:25:002
2319099126,5cyclictest0-21swapper/207:27:582
2319099122,8cyclictest0-21swapper/211:06:282
2319099122,5cyclictest0-21swapper/210:55:262
23190991210,1cyclictest0-21swapper/209:15:152
2319099120,12cyclictest0-21swapper/208:34:562
2319099120,11cyclictest0-21swapper/210:00:062
2318799129,2cyclictest30793-21cstates10:15:121
2318799129,2cyclictest15440-21cat09:45:251
2318799129,2cyclictest1166-21runrttasks08:41:291
2318799129,2cyclictest1132-21snmpd11:02:281
2318799129,2cyclictest0-21swapper/108:35:251
23187991211,1cyclictest0-21swapper/110:50:201
2318799120,5cyclictest0-21swapper/108:23:161
2318299128,3cyclictest12272-21flock09:40:200
2318299128,2cyclictest0-21swapper/007:23:420
2318299126,5cyclictest0-21swapper/012:12:120
2318299122,8cyclictest0-21swapper/010:18:380
2318299120,12cyclictest0-21swapper/012:37:460
2318299120,12cyclictest0-21swapper/008:24:340
2318299120,11cyclictest0-21swapper/010:38:130
2318299120,1cyclictest0-21swapper/010:29:520
2318299120,1cyclictest0-21swapper/008:49:490
2318299120,1cyclictest0-21swapper/007:52:110
2153752120,5pltrace0-21swapper/709:59:487
203952120,5pltrace0-21swapper/009:26:310
1039352120,5pltrace0-21swapper/609:44:236
2714452110,1pltrace0-21swapper/110:13:381
245452110,1pltrace0-21swapper/112:19:451
2435652110,0pltrace0-21swapper/710:05:227
2327052110,1pltrace0-21swapper/009:08:190
2321099117,3cyclictest1132-21snmpd12:21:127
2321099117,3cyclictest1132-21snmpd12:21:127
2321099117,2cyclictest0-21swapper/708:05:467
2321099116,2cyclictest0-21swapper/712:03:467
2321099115,5cyclictest0-21swapper/712:10:587
23210991110,1cyclictest0-21swapper/711:50:167
23210991110,1cyclictest0-21swapper/710:40:197
23210991110,1cyclictest0-21swapper/708:15:257
2321099110,11cyclictest0-21swapper/709:40:447
2321099110,11cyclictest0-21swapper/709:33:367
2321099110,11cyclictest0-21swapper/709:33:367
2320799118,2cyclictest2083-21expr11:20:126
2320799117,3cyclictest1132-21snmpd10:39:216
2320799116,3cyclictest0-21swapper/610:07:206
2320799116,3cyclictest0-21swapper/607:19:216
2320799115,5cyclictest0-21swapper/610:49:486
2320799110,10cyclictest0-21swapper/608:51:136
2320799110,1cyclictest0-21swapper/612:08:546
2320799110,1cyclictest0-21swapper/608:33:246
2320799110,1cyclictest0-21swapper/607:57:176
2320499118,2cyclictest15343-21crond11:45:005
2320499118,2cyclictest0-21swapper/507:20:155
2320499117,3cyclictest11611-21/usr/sbin/munin11:35:195
2320499117,2cyclictest0-21swapper/512:22:225
2320499117,2cyclictest0-21swapper/512:22:225
2320499117,2cyclictest0-21swapper/508:27:465
2320499116,4cyclictest0-21swapper/511:12:045
2320499116,3cyclictest0-21swapper/508:22:225
23204991110,1cyclictest0-21swapper/512:34:585
23204991110,1cyclictest0-21swapper/511:53:215
2320499110,10cyclictest0-21swapper/507:16:195
2320499110,1cyclictest0-21swapper/512:09:425
2320499110,1cyclictest0-21swapper/509:23:555
2320499110,1cyclictest0-21swapper/507:51:565
2320499110,1cyclictest0-21swapper/507:34:085
2320499110,1cyclictest0-21swapper/507:29:365
2319999119,1cyclictest0-21swapper/412:29:374
2319999119,1cyclictest0-21swapper/409:45:094
2319999119,1cyclictest0-21swapper/408:24:594
2319999118,2cyclictest0-21swapper/409:17:214
2319999110,10cyclictest0-21swapper/412:14:244
2319999110,10cyclictest0-21swapper/412:08:054
2319999110,10cyclictest0-21swapper/410:12:204
2319999110,10cyclictest0-21swapper/407:46:174
2319999110,1cyclictest0-21swapper/411:07:424
2319999110,1cyclictest0-21swapper/407:55:414
2319999110,1cyclictest0-21swapper/407:29:174
2319599118,2cyclictest25602-21perl10:05:143
2319599118,2cyclictest1970-21missed_timers10:20:183
2319599118,2cyclictest1659-21sensors12:15:233
2319599118,2cyclictest1038-21systemd-logind10:45:003
2319599117,3cyclictest1132-21snmpd07:53:213
2319599116,3cyclictest0-21swapper/310:51:483
2319599116,3cyclictest0-21swapper/309:54:143
2319599116,2cyclictest0-21swapper/310:00:363
23195991110,1cyclictest0-21swapper/311:17:503
2319599110,4cyclictest543552pltrace07:37:183
2319599110,11cyclictest0-21swapper/311:49:343
2319599110,11cyclictest0-21swapper/309:07:003
2319599110,11cyclictest0-21swapper/308:34:083
2319599110,1cyclictest0-21swapper/310:36:033
2319599110,1cyclictest0-21swapper/307:42:203
2319099119,1cyclictest0-21swapper/208:15:492
2319099117,3cyclictest0-21swapper/210:10:332
2319099115,5cyclictest0-21swapper/211:15:162
2319099112,8cyclictest0-21swapper/210:41:402
2319099112,2cyclictest0-21swapper/211:29:522
23190991110,1cyclictest0-21swapper/212:32:272
23190991110,1cyclictest0-21swapper/211:20:242
23190991110,1cyclictest0-21swapper/208:50:092
23190991110,1cyclictest0-21swapper/208:45:172
2319099111,2cyclictest6652-21cstates10:30:122
2319099110,11cyclictest0-21swapper/211:02:042
2319099110,11cyclictest0-21swapper/208:10:292
2319099110,1cyclictest0-21swapper/210:50:562
2319099110,1cyclictest0-21swapper/209:31:162
2319099110,1cyclictest0-21swapper/209:31:162
2319099110,1cyclictest0-21swapper/209:00:012
2319099110,1cyclictest0-21swapper/208:42:522
2318799119,1cyclictest0-21swapper/112:20:211
2318799119,1cyclictest0-21swapper/112:20:211
2318799119,1cyclictest0-21swapper/112:05:171
2318799119,1cyclictest0-21swapper/109:15:151
2318799119,1cyclictest0-21swapper/108:05:121
2318799118,2cyclictest396-21cut09:20:151
2318799118,2cyclictest12117-21grep11:35:221
2318799118,2cyclictest0-21swapper/108:25:141
23187991110,1cyclictest0-21swapper/109:01:511
23187991110,1cyclictest0-21swapper/107:34:451
2318799110,9cyclictest0-21swapper/109:26:361
2318799110,9cyclictest0-21swapper/107:27:161
2318799110,11cyclictest0-21swapper/112:03:591
2318799110,11cyclictest0-21swapper/111:50:041
2318799110,10cyclictest0-21swapper/107:22:371
2318799110,1cyclictest0-21swapper/111:22:141
2318799110,1cyclictest0-21swapper/109:37:381
2318799110,1cyclictest0-21swapper/109:32:561
2318799110,1cyclictest0-21swapper/109:32:561
2318799110,1cyclictest0-21swapper/107:47:131
2318799110,1cyclictest0-21swapper/107:37:501
2318299119,1cyclictest0-21swapper/009:59:260
2318299118,2cyclictest0-21swapper/010:30:170
2318299117,3cyclictest1132-21snmpd09:39:220
2318299117,2cyclictest0-21swapper/011:36:270
2318299116,3cyclictest0-21swapper/011:51:230
23182991110,1cyclictest0-21swapper/010:54:390
2318299110,11cyclictest0-21swapper/010:14:390
2318299110,11cyclictest0-21swapper/010:07:020
2318299110,11cyclictest0-21swapper/009:53:070
2318299110,1cyclictest0-21swapper/009:34:590
2318299110,1cyclictest0-21swapper/009:34:590
1146252110,4pltrace0-21swapper/110:41:191
802152100,6pltrace54-21ksoftirqd/412:33:464
2321099109,1cyclictest0-21swapper/712:16:597
2321099107,2cyclictest1132-21snmpd10:48:127
2321099106,3cyclictest1132-21snmpd11:46:157
2321099101,8cyclictest0-21swapper/710:20:187
2321099101,2cyclictest0-21swapper/709:00:167
2321099100,10cyclictest0-21swapper/711:57:537
2321099100,10cyclictest0-21swapper/707:48:517
2321099100,1cyclictest0-21swapper/711:06:037
2320799109,1cyclictest0-21swapper/610:24:476
2320799109,1cyclictest0-21swapper/608:48:476
2320799109,1cyclictest0-21swapper/607:54:416
2320799108,1cyclictest0-21swapper/608:10:296
2320799107,2cyclictest1700-21lldpd11:44:506
2320799107,2cyclictest1132-21snmpd09:09:186
2320799106,3cyclictest0-21swapper/608:02:556
2320799106,2cyclictest0-21swapper/611:19:406
2320799105,3cyclictest0-21swapper/611:36:146
2320799104,3cyclictest0-21swapper/609:47:406
2320799101,8cyclictest0-21swapper/609:36:566
2320799100,10cyclictest0-21swapper/610:18:286
2320799100,10cyclictest0-21swapper/609:13:426
2320799100,10cyclictest0-21swapper/608:57:216
2320499109,1cyclictest0-21swapper/510:52:285
2320499109,1cyclictest0-21swapper/510:09:245
2320499109,1cyclictest0-21swapper/508:14:175
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional