You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-05 - 13:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Wed Feb 05, 2025 12:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1035289996057,3cyclictest1478765-21kworker/u16:1+flush-8:019:49:195
103499725544,7sleep40-21swapper/415:42:464
103489325443,8sleep60-21swapper/615:41:206
103482825319,30sleep70-21swapper/715:40:257
103530299440,42cyclictest1349833-21latency_hist18:17:517
1035289994138,2cyclictest1122402-21kworker/u16:1+flush-8:016:33:075
1035284994139,2cyclictest1478765-21kworker/u16:1+flush-8:019:48:064
103492124131,7sleep50-21swapper/515:41:435
1035289993633,2cyclictest994668-21kworker/u16:4+events_unbound16:28:065
1035293993431,2cyclictest1414683-21kworker/u16:0+events_unbound19:08:056
1035289993431,2cyclictest978038-21kworker/u16:0+flush-8:015:53:075
1035284993432,2cyclictest994668-21kworker/u16:4+events_unbound16:27:544
1035289993331,2cyclictest1381847-21kworker/u16:4+events_unbound18:48:065
1035289993331,2cyclictest1265903-21kworker/u16:4+events_unbound17:43:095
1035284993331,2cyclictest1531280-21kworker/u16:4+events_unbound21:03:144
103530299320,30cyclictest643-21systemd-journal19:27:527
1035293993230,2cyclictest1403768-21kworker/u16:2+flush-8:019:32:476
1035284993230,2cyclictest1446738-21kworker/u16:1+events_unbound19:22:534
1035289993129,2cyclictest1265903-21kworker/u16:4+events_unbound17:38:065
1035284993129,2cyclictest1478765-21kworker/u16:1+events_unbound19:43:104
1035284993129,2cyclictest1478765-21kworker/u16:1+events_unbound19:43:104
1035284993129,2cyclictest1414683-21kworker/u16:0+events_unbound19:07:464
1035284993128,2cyclictest1265903-21kworker/u16:4+events_unbound17:47:404
15873572300,1chrt1587356-21iostat_ios20:48:077
103530299300,28cyclictest1045549-21latency_hist15:47:527
1035284993028,2cyclictest1531280-21kworker/u16:4+events_unbound20:28:154
1035284993028,2cyclictest1499659-21kworker/u16:5+events_unbound20:03:144
1035284993028,2cyclictest1350046-21kworker/u16:2+events_unbound18:43:104
1035284993027,2cyclictest1111025-21kworker/u16:2+events_unbound16:50:484
15036732290,1chrt1503674-21kthreadcore19:53:117
103530299290,27cyclictest1286739-21latency_hist17:42:527
1035293992926,2cyclictest1350046-21kworker/u16:2+events_unbound18:22:596
1035284992927,2cyclictest1531280-21kworker/u16:4+events_unbound20:57:534
1035302992826,1cyclictest0-21swapper/717:23:097
1035302992826,1cyclictest0-21swapper/717:23:097
1035284992826,2cyclictest1531280-21kworker/u16:4+flush-8:020:48:114
103530299270,25cyclictest1478014-21latency_hist19:37:517
1035289992724,2cyclictest1499659-21kworker/u16:5+flush-8:020:23:015
1035284992725,2cyclictest1122402-21kworker/u16:1+events_unbound16:32:534
1035293992624,2cyclictest1285826-21kworker/u16:3+events_unbound18:13:166
1035284992624,2cyclictest978038-21kworker/u16:0+events_unbound16:18:124
1035284992624,2cyclictest978038-21kworker/u16:0+events_unbound16:13:364
1035284992624,2cyclictest1153397-21kworker/u16:3+events_unbound17:17:354
1035293992522,3cyclictest1478765-21kworker/u16:1+events_unbound19:50:346
1035289992523,2cyclictest1531279-21kworker/u16:3+events_unbound20:33:065
1035289992523,2cyclictest1531279-21kworker/u16:3+events_unbound20:33:065
1035284992523,2cyclictest1153397-21kworker/u16:3+events_unbound17:03:284
1035289992422,2cyclictest1403768-21kworker/u16:2+events_unbound19:28:195
1035289992422,2cyclictest1329960-21kworker/u16:1+flush-8:018:33:035
1035289992422,2cyclictest1204997-21kworker/u16:2+events_unbound17:48:195
1035289992420,3cyclictest1111025-21kworker/u16:2+events_unbound16:38:105
1035289992420,3cyclictest1111025-21kworker/u16:2+events_unbound16:38:105
1035284992423,1cyclictest1371528-21kworker/u16:3+events_unbound19:16:024
1035284992422,2cyclictest1531280-21kworker/u16:4+flush-8:020:33:524
1035284992422,2cyclictest1531280-21kworker/u16:4+flush-8:020:33:524
1035284992422,2cyclictest1531280-21kworker/u16:4+events_unbound20:43:144
1035293992321,2cyclictest1403769-21kworker/u16:5+events_unbound19:33:516
1035284992322,1cyclictest1122402-21kworker/u16:1+events_unbound16:43:154
1035284992321,2cyclictest1570729-21kworker/u16:1+events_unbound20:42:024
1035284992321,2cyclictest1478765-21kworker/u16:1+events_unbound20:12:234
1035284992321,2cyclictest1296647-21kworker/u16:5+flush-8:018:17:464
1035284992320,2cyclictest994668-21kworker/u16:4+events_unbound15:48:104
1035284992320,2cyclictest994668-21kworker/u16:4+events_unbound15:48:104
1035293992220,2cyclictest1381847-21kworker/u16:4+events_unbound18:48:526
1035293992220,2cyclictest1141972-21kworker/u16:0+flush-8:016:37:476
1035289992220,2cyclictest978038-21kworker/u16:0+events_unbound16:03:095
1035289992219,2cyclictest1350046-21kworker/u16:2+events_unbound18:23:125
1035284992220,2cyclictest1204997-21kworker/u16:2+flush-8:017:28:094
103530299210,20cyclictest1479105-21latency_hist19:42:517
1035293992119,2cyclictest1499659-21kworker/u16:5+events_unbound20:23:106
1035293992119,2cyclictest1478765-21kworker/u16:1+events_unbound20:07:466
1035293992119,2cyclictest1403768-21kworker/u16:2+events_unbound18:55:516
1035293992119,2cyclictest1225637-21kworker/u16:1+events_unbound17:52:396
1035289992119,2cyclictest1225637-21kworker/u16:1+events_unbound17:32:555
1035289992119,2cyclictest1122402-21kworker/u16:1+flush-8:016:58:185
1035284992119,2cyclictest1414683-21kworker/u16:0+events_unbound19:29:024
1035284992119,2cyclictest1153397-21kworker/u16:3+flush-8:016:40:034
1035284992119,2cyclictest1153397-21kworker/u16:3+flush-8:016:40:034
1035302992017,2cyclictest1172534-21sensors_fan16:43:227
1035293992018,2cyclictest1552812-21kworker/u16:0+events_unbound20:47:516
1035289992018,2cyclictest947260-21kworker/u16:3+events_unbound15:43:235
1035289992018,2cyclictest1371528-21kworker/u16:3+events_unbound19:08:505
1035289992018,2cyclictest1329960-21kworker/u16:1+events_unbound18:42:475
1035284992018,2cyclictest1457551-21kworker/u16:3+events_unbound19:33:164
1035284992018,2cyclictest1371528-21kworker/u16:3+flush-8:018:53:174
1035284992018,2cyclictest1204997-21kworker/u16:2+events_unbound17:08:224
103530299190,17cyclictest1489478-21latency_hist19:47:517
103530299190,17cyclictest1489478-21latency_hist19:47:517
1035293991917,2cyclictest1153397-21kworker/u16:3+flush-8:017:17:466
1035289991917,2cyclictest1552812-21kworker/u16:0+events_unbound20:30:425
1035289991917,2cyclictest1141972-21kworker/u16:0+events_unbound17:14:585
1035289991916,2cyclictest978038-21kworker/u16:0+events_unbound15:57:545
1035289991916,2cyclictest1457551-21kworker/u16:3+flush-8:019:58:025
1035284991917,2cyclictest1403768-21kworker/u16:2+events_unbound20:16:514
1035284991917,2cyclictest1350046-21kworker/u16:2+events_unbound18:32:534
1035284991917,2cyclictest1153397-21kworker/u16:3+events_unbound16:56:154
1035302991816,1cyclictest1607599-21awk21:02:517
1035293991816,2cyclictest1570729-21kworker/u16:1+events_unbound20:38:386
1035293991816,2cyclictest1478765-21kworker/u16:1+events_unbound19:57:596
1035293991816,2cyclictest1153397-21kworker/u16:3+events_unbound17:21:396
1035293991816,2cyclictest1080157-21kworker/u16:5+events_unbound16:04:066
103529399180,18cyclictest0-21swapper/615:44:216
1035289991816,2cyclictest994668-21kworker/u16:4+flush-8:015:48:105
1035289991816,2cyclictest994668-21kworker/u16:4+flush-8:015:48:105
1035289991816,2cyclictest1478765-21kworker/u16:1+events_unbound20:03:005
1035289991816,2cyclictest1403768-21kworker/u16:2+events_unbound19:26:455
1035289991816,2cyclictest1296647-21kworker/u16:5+events_unbound18:09:555
1035289991815,2cyclictest1153397-21kworker/u16:3+events_unbound17:22:555
1035289991815,2cyclictest1153397-21kworker/u16:3+events_unbound17:22:555
1035284991816,2cyclictest1318546-21kworker/u16:6+events_unbound18:07:274
103530299170,1cyclictest0-21swapper/718:43:107
103530299170,15cyclictest1522494-21sendmail20:07:527
103530299170,15cyclictest1317788-21latency_hist17:57:517
1035293991715,2cyclictest1478765-21kworker/u16:1+events_unbound19:53:066
1035289991715,2cyclictest978038-21kworker/u16:0+events_unbound16:13:545
1035289991715,2cyclictest1570729-21kworker/u16:1+events_unbound20:40:165
1035289991715,2cyclictest1296647-21kworker/u16:5+events_unbound18:04:275
1035289991714,2cyclictest1531280-21kworker/u16:4+events_unbound20:59:235
103528999170,13cyclictest0-21swapper/517:03:075
1035284991715,2cyclictest978038-21kworker/u16:0+events_unbound16:10:034
1035284991715,2cyclictest1478765-21kworker/u16:1+events_unbound20:00:384
1035284991715,2cyclictest1225637-21kworker/u16:1+flush-8:017:38:174
1035284991714,2cyclictest1457551-21kworker/u16:3+events_unbound19:41:274
103530299160,16cyclictest0-21swapper/719:49:247
103530299160,14cyclictest1381858-21latency_hist18:37:527
103530299160,14cyclictest1214581-21latency_hist17:07:527
1035293991614,2cyclictest1446738-21kworker/u16:1+events_unbound19:23:066
1035293991614,2cyclictest1350046-21kworker/u16:2+events_unbound18:28:296
1035293991614,2cyclictest1214286-21kworker/u16:4+events_unbound17:07:446
1035293991614,2cyclictest1204997-21kworker/u16:2+events_unbound17:57:406
103529399160,1cyclictest0-21swapper/619:13:186
1035289991614,2cyclictest994668-21kworker/u16:4+events_unbound16:12:405
1035289991614,2cyclictest1531280-21kworker/u16:4+events_unbound20:48:425
1035289991614,2cyclictest1457551-21kworker/u16:3+events_unbound19:55:135
1035289991614,2cyclictest1403769-21kworker/u16:5+events_unbound19:12:535
1035289991614,2cyclictest1225637-21kworker/u16:1+events_unbound17:56:055
1035289991614,2cyclictest1080157-21kworker/u16:5+events_unbound16:25:595
103528999160,15cyclictest0-21swapper/516:18:175
1035284991614,2cyclictest1329960-21kworker/u16:1+events_unbound18:31:034
1035284991614,2cyclictest1316880-21kworker/u16:0+events_unbound18:08:064
1035284991613,2cyclictest1531279-21kworker/u16:3+flush-8:020:53:014
1035284991613,2cyclictest1122402-21kworker/u16:1+events_unbound16:58:044
103530299150,13cyclictest1131972-21latency_hist16:27:517
1035293991513,2cyclictest1607310-21kworker/u16:1+events_unbound21:07:016
1035293991513,2cyclictest1381847-21kworker/u16:4+events_unbound18:44:156
1035293991513,2cyclictest1285826-21kworker/u16:3+events_unbound18:21:396
1035293991513,2cyclictest1031041-21kworker/u16:2+events_unbound15:54:476
1035289991513,2cyclictest1531280-21kworker/u16:4+events_unbound20:47:165
1035289991513,2cyclictest1403768-21kworker/u16:2+events_unbound19:44:075
1035289991513,2cyclictest1403768-21kworker/u16:2+events_unbound19:44:075
1035289991513,2cyclictest1403768-21kworker/u16:2+events_unbound19:01:275
1035289991513,2cyclictest1403768-21kworker/u16:2+events_unbound19:01:275
1035289991513,2cyclictest1350046-21kworker/u16:2+events_unbound18:19:085
1035289991513,2cyclictest1204997-21kworker/u16:2+events_unbound17:28:085
1035284991513,2cyclictest1575567-21kworker/u16:2+events_unbound21:10:024
1035284991513,2cyclictest1499659-21kworker/u16:5+events_unbound20:20:514
1035284991513,2cyclictest1457551-21kworker/u16:3+events_unbound19:54:584
1035284991513,2cyclictest1141972-21kworker/u16:0+events_unbound17:36:224
1035284991513,2cyclictest1141972-21kworker/u16:0+events_unbound17:25:514
1035284991513,2cyclictest1141972-21kworker/u16:0+events_unbound17:25:514
1035284991513,2cyclictest1080157-21kworker/u16:5+events_unbound15:59:364
103530299141,4cyclictest1445020-21ntpq19:13:187
1035302991413,0cyclictest0-21swapper/715:47:537
1035302991413,0cyclictest0-21swapper/715:47:537
1035302991410,3cyclictest1350948-21awk18:22:517
103530299140,13cyclictest1435667-21turbostat19:12:517
103530299140,13cyclictest1142299-21latency_hist16:32:527
1035293991412,2cyclictest978038-21kworker/u16:0+events_unbound16:07:566
1035293991412,2cyclictest1531280-21kworker/u16:4+events_unbound21:08:386
1035293991412,2cyclictest1478765-21kworker/u16:1+events_unbound19:43:306
1035293991412,2cyclictest1478765-21kworker/u16:1+events_unbound19:43:306
1035293991412,2cyclictest1265903-21kworker/u16:4+events_unbound17:46:466
1035293991412,2cyclictest1225637-21kworker/u16:1+events_unbound17:40:526
1035293991412,2cyclictest1214286-21kworker/u16:4+events_unbound17:10:156
1035293991412,2cyclictest1141972-21kworker/u16:0+flush-8:017:27:316
1035293991412,2cyclictest1141972-21kworker/u16:0+flush-8:017:27:316
1035293991412,2cyclictest1122402-21kworker/u16:1+events_unbound17:00:236
1035293991412,2cyclictest1080157-21kworker/u16:5+events_unbound16:50:526
1035293991411,2cyclictest1552812-21kworker/u16:0+events_unbound20:27:556
1035289991412,2cyclictest1552812-21kworker/u16:0+events_unbound21:03:285
1035289991412,2cyclictest1552812-21kworker/u16:0+events_unbound20:55:175
1035289991412,2cyclictest1414683-21kworker/u16:0+flush-8:019:05:325
1035289991412,2cyclictest1381847-21kworker/u16:4+events_unbound18:43:445
1035289991412,2cyclictest1350046-21kworker/u16:2+events_unbound18:31:565
1035289991412,2cyclictest1204997-21kworker/u16:2+events_unbound17:08:105
1035289991412,2cyclictest1080157-21kworker/u16:5+events_unbound16:48:005
1035289991411,2cyclictest1371528-21kworker/u16:3+events_unbound19:17:545
1035284991412,2cyclictest994668-21kworker/u16:4+events_unbound16:25:434
1035284991412,2cyclictest978038-21kworker/u16:0+flush-8:015:53:184
1035284991412,2cyclictest978038-21kworker/u16:0+events_unbound16:03:364
1035284991412,2cyclictest947260-21kworker/u16:3+events_unbound15:45:164
1035284991412,2cyclictest1531280-21kworker/u16:4+flush-8:020:23:184
103530299136,6cyclictest27430-21gmain17:43:187
103530299133,8cyclictest0-21swapper/717:33:087
103530299132,10cyclictest1605520-21ntpq20:53:177
103530299131,2cyclictest1466936-21smart_sda19:28:217
103530299131,2cyclictest1412951-21ntpq18:53:187
103530299130,11cyclictest1574060-21users20:33:227
103530299130,11cyclictest1574060-21users20:33:227
103530299130,11cyclictest1313-21snmpd20:23:047
103530299130,11cyclictest1313-21snmpd16:50:267
103529399137,2cyclictest1139088-21cut16:28:116
103529399136,4cyclictest1056831-21kthreadcore15:48:126
103529399136,4cyclictest1056831-21kthreadcore15:48:126
1035293991311,2cyclictest1575567-21kworker/u16:2+events_unbound20:53:536
1035293991311,2cyclictest1531279-21kworker/u16:3+events_unbound20:49:236
1035293991311,2cyclictest1457551-21kworker/u16:3+events_unbound19:39:186
1035293991311,2cyclictest1446738-21kworker/u16:1+events_unbound19:21:406
1035293991311,2cyclictest1403769-21kworker/u16:5+events_unbound19:02:556
1035293991311,2cyclictest1225637-21kworker/u16:1+events_unbound17:34:526
1035293991311,2cyclictest1204997-21kworker/u16:2+events_unbound17:28:436
1035293991311,2cyclictest1122402-21kworker/u16:1+events_unbound16:46:116
1035293991311,2cyclictest1111025-21kworker/u16:2+events_unbound16:23:486
1035289991311,2cyclictest1552812-21kworker/u16:0+events_unbound21:07:535
1035289991311,2cyclictest1457551-21kworker/u16:3+events_unbound19:40:325
1035289991311,2cyclictest1403768-21kworker/u16:2+events_unbound19:33:455
1035289991311,2cyclictest1371528-21kworker/u16:3+events_unbound18:54:275
1035289991311,2cyclictest1296647-21kworker/u16:5+events_unbound18:00:115
1035289991311,2cyclictest1225637-21kworker/u16:1+events_unbound17:18:295
1035289991310,3cyclictest1111025-21kworker/u16:2+flush-8:016:53:225
1035289991310,2cyclictest1080157-21kworker/u16:5+events_unbound16:43:035
1035284991310,2cyclictest1392940-21kworker/u16:0+flush-8:018:49:144
103530299129,2cyclictest1115453-21kthreadcore16:18:107
103530299122,9cyclictest1616746-21memory21:03:117
103530299121,2cyclictest1627258-21ntpq21:08:187
103530299121,2cyclictest1520818-21ntpq20:03:137
103530299121,2cyclictest1069160-21ntpq15:53:187
103530299121,1cyclictest0-21swapper/717:08:067
103530299121,1cyclictest0-21swapper/716:53:237
103530299121,10cyclictest1416495-21hddtemp_smartct19:03:037
1035302991211,0cyclictest0-21swapper/720:13:037
1035302991211,0cyclictest0-21swapper/719:18:497
103530299120,3cyclictest1382580-21cron18:37:527
103530299120,11cyclictest96-21ksoftirqd/718:03:117
103530299120,11cyclictest1313-21snmpd18:58:417
103530299120,11cyclictest1313-21snmpd18:58:417
103530299120,11cyclictest1313-21snmpd16:35:567
103530299120,10cyclictest1576978-21hddtemp_smartct20:43:027
103530299120,10cyclictest1555279-21hddtemp_smartct20:28:087
103529399129,2cyclictest1186932-21cut16:53:126
103529399129,2cyclictest1122402-21kworker/u16:1+events_unbound16:38:006
103529399129,2cyclictest1122402-21kworker/u16:1+events_unbound16:38:006
1035293991210,2cyclictest1531280-21kworker/u16:4+events_unbound20:35:036
1035293991210,2cyclictest1531280-21kworker/u16:4+events_unbound20:35:036
1035293991210,2cyclictest1499659-21kworker/u16:5+events_unbound20:20:226
1035293991210,2cyclictest1499659-21kworker/u16:5+events_unbound20:07:546
1035293991210,2cyclictest1296647-21kworker/u16:5+events_unbound18:10:426
1035293991210,2cyclictest1285826-21kworker/u16:3+events_unbound18:00:156
1035293991210,1cyclictest1373948-21kthreadcore18:33:066
103529399120,5cyclictest0-21swapper/618:03:036
103529399120,1cyclictest0-21swapper/621:00:016
1035289991211,1cyclictest0-21swapper/520:21:525
1035289991210,2cyclictest1403768-21kworker/u16:2+events_unbound20:15:215
103528999120,1cyclictest0-21swapper/520:08:285
103528499129,2cyclictest1296647-21kworker/u16:5+events_unbound17:48:044
1035284991211,1cyclictest0-21swapper/419:20:544
1035284991210,2cyclictest1285826-21kworker/u16:3+flush-8:017:53:124
103530299116,0cyclictest0-21swapper/718:00:427
103530299113,7cyclictest0-21swapper/716:08:077
103530299112,7cyclictest0-21swapper/717:28:227
103530299111,1cyclictest1370880-21find18:28:207
103530299110,3cyclictest1195409-21taskset16:58:067
103530299110,10cyclictest1313-21snmpd18:48:027
103530299110,10cyclictest1313-21snmpd18:11:347
103530299110,10cyclictest1313-21snmpd17:14:577
103530299110,10cyclictest1313-21snmpd16:17:317
103530299110,10cyclictest1313-21snmpd16:07:197
103530299110,10cyclictest0-21swapper/717:48:167
103529399119,2cyclictest1499659-21kworker/u16:5+events_unbound20:17:096
103529399119,2cyclictest1403769-21kworker/u16:5+events_unbound18:59:306
103529399119,2cyclictest1403769-21kworker/u16:5+events_unbound18:59:306
103529399119,2cyclictest1350046-21kworker/u16:2+flush-8:018:41:386
103529399119,1cyclictest1106695-21cut16:13:116
103529399118,2cyclictest1076661-21kthreadcore15:58:126
103528999119,2cyclictest1285826-21kworker/u16:3+events_unbound18:13:525
103528499119,2cyclictest1403769-21kworker/u16:5+flush-8:019:08:174
103528499119,2cyclictest1204997-21kworker/u16:2+events_unbound17:20:404
103530299106,3cyclictest1161044-21kthreadcore16:38:117
103530299106,3cyclictest1161044-21kthreadcore16:38:117
103530299102,7cyclictest0-21swapper/715:58:077
103530299100,9cyclictest1313-21snmpd17:21:577
103530299100,4cyclictest1313-21snmpd20:40:077
103530299100,1cyclictest0-21swapper/718:23:097
103529399109,1cyclictest1113328-21kthreadcore16:18:096
103528499109,1cyclictest0-21swapper/418:24:564
103528499108,2cyclictest1381847-21kworker/u16:4+events_unbound18:57:544
103528499108,2cyclictest1381847-21kworker/u16:4+events_unbound18:57:544
103528499108,2cyclictest1350046-21kworker/u16:2+events_unbound18:41:464
103528499108,2cyclictest1296647-21kworker/u16:5+events_unbound18:22:454
10353029990,8cyclictest1313-21snmpd20:18:047
10353029990,8cyclictest1313-21snmpd20:01:127
10352849998,1cyclictest0-21swapper/418:01:554
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional