You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 03:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Thu Nov 21, 2024 12:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23272199912598,9cyclictest2348103-21latency_hist08:15:011
232721599110103,5cyclictest2387160-21kworker/u8:2+flush-179:010:25:000
23272199910699,5cyclictest2348094-21kworker/u8:0+flush-179:008:25:011
23272199910599,4cyclictest2393446-21kworker/u8:3+events_unbound11:15:251
23272159910493,7cyclictest2408067-21latency_hist11:25:010
23272199910289,10cyclictest2422650-21latency_hist12:10:011
2327215999988,8cyclictest2429030-21apt-get12:30:000
2327219999789,5cyclictest2380982-21kworker/u8:2+flush-179:010:05:011
2327215999585,7cyclictest2409640-21kworker/u8:3+flush-179:011:30:230
2327219999385,5cyclictest2333502-21kworker/u8:2+flush-179:007:50:251
2327219999383,8cyclictest2429160-21gpgv12:30:011
2327215999381,9cyclictest2417869-21latency_hist11:55:010
2327219999287,4cyclictest2353180-21kworker/u8:2+flush-179:008:40:231
2327219999286,4cyclictest2353180-21kworker/u8:2+flush-179:009:05:201
2327219999286,4cyclictest2348094-21kworker/u8:0+flush-179:008:55:011
2327219999286,4cyclictest2348094-21kworker/u8:0+flush-179:008:55:001
2327219999286,4cyclictest2341770-21kworker/u8:1+events_unbound08:00:181
2327215999179,9cyclictest2395329-21latency_hist10:45:000
2327219999084,4cyclictest2408054-21kworker/u8:1+flush-179:011:30:001
2327219998984,4cyclictest2429062-21kworker/u8:1+events_unbound12:30:281
2327219998982,6cyclictest2364586-21kworker/u8:3+flush-179:009:30:011
2327219998979,7cyclictest2367960-21/usr/sbin/munin09:15:271
2327219998882,4cyclictest2383953-21kworker/u8:1+events_unbound10:30:131
2327219998878,7cyclictest2395374-21kworker/u8:0+flush-179:010:55:191
2327219998781,4cyclictest2348094-21kworker/u8:0+events_unbound08:30:141
2327219998780,5cyclictest2395374-21kworker/u8:0+flush-179:010:50:201
2327219998780,5cyclictest2393446-21kworker/u8:3+flush-179:011:15:001
2327219998778,6cyclictest2323901-21kworker/u8:1+flush-179:007:25:201
2327219998680,4cyclictest2393446-21kworker/u8:3+flush-179:011:20:251
2327219998679,5cyclictest2387160-21kworker/u8:2+flush-179:010:25:211
2327219998579,4cyclictest2413145-21kworker/u8:2+flush-179:011:45:221
2327219998579,4cyclictest2341770-21kworker/u8:1+events_unbound08:10:001
2327219998579,4cyclictest2341770-21kworker/u8:1+events_unbound08:10:001
2327219998578,6cyclictest2353180-21kworker/u8:2+flush-179:008:45:211
2327219998578,4cyclictest2348094-21kworker/u8:0+events_unbound09:00:141
2327219998575,7cyclictest2341770-21kworker/u8:1+flush-179:007:55:191
2327219998571,11cyclictest2395586-21apt-get10:45:111
2327219998478,5cyclictest2393446-21kworker/u8:3+flush-179:010:39:551
2327219998478,4cyclictest2330367-21kworker/u8:3+flush-179:007:20:241
2327219998478,4cyclictest2330367-21kworker/u8:3+events_unbound07:35:201
2327215998474,7cyclictest2341770-21kworker/u8:1+flush-179:008:05:230
2327215998474,7cyclictest2341770-21kworker/u8:1+flush-179:008:05:230
2327219998376,5cyclictest2333502-21kworker/u8:2+flush-179:007:45:141
2327219998276,4cyclictest2411231-21kworker/u8:0+flush-179:011:55:211
2327219998275,5cyclictest2326638-21kworker/u8:2+flush-179:007:10:141
2327219998269,10cyclictest2424493-21apt-get12:15:131
2327215998274,7cyclictest2377595-21kworker/u8:1+flush-179:009:55:000
2327215998260,5cyclictest2357938-21apt-get08:45:030
2327219998173,6cyclictest2393446-21kworker/u8:3+flush-179:011:05:271
2327219998173,5cyclictest2411231-21kworker/u8:0+events_unbound11:35:031
2327219998170,8cyclictest2377551-21latency_hist09:50:001
2327215998170,9cyclictest2411739-21/usr/sbin/munin11:35:250
2327219998074,5cyclictest2333502-21kworker/u8:2+events_unbound07:40:261
2327219998074,5cyclictest2333502-21kworker/u8:2+events_unbound07:40:261
2327215998072,5cyclictest2326638-21kworker/u8:2+flush-179:007:15:220
2327215998072,5cyclictest2326638-21kworker/u8:2+flush-179:007:15:210
2327215997952,11cyclictest2358053-21gpgv08:45:000
2327215997870,6cyclictest2424272-21kworker/u8:2+flush-179:012:35:300
2327215997870,6cyclictest2348094-21kworker/u8:0+flush-179:008:15:260
2327215997870,5cyclictest2383918-21latency_hist10:10:010
2327215997870,5cyclictest2383918-21latency_hist10:10:000
2327215997868,7cyclictest2415357-21kworker/u8:1+flush-179:012:00:210
2327215997849,10cyclictest2369589-21apt-get09:20:110
2327219997769,5cyclictest2413145-21kworker/u8:2+events_unbound11:50:201
2327219997758,5cyclictest2350635-21kworker/u8:3+events_unbound08:25:201
2327215997771,4cyclictest2350035-21gpgv08:20:080
2327215997764,11cyclictest2393482-21latency_hist10:40:000
2327219997671,4cyclictest2326638-21kworker/u8:2+flush-179:007:15:191
2327219997671,4cyclictest2326638-21kworker/u8:2+flush-179:007:15:191
2327219997665,8cyclictest3105085-21systemd-journal10:15:001
2327215997669,4cyclictest2326638-21kworker/u8:2+flush-179:007:10:200
2327215997566,7cyclictest2403539-21apt-get11:10:170
2327215997565,7cyclictest2373008-21/usr/sbin/munin09:35:240
2327215997565,7cyclictest2373008-21/usr/sbin/munin09:35:230
2327215997562,10cyclictest2374634-21apt-get09:40:100
2327215997557,15cyclictest3105085-21systemd-journal07:40:000
2327219997469,4cyclictest2353180-21kworker/u8:2+events_unbound08:40:001
2327219997467,5cyclictest2364586-21kworker/u8:3+flush-179:009:15:011
2327215997464,7cyclictest2348094-21kworker/u8:0+flush-179:008:50:220
2327215997464,7cyclictest2348094-21kworker/u8:0+flush-179:008:50:220
2327219997266,3cyclictest2371158-21kworker/u8:0+flush-179:009:35:211
2327219997266,3cyclictest2371158-21kworker/u8:0+flush-179:009:35:201
2327219997265,6cyclictest2380982-21kworker/u8:2+flush-179:010:05:211
2327219997265,6cyclictest2380982-21kworker/u8:2+flush-179:010:05:201
2327215997267,4cyclictest2398542-21kworker/u8:2+flush-179:011:15:260
2327215997264,6cyclictest2348094-21kworker/u8:0+flush-179:008:55:210
2327215997261,8cyclictest2335097-21apt-get07:35:000
2327215997260,9cyclictest3105085-21systemd-journal08:00:000
2327219997165,3cyclictest2371217-21kworker/u8:1+flush-179:009:30:111
2327219997163,5cyclictest2413145-21kworker/u8:2+events_unbound11:40:161
2327215997164,5cyclictest2323901-21kworker/u8:1+flush-179:007:25:220
2327215997064,4cyclictest2350635-21kworker/u8:3+flush-179:008:35:170
2327215997063,4cyclictest2341770-21kworker/u8:1+flush-179:008:00:250
2327219996963,4cyclictest2398542-21kworker/u8:2+flush-179:011:00:151
2327219996963,4cyclictest2353180-21kworker/u8:2+flush-179:009:20:201
2327219996862,4cyclictest2333502-21kworker/u8:2+events_unbound07:30:131
2327219996861,4cyclictest2383953-21kworker/u8:1+events_unbound10:20:151
2327215996861,5cyclictest2333502-21kworker/u8:2+events_unbound07:40:140
2327215996861,5cyclictest2333502-21kworker/u8:2+events_unbound07:40:140
2327215996857,8cyclictest2333502-21kworker/u8:2+events_unbound08:10:210
2327219996760,4cyclictest2364586-21kworker/u8:3+events_unbound09:40:141
2327219996660,5cyclictest2415357-21kworker/u8:1+flush-179:012:05:001
2327215996656,7cyclictest3105085-21systemd-journal12:15:010
2327219996557,7cyclictest2415357-21kworker/u8:1+events_unbound12:10:191
2327219996457,5cyclictest2424272-21kworker/u8:2+events_unbound12:35:131
2327215996457,5cyclictest2393446-21kworker/u8:3+flush-179:011:05:240
2327219996357,4cyclictest2377540-21kworker/u8:0+events_unbound10:00:011
2327219996351,9cyclictest2426101-21apt-get12:20:131
2327215996256,4cyclictest2424272-21kworker/u8:2+flush-179:012:20:010
2327215996253,6cyclictest2323901-21kworker/u8:1+flush-179:007:20:140
2327215996153,5cyclictest2336938-21kworker/u8:4+flush-179:007:45:210
2327219996053,6cyclictest3105085-21systemd-journal09:53:391
2327219995952,5cyclictest2383953-21kworker/u8:1+events_unbound10:40:041
2327215995948,9cyclictest2371438-21apt-get09:30:090
2327215995852,4cyclictest2395374-21kworker/u8:0+flush-179:010:50:160
2327215995843,11cyclictest2382372-21pluginstate10:05:010
2327215995752,4cyclictest2398542-21kworker/u8:2+flush-179:010:55:250
2327215995751,4cyclictest2350635-21kworker/u8:3+flush-179:008:25:130
2327219995550,4cyclictest2377540-21kworker/u8:0+events_unbound10:15:171
2327215995347,4cyclictest2429062-21kworker/u8:1+flush-179:012:30:210
2327215995347,4cyclictest2364586-21kworker/u8:3+events_unbound09:10:190
2327215995247,4cyclictest2348094-21kworker/u8:0+flush-179:008:30:160
2327215995246,4cyclictest2377540-21kworker/u8:0+events_unbound10:15:200
2327219995142,7cyclictest2408054-21kworker/u8:1+events_unbound11:30:131
2327215995140,9cyclictest2400364-21apt-get11:00:100
2327215995136,6cyclictest2383953-21kworker/u8:1+events_unbound10:30:010
2327219995041,7cyclictest2348094-21kworker/u8:0+events_unbound08:15:151
2327219994944,4cyclictest2348094-21kworker/u8:0+flush-179:008:55:211
2327215994944,4cyclictest2377540-21kworker/u8:0+flush-179:010:10:210
2327215994842,5cyclictest2333502-21kworker/u8:2+events_unbound07:50:230
4505291470,13ptp4l441ktimers/307:10:003
4505291470,13ptp4l441ktimers/307:10:003
2327215994640,5cyclictest2393446-21kworker/u8:3+events_unbound10:45:190
2327215994432,10cyclictest3105085-21systemd-journal11:30:000
2327215994339,3cyclictest2413145-21kworker/u8:2+flush-179:011:40:270
232722399410,34cyclictest0-21swapper/311:23:003
2327221994137,2cyclictest0-21swapper/207:41:592
2327221994137,2cyclictest0-21swapper/207:41:592
2327215994136,4cyclictest2348094-21kworker/u8:0+flush-179:009:00:210
2327215994134,5cyclictest2411231-21kworker/u8:0+flush-179:011:55:240
232721599411,38cyclictest722-21Xorg09:16:000
2327223994037,1cyclictest0-21swapper/310:16:593
2327221994036,2cyclictest0-21swapper/207:30:592
2327221993835,2cyclictest0-21swapper/208:38:592
2327221993834,2cyclictest0-21swapper/210:54:002
232722199380,36cyclictest0-21swapper/212:07:592
232722199380,36cyclictest0-21swapper/209:34:582
232722199380,35cyclictest0-21swapper/211:44:552
232722199380,35cyclictest0-21swapper/211:24:592
2327215993833,4cyclictest2415357-21kworker/u8:1+flush-179:012:05:210
2327215993832,4cyclictest2383953-21kworker/u8:1+flush-179:010:30:250
2327215993831,5cyclictest2353180-21kworker/u8:2+events_unbound09:10:010
2327223993732,4cyclictest2422661-21turbostat12:09:593
2327215993733,3cyclictest2380781-21systemd-run10:00:000
232721599370,1cyclictest0-21swapper/009:29:590
4505291360,3ptp4l441ktimers/310:35:123
232722399360,35cyclictest0-21swapper/311:45:583
232722399360,35cyclictest0-21swapper/311:43:593
232722399360,35cyclictest0-21swapper/308:39:593
232722399360,1cyclictest0-21swapper/307:29:593
2327221993634,1cyclictest0-21swapper/212:14:592
232722199360,1cyclictest0-21swapper/210:20:592
232722199360,1cyclictest0-21swapper/209:11:592
232722199360,1cyclictest0-21swapper/208:12:592
2327223993533,1cyclictest0-21swapper/312:15:003
2327223993533,1cyclictest0-21swapper/310:22:583
2327223993532,2cyclictest0-21swapper/310:59:583
232722399350,1cyclictest0-21swapper/309:16:003
232722399350,1cyclictest0-21swapper/309:01:003
2327221993533,1cyclictest0-21swapper/211:09:592
2327221993532,2cyclictest0-21swapper/208:47:592
2327221993532,2cyclictest0-21swapper/207:24:592
2327221993531,2cyclictest0-21swapper/209:37:592
2327221993531,2cyclictest0-21swapper/209:37:592
2327215993527,5cyclictest2427433-21latency_hist12:25:010
2327223993432,1cyclictest0-21swapper/311:36:593
2327223993432,1cyclictest0-21swapper/308:51:593
2327223993432,1cyclictest0-21swapper/308:51:593
2327223993431,2cyclictest0-21swapper/310:31:003
2327223993428,4cyclictest2328719-21turbostat07:15:003
2327221993431,2cyclictest0-21swapper/211:14:562
2327221993431,2cyclictest0-21swapper/208:54:592
2327221993431,2cyclictest0-21swapper/208:54:592
2327221993431,2cyclictest0-21swapper/208:20:002
2327221993431,2cyclictest0-21swapper/208:01:592
2327221993430,2cyclictest0-21swapper/210:31:582
2327221993430,2cyclictest0-21swapper/209:49:592
2327223993331,1cyclictest0-21swapper/307:52:003
232722399330,32cyclictest0-21swapper/309:22:593
2327221993330,2cyclictest0-21swapper/207:35:592
2327221993329,2cyclictest0-21swapper/207:53:592
232721599330,31cyclictest722-21Xorg09:47:590
2327223993229,2cyclictest0-21swapper/309:32:003
232722399320,31cyclictest2425841-21turbostat12:20:003
232722199320,30cyclictest0-21swapper/210:11:592
4505291310,11ptp4l61790irq/134-eno1-TxRx-009:40:013
4505291310,11ptp4l61790irq/134-eno1-TxRx-009:40:003
2327223993129,1cyclictest0-21swapper/307:22:593
2327223993128,2cyclictest0-21swapper/309:11:593
2327223993128,2cyclictest0-21swapper/309:05:593
232722399310,27cyclictest0-21swapper/312:00:133
2327221993128,2cyclictest0-21swapper/211:39:592
2327221993128,2cyclictest0-21swapper/207:49:562
2327221993127,2cyclictest0-21swapper/208:40:592
232722199310,2cyclictest0-21swapper/212:00:142
232722399300,28cyclictest0-21swapper/308:25:123
2327221993027,2cyclictest0-21swapper/211:57:582
2327221993027,2cyclictest0-21swapper/209:00:122
2327221993027,1cyclictest0-21swapper/210:45:002
232722199300,28cyclictest0-21swapper/212:30:032
2327223992927,1cyclictest0-21swapper/310:55:003
2327223992927,1cyclictest0-21swapper/308:12:593
232722399290,28cyclictest2357927-21turbostat08:44:593
232722399290,26cyclictest0-21swapper/307:30:133
2327221992926,2cyclictest0-21swapper/210:25:592
2327221992926,2cyclictest0-21swapper/210:05:132
2327221992926,2cyclictest0-21swapper/210:05:132
2327221992926,2cyclictest0-21swapper/210:00:002
2327221992926,2cyclictest0-21swapper/207:16:002
2327221992926,2cyclictest0-21swapper/207:15:592
232722199290,28cyclictest0-21swapper/207:10:132
232722199290,27cyclictest0-21swapper/211:25:132
232722199290,27cyclictest0-21swapper/211:15:132
232722199290,27cyclictest0-21swapper/210:15:182
232722199290,27cyclictest0-21swapper/209:20:132
4505291280,8ptp4l441ktimers/312:29:553
2327223992826,1cyclictest0-21swapper/311:15:133
2327223992825,2cyclictest0-21swapper/311:25:123
2327223992825,2cyclictest0-21swapper/311:10:193
232722399280,27cyclictest0-21swapper/310:05:133
232722399280,27cyclictest0-21swapper/310:05:123
232722399280,27cyclictest0-21swapper/307:45:133
232722399280,26cyclictest0-21swapper/312:30:033
2327221992825,2cyclictest0-21swapper/211:54:592
232722199280,26cyclictest0-21swapper/210:04:582
4505291270,7ptp4l441ktimers/308:10:003
4505291270,7ptp4l441ktimers/308:10:003
4505291270,6ptp4l441ktimers/310:05:013
4505291270,4ptp4l441ktimers/309:25:003
2327223992725,1cyclictest0-21swapper/311:32:593
2327223992725,1cyclictest0-21swapper/309:45:133
2327223992725,1cyclictest0-21swapper/308:00:163
2327223992725,1cyclictest0-21swapper/307:40:593
2327223992725,1cyclictest0-21swapper/307:40:593
232722399270,26cyclictest0-21swapper/308:45:023
2327221992724,2cyclictest0-21swapper/212:15:142
2327221992724,2cyclictest0-21swapper/209:54:592
2327221992723,2cyclictest0-21swapper/211:02:582
232722199270,25cyclictest0-21swapper/208:33:592
232722199270,24cyclictest0-21swapper/209:42:592
232722199270,24cyclictest0-21swapper/208:09:552
232722199270,24cyclictest0-21swapper/208:09:552
232722199270,1cyclictest0-21swapper/211:48:002
2327215992723,3cyclictest0-21swapper/007:10:010
2327215992723,3cyclictest0-21swapper/007:10:010
2327215992722,4cyclictest2415782-21phc2sys11:45:230
4505291260,6ptp4l441ktimers/312:00:013
4505291260,6ptp4l441ktimers/310:50:013
4505291260,6ptp4l441ktimers/310:30:013
4505291260,6ptp4l441ktimers/309:45:003
4505291260,5ptp4l441ktimers/308:00:013
4505291260,4ptp4l441ktimers/311:05:003
2327223992624,2cyclictest0-21swapper/310:14:553
2327223992624,1cyclictest0-21swapper/307:35:023
2327221992623,2cyclictest0-21swapper/212:38:592
2327221992623,2cyclictest0-21swapper/208:25:592
232722199260,2cyclictest0-21swapper/212:20:592
232722199260,24cyclictest0-21swapper/210:58:002
201699260,14rtkit-daemon2325817-21apt-get07:05:091
201699260,14rtkit-daemon2325817-21apt-get07:05:091
4505291250,6ptp4l61790irq/134-eno1-TxRx-012:39:553
4505291250,6ptp4l61790irq/134-eno1-TxRx-008:20:003
4505291250,5ptp4l61790irq/134-eno1-TxRx-011:09:563
4505291250,5ptp4l61790irq/134-eno1-TxRx-009:55:013
4505291250,5ptp4l61790irq/134-eno1-TxRx-007:19:563
4505291250,5ptp4l61790irq/134-eno1-TxRx-007:19:553
4505291250,5ptp4l441ktimers/308:35:003
4505291250,4ptp4l441ktimers/309:59:553
2327223992523,1cyclictest0-21swapper/308:56:593
2327221992522,2cyclictest0-21swapper/209:15:592
2327221992522,2cyclictest0-21swapper/208:21:002
4505291240,5ptp4l441ktimers/312:25:003
4505291240,4ptp4l61790irq/134-eno1-TxRx-011:55:003
4505291240,4ptp4l61790irq/134-eno1-TxRx-010:45:003
4505291240,4ptp4l441ktimers/308:24:553
2327221992421,2cyclictest0-21swapper/209:29:002
2327221992420,2cyclictest0-21swapper/210:39:002
2327221992320,2cyclictest0-21swapper/210:49:582
2327221992320,2cyclictest0-21swapper/208:55:122
2327221992320,2cyclictest0-21swapper/207:25:132
2327221992118,2cyclictest0-21swapper/212:30:002
2327221991916,2cyclictest0-21swapper/207:58:592
2327221991816,1cyclictest0-21swapper/209:10:012
23272219996,2cyclictest0-21swapper/211:31:212
23272219930,2cyclictest0-21swapper/207:10:012
23272219930,2cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional