You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 08:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackeslot2.osadl.org (updated Fri Nov 22, 2024 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9055912430,2sleep30-21swapper/300:20:203
2500991290,3ptp4l186990irq/129-enp1s0-TxRx-023:28:483
2500991290,1ptp4l0-21swapper/320:50:263
435244992718,5cyclictest870501-21atop00:00:001
435244992615,3cyclictest1696-1atopacctd23:54:081
2500991260,5ptp4l186990irq/129-enp1s0-TxRx-000:30:003
2500991260,2ptp4l0-21swapper/321:32:153
2500991250,5ptp4l186990irq/129-enp1s0-TxRx-020:20:123
2500991250,5ptp4l186990irq/129-enp1s0-TxRx-000:35:263
2500991250,4ptp4l186990irq/129-enp1s0-TxRx-022:12:093
2500991250,3ptp4l186990irq/129-enp1s0-TxRx-023:06:283
25010912418,2phc2sys0-21swapper/300:34:123
2500991240,9ptp4l186990irq/129-enp1s0-TxRx-021:10:123
2500991240,5ptp4l186990irq/129-enp1s0-TxRx-021:20:283
2500991240,4ptp4l186990irq/129-enp1s0-TxRx-023:50:483
2500991240,4ptp4l186990irq/129-enp1s0-TxRx-022:57:163
2500991240,3ptp4l186990irq/129-enp1s0-TxRx-021:19:063
2500991240,2ptp4l0-21swapper/322:30:463
435245992320,1cyclictest0-21swapper/223:40:022
435245992316,5cyclictest0-21swapper/223:40:122
25010912314,1phc2sys0-21swapper/323:47:463
2500991230,6ptp4l186990irq/129-enp1s0-TxRx-021:41:013
2500991230,6ptp4l186990irq/129-enp1s0-TxRx-021:41:003
2500991230,5ptp4l186990irq/129-enp1s0-TxRx-022:22:373
2500991230,5ptp4l186990irq/129-enp1s0-TxRx-020:45:263
2500991230,4ptp4l186990irq/129-enp1s0-TxRx-020:10:153
2500991230,3ptp4l186990irq/129-enp1s0-TxRx-023:37:493
2500991230,3ptp4l186990irq/129-enp1s0-TxRx-022:52:143
2500991230,2ptp4l0-21swapper/319:05:113
2500991230,2ptp4l0-21swapper/300:10:013
2500991230,12ptp4l186990irq/129-enp1s0-TxRx-021:48:533
435244992214,1cyclictest191rcu_preempt21:44:271
435244992214,1cyclictest191rcu_preempt21:44:271
43524399224,13cyclictest191rcu_preempt22:10:240
43524399220,11cyclictest714208-21grep22:25:190
25010912215,2phc2sys0-21swapper/323:30:233
25010912215,2phc2sys0-21swapper/321:05:143
2500991220,8ptp4l186990irq/129-enp1s0-TxRx-000:12:353
2500991220,5ptp4l186990irq/129-enp1s0-TxRx-023:16:273
2500991220,5ptp4l186990irq/129-enp1s0-TxRx-023:11:153
2500991220,5ptp4l186990irq/129-enp1s0-TxRx-023:02:093
2500991220,5ptp4l186990irq/129-enp1s0-TxRx-022:37:113
2500991220,5ptp4l186990irq/129-enp1s0-TxRx-022:26:123
2500991220,5ptp4l186990irq/129-enp1s0-TxRx-019:35:003
2500991220,4ptp4l186990irq/129-enp1s0-TxRx-023:55:573
2500991220,4ptp4l186990irq/129-enp1s0-TxRx-023:24:063
2500991220,4ptp4l186990irq/129-enp1s0-TxRx-021:56:083
2500991220,4ptp4l186990irq/129-enp1s0-TxRx-021:52:023
2500991220,4ptp4l186990irq/129-enp1s0-TxRx-021:25:303
2500991220,4ptp4l186990irq/129-enp1s0-TxRx-000:17:113
2500991220,3ptp4l186990irq/129-enp1s0-TxRx-022:16:053
2500991220,12ptp4l186990irq/129-enp1s0-TxRx-022:01:233
2500991220,12ptp4l186990irq/129-enp1s0-TxRx-020:55:203
92812322117,2sleep20-21swapper/200:30:532
91401822117,2sleep20-21swapper/200:25:162
90262422117,2sleep20-21swapper/200:16:222
89542822117,2sleep20-21swapper/200:13:412
88617822117,2sleep20-21swapper/200:07:362
87698522117,2sleep20-21swapper/200:01:272
86850322117,2sleep20-21swapper/223:56:462
85530822117,2sleep20-21swapper/223:50:172
85293022117,2sleep20-21swapper/223:49:372
83093222117,2sleep20-21swapper/223:31:492
82379222117,2sleep20-21swapper/223:29:452
79687822117,2sleep20-21swapper/223:12:492
74406222117,2sleep20-21swapper/222:41:112
72647922117,2sleep20-21swapper/222:30:572
68499322117,2sleep20-21swapper/222:08:292
67646622117,2sleep20-21swapper/222:03:322
65871622117,2sleep20-21swapper/221:52:292
64242322117,2sleep20-21swapper/221:44:002
64242322117,2sleep20-21swapper/221:44:002
63316422117,2sleep20-21swapper/221:37:552
60608422117,2sleep20-21swapper/221:20:432
59870522117,2sleep20-21swapper/221:17:512
58903622117,2sleep20-21swapper/221:11:012
51558322117,2sleep20-21swapper/220:10:262
50535222117,2sleep20-21swapper/220:05:172
435244992113,1cyclictest191rcu_preempt22:31:181
43524399214,7cyclictest577222-21aten2.4-expect21:05:130
435243992113,6cyclictest0-21swapper/022:10:010
43490122117,3sleep20-21swapper/219:07:592
25010912115,2phc2sys0-21swapper/323:44:103
25010912115,2phc2sys0-21swapper/319:56:013
25010912115,2phc2sys0-21swapper/319:45:243
2500991210,4ptp4l186990irq/129-enp1s0-TxRx-022:45:533
2500991210,4ptp4l186990irq/129-enp1s0-TxRx-022:42:013
2500991210,4ptp4l186990irq/129-enp1s0-TxRx-020:25:153
2500991210,4ptp4l186990irq/129-enp1s0-TxRx-019:10:193
2500991210,2ptp4l0-21swapper/319:50:123
924598200,4rtkit-daemon435241-21cyclictest20:48:111
924598200,4rtkit-daemon435241-21cyclictest20:18:481
81465622017,2sleep20-21swapper/223:23:472
78715622017,2sleep20-21swapper/223:06:162
71091722017,2sleep20-21swapper/222:23:442
69323122017,2sleep20-21swapper/222:12:532
66655922017,2sleep20-21swapper/221:56:142
64914222017,2sleep20-21swapper/221:45:442
62359522017,2sleep20-21swapper/221:31:052
61540622017,2sleep20-21swapper/221:27:082
58243022017,2sleep20-21swapper/221:05:522
57655522017,2sleep20-21swapper/221:03:002
57031722017,2sleep20-21swapper/220:55:262
56438922017,2sleep20-21swapper/220:51:542
55826222017,2sleep20-21swapper/220:45:282
55252122017,2sleep20-21swapper/220:44:202
54618322017,2sleep20-21swapper/220:35:472
53434522017,2sleep20-21swapper/220:29:132
52807122017,2sleep20-21swapper/220:20:562
52203422017,2sleep20-21swapper/220:16:192
50393522017,2sleep20-21swapper/220:01:362
49187922017,2sleep20-21swapper/219:51:542
47365922017,2sleep20-21swapper/219:35:562
46763222017,2sleep20-21swapper/219:31:072
46173822017,2sleep20-21swapper/219:28:192
45549122017,2sleep20-21swapper/219:23:452
43524499208,3cyclictest0-21swapper/122:15:261
435244992017,2cyclictest608738-21/usr/sbin/munin21:25:011
435244992011,2cyclictest191rcu_preempt00:00:211
43524399209,6cyclictest0-21swapper/021:15:010
435243992017,2cyclictest441630-21cpu19:15:020
435243992014,4cyclictest1695-21dbus-daemon19:25:000
435243992013,1cyclictest0-21swapper/021:47:440
435243992011,5cyclictest669221-21apt-get22:00:110
25010912015,3phc2sys0-21swapper/321:35:513
25010912015,3phc2sys0-21swapper/320:37:283
25010912015,3phc2sys0-21swapper/320:00:083
25010912015,3phc2sys0-21swapper/319:17:213
2500991200,4ptp4l561ktimers/319:35:293
2500991200,4ptp4l186990irq/129-enp1s0-TxRx-020:44:493
2500991200,4ptp4l186990irq/129-enp1s0-TxRx-020:19:493
2500991200,4ptp4l186990irq/129-enp1s0-TxRx-020:05:503
2500991200,4ptp4l186990irq/129-enp1s0-TxRx-019:40:373
2500991200,4ptp4l186990irq/129-enp1s0-TxRx-019:22:293
2500991200,3ptp4l186990irq/129-enp1s0-TxRx-022:06:013
87645621916,1sleep30-21swapper/300:00:333
435245991917,1cyclictest0-21swapper/222:50:012
435244991916,2cyclictest824511-21latency_hist23:30:021
435244991916,2cyclictest583030-21wc21:10:011
435244991916,2cyclictest1-21systemd22:55:011
435244991916,2cyclictest0-21swapper/100:25:011
43524399199,5cyclictest0-21swapper/000:20:220
43524399198,6cyclictest0-21swapper/023:20:220
43524399198,2cyclictest611820-21kthreadcore21:25:210
435243991916,2cyclictest638890-21kthreadcore21:40:190
435243991916,2cyclictest638890-21kthreadcore21:40:190
435243991910,7cyclictest0-21swapper/021:55:180
43524399190,14cyclictest0-21swapper/000:37:370
25010911915,2phc2sys0-21swapper/319:25:383
2500991190,4ptp4l186990irq/129-enp1s0-TxRx-021:02:533
2500991190,4ptp4l186990irq/129-enp1s0-TxRx-020:34:313
435245991816,1cyclictest0-21swapper/222:40:012
435245991815,1cyclictest0-21swapper/219:15:022
43524499182,6cyclictest36-21ksoftirqd/121:47:441
435244991815,2cyclictest1726-21systemd-logind00:15:011
435244991815,2cyclictest0-21swapper/122:25:001
435244991814,3cyclictest904734-21sendmail00:20:001
435244991811,5cyclictest746313-21cat22:45:001
43524399189,5cyclictest879103-21munin-run00:05:000
43524399189,4cyclictest178850irq/121-eno123:31:080
43524399188,5cyclictest0-21swapper/022:55:230
435243991815,2cyclictest870498-21munin-run00:00:000
435243991812,1cyclictest601rcuop/319:15:200
435244991714,2cyclictest884780-21timerwakeupswit00:05:281
435244991714,2cyclictest841862-21apt-get23:40:121
435244991714,2cyclictest781141-21acpi23:05:111
435244991714,2cyclictest685930-21cut22:10:011
435244991714,2cyclictest435241-21cyclictest21:30:221
435244991714,2cyclictest1731-21in:imuxsock21:30:011
435244991713,3cyclictest1726-21systemd-logind23:05:011
435244991713,2cyclictest655132-21kthreadcore21:50:221
435244991712,3cyclictest807322-21aten_repower_fr23:20:141
43524499171,15cyclictest480421-21apt-get19:45:131
43524399176,1cyclictest191rcu_preempt22:42:020
43524399173,9cyclictest746717-21acpi22:45:110
43524399173,5cyclictest171ktimers/000:16:500
43524399173,12cyclictest823981-21munin-plugin-st23:30:000
435243991714,2cyclictest760311-21sh22:50:290
435243991714,2cyclictest652856-21gltestperf21:50:180
435243991714,2cyclictest504300-21latency_hist20:05:010
435243991714,2cyclictest0-21swapper/023:10:010
435243991713,3cyclictest1700-21gmain19:10:020
435243991713,3cyclictest0-21swapper/022:35:000
435243991711,5cyclictest595805-21runrttasks21:15:210
435243991711,5cyclictest582763-21diskmemload00:32:020
435243991711,2cyclictest626035-21kworker/0:2+events_freezable21:40:010
435243991711,1cyclictest601rcuop/322:18:390
435243991710,2cyclictest171ktimers/023:15:120
43524399170,1cyclictest241rcuc/023:40:160
43524599161,8cyclictest0-21swapper/223:20:012
435245991613,1cyclictest0-21swapper/223:00:022
43524499169,1cyclictest191rcu_preempt22:28:351
43524499167,1cyclictest501rcuop/222:01:111
435244991614,2cyclictest825344-21kworker/u16:2+events_freezable_pwr_efficient23:38:021
435244991614,2cyclictest474226-21kworker/u16:2+events_freezable_pwr_efficient21:15:151
435244991613,2cyclictest931671-21gltestperf00:35:161
435244991613,2cyclictest802325-21kthreadcore23:15:221
435244991613,2cyclictest790109-21aten2.4-expect23:10:101
435244991613,2cyclictest540695-21systemd-run20:35:021
435244991613,2cyclictest477355-21kthreadcore19:40:221
435244991613,2cyclictest463541-21kthreadcore19:30:171
435244991613,2cyclictest1726-21systemd-logind22:15:011
435244991613,1cyclictest191rcu_preempt22:46:461
435244991612,3cyclictest916678-21kthreadcore00:25:211
435244991612,3cyclictest523987-21kthreadcore20:20:191
435244991612,3cyclictest1726-21systemd-logind20:55:011
43524399169,6cyclictest468356-21aten2.4-expect19:35:110
43524399164,8cyclictest178850irq/121-eno123:39:390
43524399164,7cyclictest522763-21aten2.4-expect20:20:120
43524399164,6cyclictest456362-21aten2.4-expect19:25:130
43524399163,7cyclictest178850irq/121-eno121:32:350
43524399162,2cyclictest0-21swapper/023:03:270
435243991614,1cyclictest178850irq/121-eno100:07:540
435243991613,2cyclictest708634-21mailstats22:20:240
435243991613,2cyclictest492597-21aten_repower_po19:55:120
435243991611,4cyclictest1696-1atopacctd20:55:160
43524399161,10cyclictest528424-21perf20:29:550
435243991610,2cyclictest178850irq/121-eno123:10:160
435245991513,1cyclictest0-21swapper/220:30:192
435245991513,1cyclictest0-21swapper/200:21:582
435245991512,1cyclictest0-21swapper/222:50:072
43524499159,4cyclictest191rcu_preempt00:30:511
43524499158,1cyclictest501rcuop/222:59:381
43524499158,1cyclictest191rcu_preempt23:30:161
43524499158,1cyclictest191rcu_preempt21:59:111
43524499156,2cyclictest491rcuog/221:14:111
435244991512,2cyclictest462180-21latency_hist19:30:021
435244991512,2cyclictest443044-21kthreadcore19:15:031
435244991512,1cyclictest0-21swapper/119:15:011
435244991511,3cyclictest628370-21kthreadcore21:35:161
435244991511,3cyclictest548694-21kthreadcore20:40:181
435244991511,2cyclictest0-21swapper/122:40:001
435244991510,3cyclictest36-21ksoftirqd/123:50:011
43524499150,3cyclictest0-21swapper/120:37:401
43524499150,14cyclictest498186-21wc20:00:011
43524399159,2cyclictest582763-21diskmemload21:20:150
43524399157,7cyclictest0-21swapper/000:10:300
43524399157,4cyclictest729340-21kworker/u16:0+rpciod22:38:510
43524399155,1cyclictest191rcu_preempt20:05:120
43524399153,7cyclictest462381-21aten2.4-expect19:30:120
43524399153,6cyclictest480480-21aten2.4-expect19:45:130
435243991512,2cyclictest914035-21tr00:25:150
435243991512,2cyclictest857327-21kthreadcore23:50:170
435243991512,2cyclictest480120-21sh19:45:000
435243991512,2cyclictest0-21swapper/020:45:010
435245991412,1cyclictest0-21swapper/223:02:312
435245991412,1cyclictest0-21swapper/219:45:152
435245991412,1cyclictest0-21swapper/200:35:202
435245991411,2cyclictest442256-21taskset19:15:042
435245991411,1cyclictest0-21swapper/222:25:192
435245991411,1cyclictest0-21swapper/219:40:222
435245991410,2cyclictest0-21swapper/222:15:192
43524499143,2cyclictest576768-21systemctl21:05:001
435244991412,2cyclictest461701-21kworker/u16:0+events_freezable_pwr_efficient19:36:241
435244991412,1cyclictest529609-21grep20:25:161
435244991412,1cyclictest501267-21kthreadcore20:00:221
435244991411,2cyclictest511233-21fschecks_time20:10:171
435244991411,2cyclictest507508-21kthreadcore20:05:191
435244991411,2cyclictest487065-21fschecks_time19:50:151
435244991411,2cyclictest0-21swapper/119:25:001
435244991410,3cyclictest569255-21kthreadcore20:55:171
43524399148,1cyclictest191rcu_preempt23:45:280
43524399146,4cyclictest552989-21aten2.4-expect20:45:110
43524399143,4cyclictest571078-21head21:00:120
43524399141,7cyclictest474056-21kworker/0:1+rcu_gp20:55:000
43524399141,3cyclictest541457-21grep20:35:140
435243991411,2cyclictest534904-21aten_repower_vo20:30:130
435243991411,2cyclictest517778-21irqrtprio20:15:190
435243991411,2cyclictest515492-21ptp4l-jitter20:10:240
43524599139,2cyclictest0-21swapper/219:55:192
43524399135,4cyclictest486528-21aten2.4-expect19:50:120
43524499129,2cyclictest1695-21dbus-daemon19:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional