You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Sat Jul 27, 2024 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
202942207192,10sleep10-21swapper/119:09:121
201762185167,13sleep00-21swapper/019:07:380
200602184169,11sleep30-21swapper/319:06:093
201162117101,12sleep20-21swapper/219:06:522
172942430,0sleep20-21swapper/223:50:562
132822420,0sleep00-21swapper/021:55:440
140152400,0sleep00-21swapper/023:17:030
4522390,0sleep30-21swapper/322:21:513
5342100,1sleep1201rcuc/122:21:521
2051699100,2cyclictest0-21swapper/023:00:120
29836290,0sleep10-21swapper/121:31:591
205289993,5cyclictest14764-21diskstats21:05:143
205289991,1cyclictest21129-21awk00:10:003
205289990,2cyclictest0-21swapper/323:00:123
205289990,1cyclictest21169-21df_inode19:10:143
205289982,1cyclictest16151-21tail21:25:213
205289982,1cyclictest15998-21ssh23:18:193
205289981,1cyclictest0-21swapper/319:55:113
205289980,1cyclictest29381-21ssh23:57:183
205289980,1cyclictest28013-21basename22:20:013
205289980,1cyclictest26777-21ssh23:40:093
205289980,1cyclictest12431-21ssh21:40:013
205169982,1cyclictest20719-21ssh22:48:190
205169982,1cyclictest19316-21grep22:15:140
205169981,3cyclictest16145-21idleruntime-cro23:35:000
205169981,1cyclictest0-21swapper/019:55:110
205169981,1cyclictest0-21swapper/019:10:100
205169980,4cyclictest13782-21munin-run23:50:000
205169980,1cyclictest18523-21apt00:25:000
205289972,1cyclictest13317-21ssh21:55:463
205289971,1cyclictest4868-21expr23:45:113
205289971,1cyclictest3479-21grep00:00:223
205289971,1cyclictest32559-21if_eth120:35:153
205289971,1cyclictest31168-21cat00:15:003
205289971,1cyclictest30997-21snmpd20:34:003
205289971,1cyclictest30997-21snmpd20:18:243
205289971,1cyclictest25112-21ntpq19:15:163
205289971,1cyclictest12478-21cat21:00:173
205289971,0cyclictest0-21swapper/321:50:153
205289970,1cyclictest9943-21cut00:20:123
205289970,1cyclictest8939-21munin-run19:50:003
205289970,1cyclictest8386-21users22:25:303
205289970,1cyclictest7415-21hddtemp_smartct00:35:133
205289970,1cyclictest7181-21ls23:30:153
205289970,1cyclictest4436-21df22:40:123
205289970,1cyclictest3914-21munin-run20:45:003
205289970,1cyclictest32338-21hddtemp_smartct23:10:143
205289970,1cyclictest30043-21cut23:25:143
205289970,1cyclictest29578-21fw_packets19:25:153
205289970,1cyclictest29559-21diskstats00:30:113
205289970,1cyclictest2726-21ssh21:35:113
205289970,1cyclictest27057-21ntp_states21:30:373
205289970,1cyclictest26889-21df19:20:143
205289970,1cyclictest25341-21ntp_states22:50:183
205289970,1cyclictest24098-21memory21:45:163
205289970,1cyclictest2337-21perl22:55:163
205289970,1cyclictest22037-21df_inode23:05:123
205289970,1cyclictest220-21systemd-journal20:55:003
205289970,1cyclictest220-21systemd-journal20:05:003
205289970,1cyclictest220-21systemd-journal00:20:003
205289970,1cyclictest20169-21ssh00:25:163
205289970,1cyclictest19231-21ssh23:20:123
205289970,1cyclictest17835-21expr21:10:143
205289970,1cyclictest17114-21if_eth122:30:153
205289970,1cyclictest14218-21apt23:50:093
205289970,1cyclictest0-21swapper/319:40:133
205169972,1cyclictest5444-21ssh21:52:090
205169972,1cyclictest2690-21ssh22:39:350
205169972,1cyclictest25361-21wget21:30:170
205169972,1cyclictest23528-21ssh23:38:140
205169971,1cyclictest28190-21ssh00:29:470
205169971,1cyclictest25763-21ssh21:45:510
205169970,1cyclictest11308-21latency_hist21:00:010
205289961,1cyclictest9333-21cpuspeed20:55:133
205289961,1cyclictest30929-21aten2_r5power_c22:05:143
205289960,1cyclictest8303-21rm22:10:063
205289960,1cyclictest4660-21df_inode20:45:123
205289960,1cyclictest4565-21dpkg21:20:013
205289960,1cyclictest32592-21memory19:30:193
205289960,1cyclictest27344-21tune2fs20:25:133
205289960,1cyclictest271562sleep321:15:123
205289960,1cyclictest26092-21sh22:35:043
205289960,1cyclictest24753-21/usr/sbin/munin20:20:133
205289960,1cyclictest2379-21hddtemp_smartct19:35:143
205289960,1cyclictest20696-21sh22:00:073
205289960,1cyclictest20696-21sh22:00:073
205289960,1cyclictest19765-21df20:10:123
205289960,1cyclictest17912-21irqstats23:35:143
205289960,1cyclictest17311-21df_inode20:05:133
205289960,1cyclictest13839-21ssh22:45:103
205289960,1cyclictest10456-21memory19:50:183
205259961,5cyclictest0-21swapper/221:55:422
205169961,1cyclictest30997-21snmpd00:07:290
205169961,1cyclictest16833-21ssh21:10:060
205169960,3cyclictest8927-21apt-get19:50:000
205259955,0cyclictest0-21swapper/219:30:202
205259950,5cyclictest0-21swapper/223:01:542
205259950,5cyclictest0-21swapper/220:15:112
205259950,4cyclictest18408-21ssh23:19:512
205259950,0cyclictest0-21swapper/200:26:002
205259950,0cyclictest0-21swapper/200:00:502
205179951,3cyclictest0-21swapper/122:50:081
205179951,3cyclictest0-21swapper/120:00:101
205179951,2cyclictest0-21swapper/121:10:101
205179950,5cyclictest0-21swapper/123:15:321
205179950,5cyclictest0-21swapper/120:50:361
205179950,5cyclictest0-21swapper/100:20:171
205169955,0cyclictest0-21swapper/023:20:240
205169950,5cyclictest0-21swapper/022:10:080
205169950,4cyclictest28446-21ssh22:20:100
205169950,4cyclictest0-21swapper/020:05:110
7071240,0sleep00-21swapper/021:20:210
205259944,0cyclictest31-21ksoftirqd/220:15:002
205259944,0cyclictest0-21swapper/222:01:062
205259944,0cyclictest0-21swapper/222:01:052
205259944,0cyclictest0-21swapper/221:52:212
205259944,0cyclictest0-21swapper/221:37:392
205259944,0cyclictest0-21swapper/221:27:022
205259944,0cyclictest0-21swapper/221:15:062
205259944,0cyclictest0-21swapper/219:11:332
205259943,1cyclictest4879-21df23:45:122
205259943,1cyclictest23895-21cut22:50:092
205259942,2cyclictest0-21swapper/221:47:542
205259941,3cyclictest9088-21kworker/2:123:38:462
205259941,3cyclictest11060-21nscd19:25:152
205259941,2cyclictest0-21swapper/219:40:122
205259940,4cyclictest8339-21/usr/sbin/munin22:10:162
205259940,4cyclictest24210-21/usr/sbin/munin20:20:192
205259940,4cyclictest0-21swapper/223:56:362
205259940,4cyclictest0-21swapper/222:55:122
205259940,4cyclictest0-21swapper/219:50:112
205259940,4cyclictest0-21swapper/219:20:172
205259940,4cyclictest0-21swapper/200:30:082
205259940,3cyclictest31590-21sed20:35:002
205259940,3cyclictest26917-21/usr/sbin/munin23:40:092
205259940,3cyclictest19911-21ntp_states22:15:182
205259940,3cyclictest15473-21/usr/sbin/munin20:00:182
205259940,3cyclictest14004-21cut21:05:012
205259940,3cyclictest10905-21sh23:31:412
205259940,3cyclictest0-21swapper/219:45:012
205259940,1cyclictest6603-21/usr/sbin/munin20:50:172
205259940,1cyclictest26677-21/usr/sbin/munin20:25:182
205259940,1cyclictest22571-21hddtemp_smartct00:10:162
205259940,1cyclictest16722-21cut22:30:122
205259940,0cyclictest0-21swapper/223:05:242
205179944,0cyclictest0-21swapper/123:56:251
205179944,0cyclictest0-21swapper/123:12:031
205179944,0cyclictest0-21swapper/122:57:211
205179944,0cyclictest0-21swapper/122:25:031
205179944,0cyclictest0-21swapper/121:37:461
205179944,0cyclictest0-21swapper/100:34:591
205179942,1cyclictest0-21swapper/100:25:091
205179941,2cyclictest0-21swapper/123:45:101
205179941,2cyclictest0-21swapper/123:35:011
205179941,2cyclictest0-21swapper/122:45:101
205179941,2cyclictest0-21swapper/122:40:101
205179941,2cyclictest0-21swapper/121:55:101
205179941,2cyclictest0-21swapper/121:50:121
205179941,2cyclictest0-21swapper/121:20:021
205179941,2cyclictest0-21swapper/121:15:111
205179941,2cyclictest0-21swapper/121:05:121
205179941,2cyclictest0-21swapper/120:15:111
205179941,2cyclictest0-21swapper/119:45:111
205179941,2cyclictest0-21swapper/119:40:121
205179941,2cyclictest0-21swapper/119:25:101
205179941,2cyclictest0-21swapper/100:15:101
205179941,1cyclictest0-21swapper/123:00:131
205179940,4cyclictest0-21swapper/123:05:041
205179940,0cyclictest0-21swapper/123:42:191
205179940,0cyclictest0-21swapper/123:31:411
205179940,0cyclictest0-21swapper/121:47:491
205169943,1cyclictest11694-21ssh22:27:350
205169942,1cyclictest4989-21if_eth422:40:170
205169940,4cyclictest0-21swapper/023:25:090
205169940,4cyclictest0-21swapper/023:05:100
205169940,4cyclictest0-21swapper/021:25:130
205169940,4cyclictest0-21swapper/021:00:110
205169940,4cyclictest0-21swapper/000:10:280
205169940,3cyclictest9305-21dpkg19:50:000
205169940,3cyclictest9299-21ssh00:35:470
205169940,3cyclictest7172-21ssh00:19:070
205169940,3cyclictest4178-21ssh00:00:340
205169940,3cyclictest31402-21ssh23:10:100
205169940,3cyclictest31191-21apt-config19:30:000
205169940,3cyclictest31170-21expr22:05:160
205169940,3cyclictest28124-21ssh23:56:300
205169940,3cyclictest26863-21ssh21:15:050
205169940,3cyclictest23830-21ssh22:50:080
205169940,3cyclictest2209-21perl19:35:140
205169940,3cyclictest20767-21ssh22:00:090
205169940,3cyclictest20767-21ssh22:00:090
205169940,3cyclictest16192-21ssh23:50:200
205169940,3cyclictest13399-21diskstats21:40:130
205169940,3cyclictest12271-21ssh21:40:000
205169940,3cyclictest0-21swapper/020:45:090
205259932,1cyclictest30997-21snmpd22:40:012
205259932,1cyclictest28857-21expr22:20:112
205259932,1cyclictest25981-21chmod22:35:012
205259932,1cyclictest12774-21ssh21:40:102
205259930,3cyclictest8801-21rm00:20:012
205259930,3cyclictest5871-21sh22:25:032
205259930,3cyclictest5818-21hddtemp_smartct21:20:132
205259930,3cyclictest25108-21sh21:30:112
205259930,3cyclictest17367-21cstates21:10:122
205259930,3cyclictest11521-21sh00:05:052
205259930,3cyclictest0-21swapper/223:25:092
205259930,3cyclictest0-21swapper/223:20:132
205259930,3cyclictest0-21swapper/222:45:072
205259930,3cyclictest0-21swapper/222:05:142
205259930,3cyclictest0-21swapper/221:05:012
205259930,3cyclictest0-21swapper/220:55:052
205259930,3cyclictest0-21swapper/220:45:092
205259930,3cyclictest0-21swapper/220:35:052
205259930,3cyclictest0-21swapper/220:05:052
205259930,3cyclictest0-21swapper/219:55:132
205259930,3cyclictest0-21swapper/219:35:102
205259930,3cyclictest0-21swapper/219:15:012
205259930,3cyclictest0-21swapper/200:35:092
205259930,3cyclictest0-21swapper/200:15:012
205259930,2cyclictest0-21swapper/220:40:112
205259930,1cyclictest0-21swapper/223:10:132
205179933,0cyclictest0-21swapper/119:11:141
205179933,0cyclictest0-21swapper/100:38:281
205179931,2cyclictest0-21swapper/122:30:091
205179931,1cyclictest0-21swapper/123:50:091
205179931,1cyclictest0-21swapper/123:25:091
205179931,1cyclictest0-21swapper/123:20:121
205179931,1cyclictest0-21swapper/122:35:031
205179931,1cyclictest0-21swapper/122:15:091
205179931,1cyclictest0-21swapper/122:10:081
205179931,1cyclictest0-21swapper/122:05:141
205179931,1cyclictest0-21swapper/121:40:111
205179931,1cyclictest0-21swapper/121:25:121
205179931,1cyclictest0-21swapper/121:00:111
205179931,1cyclictest0-21swapper/120:55:111
205179931,1cyclictest0-21swapper/120:40:111
205179931,1cyclictest0-21swapper/120:20:101
205179931,1cyclictest0-21swapper/120:10:101
205179931,1cyclictest0-21swapper/120:05:111
205179931,1cyclictest0-21swapper/119:35:101
205179931,1cyclictest0-21swapper/119:30:121
205179931,1cyclictest0-21swapper/119:20:121
205179931,1cyclictest0-21swapper/100:10:121
205179931,1cyclictest0-21swapper/100:05:091
205179931,1cyclictest0-21swapper/100:00:111
205179930,0cyclictest0-21swapper/120:36:381
205169932,1cyclictest31375-21apt-config19:30:000
205169931,1cyclictest0-21swapper/022:30:090
205169931,1cyclictest0-21swapper/019:20:120
205169930,3cyclictest0-21swapper/023:40:010
205169930,3cyclictest0-21swapper/022:55:120
205169930,3cyclictest0-21swapper/021:05:010
205169930,3cyclictest0-21swapper/020:50:100
205169930,3cyclictest0-21swapper/020:40:110
205169930,3cyclictest0-21swapper/020:35:100
205169930,3cyclictest0-21swapper/020:30:110
205169930,3cyclictest0-21swapper/020:25:100
205169930,3cyclictest0-21swapper/020:20:100
205169930,3cyclictest0-21swapper/020:15:110
205169930,3cyclictest0-21swapper/020:10:100
205169930,3cyclictest0-21swapper/020:00:100
205169930,3cyclictest0-21swapper/019:40:120
205169930,3cyclictest0-21swapper/019:15:100
205169930,3cyclictest0-21swapper/000:30:020
205179921,1cyclictest0-21swapper/119:50:011
205179920,1cyclictest0-21swapper/122:00:021
205179920,1cyclictest0-21swapper/122:00:021
205179920,1cyclictest0-21swapper/120:45:091
205179920,1cyclictest0-21swapper/120:30:111
205179920,1cyclictest0-21swapper/120:25:101
205179920,1cyclictest0-21swapper/119:55:121
205179920,1cyclictest0-21swapper/119:15:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional