You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-21 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Nov 21, 2024 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21593992422,1cyclictest609-21polkitd08:15:003
2158899240,23cyclictest0-21swapper/208:15:002
21586992421,2cyclictest313-21systemd-journal10:20:001
2158699240,23cyclictest17717-21cron09:15:001
21593992320,2cyclictest15346-21perf09:10:013
2159399230,22cyclictest5222-21cron10:00:013
2159399230,22cyclictest12937-21awk09:05:003
2159399230,22cyclictest0-21swapper/312:10:003
21588992318,2cyclictest28-21ksoftirqd/211:30:002
2158899230,22cyclictest5565-21apt-get10:00:362
2158899230,0cyclictest0-21swapper/209:05:352
21583992322,1cyclictest2421-21apt-get11:00:360
2158399230,22cyclictest17743-21turbostat09:15:000
2158399230,0cyclictest0-21swapper/009:50:400
21593992220,1cyclictest11729-21idleruntime-cro11:20:003
2159399220,2cyclictest25605-21ls09:35:003
2159399220,21cyclictest4819-21apt-get11:05:383
2159399220,21cyclictest1641-21cron07:35:003
2159399220,21cyclictest13714-21perf08:00:003
21588992220,1cyclictest13072-21apt-get10:15:032
2158899220,21cyclictest4402-21apt-get07:40:352
2158899220,21cyclictest29443-21apt-get07:25:352
2158899220,21cyclictest29158-21munin-run07:25:002
2158899220,21cyclictest28088-21turbostat08:30:002
2158899220,21cyclictest2132-21munin-run11:00:002
2158899220,21cyclictest1-21systemd12:05:002
2158899220,21cyclictest0-21swapper/208:35:372
2158899220,21cyclictest0-21swapper/208:15:382
2158899220,1cyclictest0-21swapper/209:30:002
21586992219,0cyclictest151rcu_preempt11:55:001
2158699220,22cyclictest0-21swapper/109:05:351
2158699220,21cyclictest8885-21apt07:50:001
2158699220,21cyclictest4076-21apt-get12:10:401
2158699220,21cyclictest31398-21munin-run12:00:001
2158699220,21cyclictest30465-21cron09:45:001
2158699220,21cyclictest30465-21cron09:45:001
2158699220,21cyclictest20461-21apt-get09:20:361
2158699220,21cyclictest12446-21apt-get10:15:001
21583992220,1cyclictest6506-21ls07:45:000
21583992219,2cyclictest11705-21turbostat.cron11:20:000
2158399220,22cyclictest0-21swapper/008:55:350
2158399220,21cyclictest593-21dbus-daemon12:20:000
2158399220,21cyclictest16087-21idleruntime-cro08:05:000
2158399220,21cyclictest1560-21latency_hist07:35:000
2158399220,21cyclictest1244-21apt-get08:40:350
2158399220,21cyclictest10987-21apt-get12:25:020
2158399220,21cyclictest0-21swapper/012:05:360
2158399220,21cyclictest0-21swapper/009:40:350
2158399220,21cyclictest0-21swapper/009:40:350
2158399220,21cyclictest0-21swapper/007:20:350
2158399220,1cyclictest0-21swapper/011:55:000
21593992121,0cyclictest0-21swapper/310:10:363
21593992119,1cyclictest1339-21cron12:05:003
21593992119,1cyclictest0-21swapper/307:25:003
2159399210,20cyclictest0-21swapper/311:35:383
2159399210,20cyclictest0-21swapper/310:30:353
2159399210,20cyclictest0-21swapper/310:05:363
2159399210,20cyclictest0-21swapper/308:35:373
2159399210,20cyclictest0-21swapper/307:50:353
2159399210,20cyclictest0-21swapper/307:15:343
2159399210,0cyclictest0-21swapper/312:30:413
2159399210,0cyclictest0-21swapper/308:05:373
21588992120,1cyclictest3650-21apt-get08:45:352
21588992120,1cyclictest0-21swapper/210:40:372
21588992119,1cyclictest9608-21apt-get11:15:372
2158899210,20cyclictest0-21swapper/212:35:352
2158899210,20cyclictest0-21swapper/208:40:352
2158899210,1cyclictest6468-21apt-get12:15:382
2158899210,0cyclictest0-21swapper/212:20:382
2158899210,0cyclictest0-21swapper/209:40:352
2158899210,0cyclictest0-21swapper/209:40:352
2158899210,0cyclictest0-21swapper/209:35:362
2158899210,0cyclictest0-21swapper/208:55:352
2158899210,0cyclictest0-21swapper/207:50:352
21586992121,0cyclictest20355-21apt-get10:30:361
21586992121,0cyclictest0-21swapper/111:40:371
21586992120,1cyclictest14007-21apt-get08:00:391
21586992119,1cyclictest20135-21idleruntime-cro09:20:001
2158699210,20cyclictest0-21swapper/107:25:001
2158699210,0cyclictest8443-21apt-get08:55:351
2158699210,0cyclictest0-21swapper/110:55:361
2158699210,0cyclictest0-21swapper/110:50:361
21583992121,0cyclictest29443-21apt-get07:25:350
21583992121,0cyclictest25175-21apt-get10:40:370
21583992121,0cyclictest0-21swapper/011:45:370
21583992121,0cyclictest0-21swapper/008:50:000
21583992121,0cyclictest0-21swapper/008:35:360
21583992120,1cyclictest30017-21apt-get10:50:350
21583992120,0cyclictest0-21swapper/010:25:350
21583992119,1cyclictest10067-21latency_hist10:10:000
21583992119,1cyclictest0-21swapper/010:15:410
2158399210,21cyclictest0-21swapper/007:15:340
2158399210,20cyclictest0-21swapper/008:10:360
2158399210,1cyclictest6171-21cat12:15:000
2158399210,1cyclictest32414-21apt-get10:55:360
2158399210,0cyclictest0-21swapper/008:20:400
21593992020,0cyclictest0-21swapper/310:15:403
21593992019,1cyclictest2137-21awk11:00:003
21593992018,2cyclictest34-21ksoftirqd/311:50:443
21593992018,1cyclictest34-21ksoftirqd/309:50:003
2159399200,19cyclictest13775-21cat12:30:013
2159399200,0cyclictest0-21swapper/311:25:363
2159399200,0cyclictest0-21swapper/310:50:353
2159399200,0cyclictest0-21swapper/308:30:363
2159399200,0cyclictest0-21swapper/308:25:353
21588992020,0cyclictest0-21swapper/207:10:352
21588992020,0cyclictest0-21swapper/207:10:352
21588992019,1cyclictest28-21ksoftirqd/211:00:362
21588992019,1cyclictest28-21ksoftirqd/209:50:402
21588992019,1cyclictest28-21ksoftirqd/209:50:012
21588992018,1cyclictest28-21ksoftirqd/210:25:352
21588992018,1cyclictest0-21swapper/212:25:022
2158899200,20cyclictest0-21swapper/212:30:412
2158899200,20cyclictest0-21swapper/210:45:362
2158899200,19cyclictest22034-21apt-get11:40:362
2158899200,0cyclictest11585-21apt-get07:55:352
2158899200,0cyclictest0-21swapper/211:50:442
2158899200,0cyclictest0-21swapper/209:00:352
2158699202,17cyclictest22-21ksoftirqd/111:05:381
21586992020,0cyclictest31690-21apt-get12:00:371
21586992020,0cyclictest27569-21apt-get10:45:361
21586992020,0cyclictest21187-21apt-get08:15:371
21586992020,0cyclictest14077-21apt-get12:30:411
21586992020,0cyclictest0-21swapper/110:40:371
21586992020,0cyclictest0-21swapper/108:40:351
21586992020,0cyclictest0-21swapper/107:40:351
21586992019,1cyclictest9189-21apt-get07:50:351
21586992018,2cyclictest23616-21apt-get08:20:401
21586992018,1cyclictest30561-21apt-get08:35:011
2158699200,20cyclictest21840-21apt-get07:10:351
2158699200,20cyclictest21840-21apt-get07:10:351
2158699200,0cyclictest0-21swapper/109:00:351
21583992020,0cyclictest23939-21perf07:15:000
21583992020,0cyclictest23939-21perf07:15:000
21583992020,0cyclictest0-21swapper/012:35:350
21583992020,0cyclictest0-21swapper/011:40:370
21583992020,0cyclictest0-21swapper/011:30:370
21583992020,0cyclictest0-21swapper/011:05:370
21583992020,0cyclictest0-21swapper/010:00:360
21583992020,0cyclictest0-21swapper/009:15:380
21583992019,1cyclictest26005-21apt-get08:25:350
21583992019,1cyclictest22758-21apt-get10:35:350
2158399200,20cyclictest0-21swapper/011:20:020
2158399200,0cyclictest0-21swapper/012:00:360
2158399200,0cyclictest0-21swapper/011:25:360
2158399200,0cyclictest0-21swapper/009:45:360
21593991919,0cyclictest0-21swapper/310:45:363
21593991919,0cyclictest0-21swapper/310:00:363
21593991919,0cyclictest0-21swapper/309:40:353
21593991919,0cyclictest0-21swapper/309:40:353
21593991919,0cyclictest0-21swapper/309:35:363
21593991919,0cyclictest0-21swapper/308:55:353
21593991919,0cyclictest0-21swapper/308:00:393
21593991919,0cyclictest0-21swapper/307:40:353
21593991918,1cyclictest34-21ksoftirqd/311:25:003
21593991918,1cyclictest0-21swapper/307:25:353
21588991919,0cyclictest0-21swapper/211:55:362
21588991919,0cyclictest0-21swapper/211:20:372
21588991919,0cyclictest0-21swapper/211:05:382
21588991919,0cyclictest0-21swapper/210:30:352
21588991919,0cyclictest0-21swapper/208:05:372
21588991918,1cyclictest28-21ksoftirqd/211:45:372
21588991918,1cyclictest24853-21ls10:40:002
21588991917,1cyclictest1-21systemd08:55:002
21588991917,1cyclictest1-21systemd08:55:002
21586991919,0cyclictest730-21apt-get09:50:401
21586991919,0cyclictest16466-21apt-get12:35:361
21586991919,0cyclictest0-21swapper/110:00:361
21586991919,0cyclictest0-21swapper/109:55:371
21586991919,0cyclictest0-21swapper/108:45:351
21586991919,0cyclictest0-21swapper/107:55:351
21586991918,0cyclictest0-21swapper/108:30:361
21586991918,0cyclictest0-21swapper/107:25:351
21583991919,0cyclictest0-21swapper/012:00:000
21583991919,0cyclictest0-21swapper/009:05:350
21583991919,0cyclictest0-21swapper/007:35:340
21593991818,0cyclictest0-21swapper/312:35:353
21593991818,0cyclictest0-21swapper/312:20:383
21593991818,0cyclictest0-21swapper/312:15:383
21593991818,0cyclictest0-21swapper/308:40:353
21593991818,0cyclictest0-21swapper/308:15:383
21593991816,2cyclictest34-21ksoftirqd/312:15:013
21593991816,1cyclictest0-21swapper/310:40:373
21588991818,0cyclictest0-21swapper/212:10:412
21588991817,1cyclictest19611-21apt-get11:35:382
21586991818,0cyclictest7220-21apt-get11:10:431
21586991818,0cyclictest0-21swapper/108:10:361
21586991818,0cyclictest0-21swapper/107:35:341
2158699180,17cyclictest16398-21apt-get08:05:371
21583991818,0cyclictest0-21swapper/011:10:420
21583991818,0cyclictest0-21swapper/007:55:350
21583991817,1cyclictest25902-21apt-get09:35:360
21593991717,0cyclictest24433-21apt-get11:45:373
21593991717,0cyclictest0-21swapper/311:10:423
21593991717,0cyclictest0-21swapper/309:10:353
21593991716,1cyclictest34-21ksoftirqd/309:50:403
21593991715,1cyclictest612-21gdbus10:40:003
21588991717,0cyclictest0-21swapper/210:05:362
21588991717,0cyclictest0-21swapper/209:10:342
21588991717,0cyclictest0-21swapper/207:15:342
21588991716,1cyclictest28-21ksoftirqd/210:00:012
21588991715,1cyclictest1879-21apt-get07:35:342
2158899170,1cyclictest30552-21awk08:35:002
21586991717,0cyclictest24628-21apt-get07:15:341
21586991717,0cyclictest0-21swapper/108:50:351
21586991717,0cyclictest0-21swapper/108:50:351
2158699170,17cyclictest14816-21apt-get11:25:361
21583991717,0cyclictest0-21swapper/012:30:410
21583991717,0cyclictest0-21swapper/008:30:360
21583991717,0cyclictest0-21swapper/008:05:370
21583991717,0cyclictest0-21swapper/007:50:350
21583991715,1cyclictest3149-21apt-get09:55:360
2158399170,16cyclictest10356-21apt-get10:10:360
21593991616,0cyclictest0-21swapper/311:40:363
21593991616,0cyclictest0-21swapper/308:45:353
21593991616,0cyclictest0-21swapper/307:10:353
21593991616,0cyclictest0-21swapper/307:10:353
21593991615,1cyclictest29220-21apt-get11:55:363
21593991615,0cyclictest0-21swapper/310:25:353
21593991614,1cyclictest0-21swapper/308:50:463
21593991614,1cyclictest0-21swapper/308:50:463
21593991614,1cyclictest0-21swapper/307:37:413
21588991616,0cyclictest6795-21apt-get07:45:352
21588991616,0cyclictest0-21swapper/210:20:372
21588991615,1cyclictest28-21ksoftirqd/209:20:422
21586991616,0cyclictest0-21swapper/112:05:361
21586991616,0cyclictest0-21swapper/110:20:361
21586991616,0cyclictest0-21swapper/110:05:351
21586991614,1cyclictest4515-21dump-pmu-power11:05:001
2158699160,1cyclictest14518-21perf11:25:001
2158699160,15cyclictest32402-21tune2fs07:30:381
2158699160,15cyclictest19847-21sshd10:27:441
21583991616,0cyclictest0-21swapper/008:50:360
21583991616,0cyclictest0-21swapper/008:50:360
21583991615,1cyclictest0-21swapper/011:35:380
21583991615,0cyclictest6795-21apt-get07:45:350
2158399160,1cyclictest0-21swapper/010:20:420
21593991515,0cyclictest34-21ksoftirqd/311:00:363
2159399150,1cyclictest0-21swapper/309:20:003
2159399150,14cyclictest0-21swapper/310:24:133
21588991515,0cyclictest0-21swapper/208:20:402
21588991513,1cyclictest7967-21modprobe11:10:462
21588991513,1cyclictest18011-21/usr/sbin/munin09:15:382
21588991513,1cyclictest0-21swapper/207:30:432
21588991512,2cyclictest16103-21latency_hist08:05:002
2158899150,8cyclictest271rcuc/212:05:362
21586991515,0cyclictest17214-21apt-get11:30:371
21586991515,0cyclictest0-21swapper/112:15:381
21586991515,0cyclictest0-21swapper/111:45:381
21586991513,1cyclictest0-21swapper/109:30:391
2158699150,3cyclictest28098-21dump-pmu-power08:30:001
2158399158,6cyclictest7677-21munin-node08:18:470
21583991515,0cyclictest0-21swapper/010:45:360
21583991514,1cyclictest21508-21ntp_states09:20:410
21583991513,1cyclictest23216-21munin-run09:30:000
21583991513,1cyclictest0-21swapper/012:20:480
21583991513,1cyclictest0-21swapper/010:30:440
2159399148,4cyclictest151rcu_preempt09:20:383
2159399140,14cyclictest0-21swapper/311:31:333
2159399140,14cyclictest0-21swapper/308:20:093
2159399140,0cyclictest0-21swapper/309:27:303
2159399140,0cyclictest0-21swapper/307:45:373
2158899140,14cyclictest0-21swapper/211:32:212
2158899140,14cyclictest0-21swapper/210:10:222
2158899140,0cyclictest0-21swapper/210:50:072
2158899140,0cyclictest0-21swapper/209:32:122
21586991414,0cyclictest0-21swapper/110:36:571
21586991414,0cyclictest0-21swapper/109:45:071
2158699140,14cyclictest0-21swapper/112:26:411
2158699140,14cyclictest0-21swapper/111:17:581
2158699140,14cyclictest0-21swapper/109:26:261
21583991414,0cyclictest0-21swapper/009:00:350
2158399140,0cyclictest0-21swapper/009:30:380
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional