You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-21 - 21:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Thu Nov 21, 2024 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134572108728,10830sleep01272-21kworker/0:008:21:150
68672805517,8017sleep331891-21kworker/u8:209:43:063
6838245706,4507sleep01272-21kworker/0:007:59:140
20545210918,1044sleep08363-1kworker/0:0H08:42:510
2571628556,838sleep28296-1kworker/2:2H08:57:552
14110276821,729sleep314123-21irqstats08:22:593
30697272311,694sleep28296-1kworker/2:2H09:13:012
4766249813,207sleep04891-21date11:17:400
5327243360,297sleep05357-21kernelversion09:37:590
19261231718,283sleep319297-21/usr/sbin/munin12:02:573
243842295125,91sleep00-21swapper/007:13:200
493722798,232sleep05096-21sed07:53:090
493722798,232sleep05096-21sed07:53:090
2943422748,253sleep029527-21sed09:08:070
24697226814,239sleep224738-21ntpq07:13:262
24712226518,226sleep224852-21latency_hist10:37:402
22447224739,186sleep00-21swapper/007:08:030
2640022468,226sleep026543-21sed12:23:250
23040223635,180sleep022256-21/usr/sbin/munin12:13:110
13916223417,198sleep313918-21sshd11:43:553
24671222932,132sleep00-21swapper/008:53:330
18943222250,111sleep00-21swapper/010:18:080
1824222024,181sleep00-21swapper/011:07:590
1824222024,181sleep00-21swapper/011:07:590
30887220724,99sleep00-21swapper/007:33:020
4175220410,105sleep04332-21unixbench_multi09:33:100
31293220166,56sleep00-21swapper/009:13:160
2845421978,14sleep00-21swapper/010:48:040
23854219715,166sleep323919-21awk10:33:053
32500219414,161sleep232647-21ls11:02:402
32500219414,161sleep232647-21ls11:02:402
3197218713,137sleep03208-21sh09:32:380
27934218425,144sleep00-21swapper/007:23:090
27934218425,144sleep00-21swapper/007:23:090
1759521838,118sleep017615-21dpkg11:57:490
13416218186,78sleep00-21swapper/011:43:070
23528217927,127sleep223688-21munin-node07:12:412
5129217814,151sleep35137-21df09:37:533
872621766,156sleep08363-1kworker/0:0H09:48:000
27949217477,19sleep00-21swapper/009:03:180
12079217413,147sleep212086-21diskstats09:57:552
8169217155,101sleep08351-21sed09:47:400
8030217010,146sleep28040-21unin-run11:27:392
8030217010,146sleep28040-21unin-run11:27:392
7911216533,80sleep00-21swapper/008:03:100
2004421646,16sleep10-21swapper/112:03:111
17989216214,30sleep20-21swapper/211:58:032
25254215935,73sleep025391-21gpgv12:22:400
25254215935,73sleep025391-21gpgv12:22:400
16416215567,50sleep016429-21munin-run10:12:390
12464215364,47sleep012481-21open_inodes09:58:060
6536215248,15sleep00-21swapper/011:22:400
26359214911,124sleep026431-21mailstats07:18:080
16018214972,42sleep016029-21sort11:52:480
29564214568,38sleep00-21swapper/010:52:490
30846214328,95sleep130887-21idleruntime-cro10:57:381
2268121438,120sleep022759-21switchtime08:48:080
30274213821,92sleep230482-21latency_hist09:12:412
1738421389,114sleep00-21swapper/010:13:110
23922213252,41sleep00-21swapper/010:33:060
23989213118,92sleep124118-21timerwakeupswit10:33:071
25448213011,95sleep20-21swapper/210:38:132
30200212633,80sleep030203-21cron07:32:380
25620212629,83sleep025737-21cat10:38:150
13938212626,88sleep013968-21grep08:22:560
2552021259,100sleep025660-21apt-key08:57:410
2182221256,82sleep00-21swapper/010:27:530
3115521229,57sleep00-21swapper/010:57:490
3115521229,57sleep00-21swapper/010:57:490
21342212232,19sleep00-21swapper/010:25:150
26091211968,33sleep126122-21/usr/sbin/munin08:58:111
11469211913,91sleep111582-21unixbench_singl08:13:121
20678211726,65sleep220760-21fschecks_time08:43:142
20146211714,85sleep120308-21ls08:42:401
1137211711,88sleep21156-21apt-get07:42:492
8551211510,81sleep00-21swapper/011:28:010
20064211433,25sleep020145-21apt-key08:42:390
122521148,94sleep01227-21sshd11:03:470
122521148,94sleep01227-21sshd11:03:470
20899211316,27sleep00-21swapper/012:08:020
19898211310,81sleep020040-21unixbench_singl12:03:100
1909211214,80sleep11911-21munin-node09:27:461
1583721128,90sleep015986-21sed11:52:400
1112211213,81sleep0227-21systemd-journal09:22:410
12057211110,17sleep00-21swapper/008:17:390
2780321108,52sleep00-21swapper/010:47:400
21527211021,70sleep121543-21cron10:27:381
22281210923,42sleep30-21swapper/312:12:543
25945210714,72sleep326002-21/usr/sbin/munin12:23:163
1394821068,85sleep013981-21ntp_kernel_pll_10:03:020
1394821068,85sleep013981-21ntp_kernel_pll_10:03:020
10223210635,11sleep00-21swapper/011:33:060
15833210514,74sleep315975-21unixbench_singl10:08:103
12315210516,74sleep312378-21cpuspeed08:17:593
25681210220,62sleep325179-21apt-get08:57:483
13969210125,51sleep114048-21fschecks_time08:22:571
2934029918,58sleep329360-21ls10:52:393
2718029923,18sleep021844-21cyclictest12:27:530
321829838,16sleep00-21swapper/007:48:080
274672987,16sleep30-21swapper/307:22:563
274672987,16sleep30-21swapper/307:22:563
895729711,70sleep19087-21unixbench_multi11:28:101
781929712,71sleep27852-21idleruntime08:03:082
3272129711,69sleep1303-21/usr/sbin/munin11:02:551
3272129711,69sleep1303-21/usr/sbin/munin11:02:551
1177029740,43sleep00-21swapper/011:38:030
1177029740,43sleep00-21swapper/011:38:030
39372955,12sleep30-21swapper/307:52:153
526629412,64sleep15273-21/usr/sbin/munin11:17:571
2891429419,23sleep10-21swapper/109:07:561
1148629423,53sleep311528-21fschecks_count11:37:583
1148629423,53sleep311528-21fschecks_count11:37:583
2531429322,50sleep325514-21latency_hist08:57:403
924629112,45sleep10-21swapper/108:07:561
630629120,49sleep16367-21/usr/sbin/munin07:58:101
3180729113,59sleep331930-21/usr/sbin/munin10:58:083
3180729113,59sleep331930-21/usr/sbin/munin10:58:083
197129119,19sleep20-21swapper/211:08:052
197129119,19sleep20-21swapper/211:08:052
1043629120,13sleep00-21swapper/009:52:570
1026129116,56sleep110285-21ntp_states11:33:071
2883229011,62sleep228902-21fschecks_time12:32:552
1650528919,49sleep116618-21latency_hist10:12:401
656528817,55sleep26736-21apt-config09:42:412
2658028824,42sleep126618-21grep10:42:561
2455928822,50sleep30-21swapper/307:13:233
1671128822,40sleep316758-21grep08:32:403
1043128836,35sleep327-21ksoftirqd/309:52:563
2865228727,44sleep128673-21awk07:27:381
2865228727,44sleep128673-21awk07:27:381
1973928720,19sleep20-21swapper/210:22:382
91792868,13sleep00-21swapper/008:07:550
2881928618,13sleep00-21swapper/012:32:530
160112869,22sleep00-21swapper/008:28:090
1039928620,41sleep19913-21apt-get09:52:501
2093928517,53sleep120995-21/usr/sbin/munin12:08:031
3108528112,51sleep131117-21/usr/sbin/munin09:13:131
3025128115,15sleep30-21swapper/307:32:413
2065828115,49sleep320705-21tr10:23:123
738328014,48sleep37500-21timerwakeupswit11:23:083
738328014,48sleep37500-21timerwakeupswit11:23:083
1111728026,34sleep20-21swapper/208:13:052
2720327912,33sleep20-21swapper/212:27:552
1054127917,29sleep30-21swapper/311:33:133
462827828,33sleep24657-21apt-get11:17:382
2056527817,42sleep120539-21apt-get08:43:041
934227718,38sleep29498-21tail09:48:182
395327721,29sleep10-21swapper/109:33:071
2734527725,34sleep127243-21/usr/sbin/munin09:02:551
1875427712,47sleep318752-21apt-get08:37:493
556527624,35sleep25609-21ntpq11:18:102
2542027626,27sleep10-21swapper/112:22:411
845127522,28sleep20-21swapper/211:27:592
3163727526,23sleep20-21swapper/207:36:152
2901227517,40sleep229021-21/usr/sbin/munin07:27:552
1662127528,30sleep216381-21apt-get10:12:412
727727416,43sleep37477-21latency_hist08:02:413
2076127418,43sleep320821-21if_eth008:43:153
915127321,28sleep30-21swapper/308:07:493
2239427312,37sleep221794-21/usr/sbin/munin10:28:082
2204427319,27sleep20-21swapper/212:12:412
182062739,50sleep018277-21modprobe08:37:380
2620427217,36sleep326220-21apt10:42:383
326392719,46sleep00-21swapper/009:18:080
297362710,7sleep10-21swapper/107:28:101
1873327113,41sleep218226-21apt-get08:37:482
1612227114,40sleep116261-21unixbench_singl08:28:101
76952705,11sleep00-21swapper/011:23:490
76952705,11sleep00-21swapper/011:23:490
326172707,46sleep00-21swapper/007:38:110
2921227018,36sleep329277-21awk12:33:053
2809327019,29sleep228244-21grep07:23:112
2809327019,29sleep228244-21grep07:23:112
23802705,54sleep00-21swapper/007:47:070
2235827017,29sleep20-21swapper/212:12:572
2219427015,24sleep10-21swapper/108:47:571
2207627020,23sleep30-21swapper/307:07:543
2075227026,26sleep120794-21sort07:02:561
465926913,32sleep20-21swapper/207:53:032
465926913,32sleep20-21swapper/207:53:032
331526917,35sleep13391-21find07:48:091
2229826926,28sleep122344-21irqstats07:08:001
1459226915,42sleep114643-21fschecks_count11:48:051
357126812,40sleep33631-21fschecks_time09:32:573
3247226820,24sleep10-21swapper/111:02:391
3247226820,24sleep10-21swapper/111:02:391
2427026813,37sleep124318-21ntp_states12:18:051
2427026813,37sleep124318-21ntp_states12:18:051
1449926811,15sleep20-21swapper/211:47:582
35026722,23sleep20-21swapper/207:38:132
1487126714,27sleep20-21swapper/210:07:402
1487126714,27sleep20-21swapper/210:07:402
2288826612,21sleep20-21swapper/208:48:102
1180326611,38sleep111843-21/usr/sbin/munin11:38:041
1180326611,38sleep111843-21/usr/sbin/munin11:38:041
3224726512,19sleep10-21swapper/107:38:001
2775026522,26sleep227770-21latency_hist10:47:382
1845926518,27sleep30-21swapper/308:37:413
1738126522,20sleep30-21swapper/311:57:403
1261426518,21sleep30-21swapper/309:58:093
760526423,20sleep10-21swapper/108:02:511
242026421,28sleep32449-21munin-run07:47:393
2418026417,28sleep123349-21apt-get07:12:481
2199126425,23sleep30-21swapper/312:12:383
1770626417,32sleep117732-21fschecks_count11:57:581
2447126317,27sleep223842-21/usr/sbin/munin12:18:102
2447126317,27sleep223842-21/usr/sbin/munin12:18:102
2388926320,24sleep223380-21apt-get08:52:512
65326218,25sleep30-21swapper/309:18:153
2267726218,25sleep10-21swapper/112:13:041
191026219,28sleep11970-21/usr/sbin/munin11:08:041
191026219,28sleep11970-21/usr/sbin/munin11:08:041
152526219,19sleep30-21swapper/311:07:373
152526219,19sleep30-21swapper/311:07:373
1239826220,23sleep20-21swapper/208:18:022
817926119,27sleep38250-21/usr/sbin/munin08:03:173
717826117,29sleep27275-21latency_hist08:02:402
3229126112,35sleep132328-21fschecks_count09:18:011
30526117,29sleep2332-21/usr/sbin/munin11:02:562
30526117,29sleep2332-21/usr/sbin/munin11:02:562
2756726111,33sleep227634-21memory09:03:002
268402615,13sleep20-21swapper/207:19:112
2338626115,24sleep30-21swapper/308:52:393
1816226120,20sleep20-21swapper/210:17:372
175872614,12sleep30-21swapper/311:57:443
1579826119,29sleep315835-21latency_hist11:52:393
1353326117,25sleep313552-21apt-get10:02:493
1353326117,25sleep313552-21apt-get10:02:493
1335326113,32sleep113397-21/usr/sbin/munin11:43:041
113432617,13sleep20-21swapper/209:53:152
925726022,18sleep10-21swapper/109:48:171
268292606,11sleep20-21swapper/212:26:342
1755526020,21sleep10-21swapper/108:33:111
1703626019,19sleep20-21swapper/208:33:002
551625915,19sleep10-21swapper/107:57:381
551625915,19sleep10-21swapper/107:57:381
2856225920,21sleep30-21swapper/312:32:403
267662597,35sleep10-21swapper/107:18:131
2454825920,18sleep30-21swapper/312:18:113
2454825920,18sleep30-21swapper/312:18:113
2202125921,18sleep10-21swapper/110:27:581
2069325916,26sleep020751-21/usr/sbin/munin07:02:550
170025918,21sleep30-21swapper/309:27:403
143325914,19sleep10-21swapper/107:43:001
305272587,40sleep30-21swapper/310:53:323
2804325816,25sleep128035-21apt-get10:47:531
163582586,16sleep20-21swapper/208:28:442
925425716,25sleep29260-21diskstats08:07:572
375425714,19sleep20-21swapper/209:33:002
2905125715,26sleep329095-21irqstats09:07:583
222625714,27sleep32273-21grep11:08:103
222625714,27sleep32273-21grep11:08:103
1625125719,18sleep20-21swapper/211:52:582
1617725715,16sleep10-21swapper/111:52:571
1433825713,29sleep214419-21cut08:23:072
101225712,25sleep21-21systemd09:22:382
901925612,23sleep30-21swapper/309:48:113
649225618,25sleep30-21swapper/311:22:393
38412565,42sleep20-21swapper/207:51:012
333925614,22sleep33358-21apt-get11:12:503
3206425614,25sleep332057-21apt-get07:37:543
3119825617,17sleep30-21swapper/309:13:153
1701925616,23sleep317105-21perl10:13:023
827725514,25sleep38273-21apt-get11:27:523
3062325512,17sleep30-21swapper/307:32:573
2355825518,18sleep10-21swapper/108:52:411
2002525514,18sleep10-21swapper/110:22:411
1917525513,24sleep219169-21apt-get12:02:522
119922559,26sleep30-21swapper/308:17:383
535925416,24sleep15426-21memory09:38:001
2047425414,20sleep30-21swapper/307:02:453
2036725413,23sleep220462-21perl10:23:072
681225313,23sleep16334-21apt-get09:42:441
2846025323,18sleep10-21swapper/112:32:381
2785625312,18sleep30-21swapper/309:03:173
193325312,24sleep31930-21apt-get09:27:483
164925314,23sleep11678-21munin-run09:27:381
1276825314,18sleep20-21swapper/211:42:412
66862525,12sleep10-21swapper/111:22:411
66862525,12sleep10-21swapper/111:22:411
344125212,23sleep13448-21diskstats11:12:561
2734425213,24sleep327395-21users10:43:113
1979925216,21sleep319895-21systemctl10:22:393
1197425216,21sleep20-21swapper/211:38:092
1197425216,21sleep20-21swapper/211:38:092
3007325120,17sleep20-21swapper/210:53:072
2518725113,16sleep10-21swapper/110:38:041
2227925113,22sleep322285-21ntp_states10:28:043
1686325115,20sleep10-21swapper/110:12:481
324725016,18sleep20-21swapper/209:32:402
1274325013,22sleep112767-21ntp_states08:18:091
3007524911,21sleep130085-21munin-run12:37:391
1398424912,22sleep114010-21ntp_states10:03:031
1398424912,22sleep114010-21ntp_states10:03:031
2182024812,16sleep20-21swapper/207:07:402
2632024715,15sleep30-21swapper/307:18:073
1200024712,20sleep111997-21apt-get09:57:501
425524511,19sleep34250-21apt-get07:52:513
425524511,19sleep34250-21apt-get07:52:513
304862459,17sleep10-21swapper/107:32:511
2930024510,14sleep20-21swapper/210:52:382
50612448,13sleep20-21swapper/209:37:522
102862447,14sleep20-21swapper/211:33:082
16902377,15sleep20-21swapper/209:27:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional