You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-23 - 21:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Nov 23, 2024 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21355235757,288sleep321395-21ntpq09:12:563
3181223449,320sleep331685-21latency_hist08:07:343
16301230622,266sleep016364-21latency_hist10:42:330
12697230587,191sleep312852-21latency_hist08:47:343
210102297123,129sleep321182-21apt-key10:57:333
13937229714,207sleep314020-21install07:12:333
13937229714,207sleep314020-21install07:12:333
25721227890,161sleep30-21swapper/307:47:443
27955226090,111sleep30-21swapper/311:17:553
27955226090,111sleep30-21swapper/311:17:553
44562241131,85sleep30-21swapper/310:02:573
44562241131,85sleep30-21swapper/310:02:573
30023222751,97sleep330057-21munin-plugin-st09:42:303
30023222751,97sleep330057-21munin-plugin-st09:42:303
10406222657,106sleep30-21swapper/308:38:253
10406222657,106sleep30-21swapper/308:38:253
7581222412,198sleep37627-21cat10:12:573
7581222412,198sleep37627-21cat10:12:573
16421222190,27sleep30-21swapper/308:57:593
16421222190,27sleep30-21swapper/308:57:593
8464221618,103sleep38503-21fschecks_count12:02:513
8464221618,103sleep38503-21fschecks_count12:02:513
139642214104,94sleep314032-21awk10:33:033
3847221147,150sleep33816-21latency_hist10:02:323
3847221147,150sleep33816-21latency_hist10:02:323
15561220616,134sleep315602-21proc_pri12:23:053
3965220228,117sleep34030-21latency11:47:543
191122019,174sleep32010-21latency_hist11:42:333
31951218925,101sleep30-21swapper/309:47:553
8980217810,154sleep29055-21find12:03:022
8980217810,154sleep29055-21find12:03:022
17152216911,143sleep317245-21mailstats12:28:063
27372215710,131sleep327425-21sendmail11:17:313
30777215690,22sleep30-21swapper/309:42:593
30777215690,22sleep30-21swapper/309:42:593
4356215516,124sleep14411-21cat10:02:551
4356215516,124sleep14411-21cat10:02:551
3478215193,44sleep33629-21latency_hist11:47:333
2981921496,12sleep00-21swapper/009:39:530
2981921496,12sleep00-21swapper/009:39:530
22232214917,21sleep30-21swapper/310:58:103
4945214763,19sleep34896-21apt-get08:23:123
14936214749,18sleep30-21swapper/308:52:583
14936214749,18sleep30-21swapper/308:52:583
15610214430,101sleep315616-21run-parts07:14:313
17838214110,61sleep317834-21apt-get09:02:433
15313214121,102sleep115466-21sed07:13:031
15175213955,73sleep315220-21cat10:37:523
6905213876,45sleep30-21swapper/311:57:513
4894212820,91sleep24391-21apt-get08:22:422
31022212830,25sleep331038-21apt-get11:31:033
18177212810,106sleep318260-21fschecks_time10:47:523
24064212740,48sleep00-21swapper/009:22:420
20667212739,18sleep3304-21systemd-logind09:12:323
20667212739,18sleep3304-21systemd-logind09:12:323
2548021268,102sleep30-21swapper/311:08:153
24621212511,96sleep324065-21/usr/sbin/munin09:23:013
14186212520,87sleep014215-21munin-plugin-st08:52:310
6602212415,13sleep30-21swapper/310:08:463
10786212414,93sleep210874-21ls12:08:152
24177212313,91sleep30-21swapper/307:42:483
3104212259,50sleep32581-21apt-get08:17:433
28445212221,86sleep028472-21munin-run09:37:310
28445212221,86sleep028472-21munin-run09:37:310
25163212212,95sleep025239-21timerandwakeup11:08:030
17172212212,94sleep117305-21timerwakeupswit10:43:021
3169121207,95sleep30-21swapper/311:32:523
8431211925,76sleep30-21swapper/308:32:563
9251211812,89sleep30-21swapper/310:18:023
9251211812,89sleep30-21swapper/310:18:023
1135211758,20sleep3934-21/usr/sbin/munin09:52:523
10642211751,16sleep30-21swapper/312:08:123
12636211621,74sleep012665-21munin-plugin-st08:47:300
21448211514,88sleep221441-21apt-get10:57:442
20900211559,18sleep320995-21perl07:32:513
20900211559,18sleep320995-21perl07:32:513
2698021149,23sleep30-21swapper/309:32:343
5549211248,18sleep30-21swapper/311:52:583
12550211136,19sleep30-21swapper/310:28:073
28160211015,74sleep127665-21/usr/sbin/munin11:18:011
28160211015,74sleep127665-21/usr/sbin/munin11:18:011
28152210910,86sleep028159-21ntp_states11:18:000
28152210910,86sleep028159-21ntp_states11:18:000
26101210811,82sleep026098-21apt-get11:12:440
16634210814,79sleep116648-21awk07:18:001
3786210719,68sleep03673-21/usr/sbin/munin11:47:510
27161210713,74sleep327179-21apt-get09:32:423
27161210713,74sleep327179-21apt-get09:32:423
2517521075,90sleep00-21swapper/007:44:100
8139210515,67sleep18220-21date12:02:331
2600210415,72sleep22609-21/usr/sbin/munin09:57:472
2600210415,72sleep22609-21/usr/sbin/munin09:57:472
28933210210,74sleep028936-21/usr/sbin/munin07:57:500
2859210224,51sleep03005-21unixbench_singl11:43:030
23880210117,59sleep024038-21latency_hist09:22:340
1335621005,54sleep30-21swapper/312:17:483
1335621005,54sleep30-21swapper/312:17:483
138642997,82sleep30-21swapper/308:48:223
233272986,34sleep30-21swapper/307:38:243
1828929817,63sleep218396-21tune2fs12:32:532
3016829713,72sleep330285-21cut08:02:333
3007429719,56sleep130049-21latency_hist09:42:321
3007429719,56sleep130049-21latency_hist09:42:321
296292979,73sleep329638-21munin-run11:27:323
2934229718,64sleep229438-21switchtime09:38:062
2934229718,64sleep229438-21switchtime09:38:062
230962958,14sleep30-21swapper/311:02:503
193062955,77sleep30-21swapper/312:35:403
1104229521,29sleep10-21swapper/107:02:471
1665929420,49sleep20-21swapper/208:58:062
1665929420,49sleep20-21swapper/208:58:062
230432938,70sleep323118-21mailstats09:18:013
1156929311,65sleep30-21swapper/310:27:303
1152329313,64sleep311555-21ps07:03:093
77432929,15sleep30-21swapper/308:32:323
1694729111,64sleep30-21swapper/310:42:563
65722900,6sleep10-21swapper/111:57:311
2525929020,40sleep10-21swapper/111:08:041
175929020,51sleep21797-21grep08:13:022
1217328910,16sleep30-21swapper/312:13:003
3077128816,51sleep20-21swapper/209:42:582
3077128816,51sleep20-21swapper/209:42:582
1490228823,33sleep10-21swapper/112:22:351
1490228823,33sleep10-21swapper/112:22:351
1362728825,31sleep20-21swapper/210:32:542
272722878,14sleep30-21swapper/307:52:453
119592875,11sleep00-21swapper/007:03:140
66828633,29sleep20-21swapper/209:52:302
327222869,65sleep1370-21snmpd11:37:221
2158528613,53sleep021715-21users07:33:040
2158528613,53sleep021715-21users07:33:040
1670628611,59sleep30-21swapper/307:18:023
144862866,14sleep20-21swapper/212:21:312
144862866,14sleep20-21swapper/212:21:312
2455628514,55sleep124548-21ntp_states09:22:591
1475628527,27sleep10-21swapper/110:37:331
3123728417,43sleep00-21swapper/011:32:330
1268828426,32sleep20-21swapper/208:47:332
487628314,44sleep14877-21/usr/sbin/munin08:22:411
1993628316,52sleep020038-21timerwakeupswit07:28:040
1993628316,52sleep020038-21timerwakeupswit07:28:040
2238728228,35sleep222380-21apt-get07:37:462
1187128232,27sleep10-21swapper/108:43:001
9922809,10sleep30-21swapper/308:12:223
1744128014,48sleep117526-21gpgv09:02:331
1744128014,48sleep117526-21gpgv09:02:331
1112428017,50sleep211144-21cat08:42:322
1112428017,50sleep211144-21cat08:42:322
2789127919,39sleep028025-21timerwakeupswit07:53:020
2214927920,27sleep10-21swapper/107:37:331
2214927920,27sleep10-21swapper/107:37:331
188712798,11sleep30-21swapper/307:26:133
701127713,48sleep17087-21switchtime08:28:021
3215527712,50sleep131680-21apt-get08:07:361
3091527715,20sleep10-21swapper/109:43:011
3091527715,20sleep10-21swapper/109:43:011
201802775,11sleep30-21swapper/307:28:073
201802775,11sleep30-21swapper/307:28:073
267292767,12sleep00-21swapper/009:30:210
25112767,59sleep30-21swapper/308:16:543
1118427529,24sleep20-21swapper/207:02:522
2885127417,31sleep10-21swapper/107:57:471
1976227416,26sleep20-21swapper/207:28:022
1976227416,26sleep20-21swapper/207:28:022
1640327427,19sleep20-21swapper/212:27:322
1517727417,19sleep00-21swapper/012:22:500
1478227418,34sleep014579-21apt-get12:22:340
1478227418,34sleep014579-21apt-get12:22:340
921327311,26sleep09220-21ntp_states10:17:590
921327311,26sleep09220-21ntp_states10:17:590
60982736,14sleep00-21swapper/011:53:070
2057127313,47sleep120629-21latency_hist07:32:331
2057127313,47sleep120629-21latency_hist07:32:331
178582739,11sleep00-21swapper/012:31:580
1423727222,28sleep20-21swapper/208:52:332
1290227218,34sleep112919-21/usr/sbin/munin08:47:441
1290227213,28sleep00-21swapper/007:07:540
1290227213,28sleep00-21swapper/007:07:540
31322716,14sleep10-21swapper/108:17:571
2719127117,35sleep227160-21/usr/sbin/munin09:32:492
2719127117,35sleep227160-21/usr/sbin/munin09:32:492
259312719,14sleep10-21swapper/107:47:501
250527121,32sleep02522-21/usr/sbin/munin09:57:410
250527121,32sleep02522-21/usr/sbin/munin09:57:410
226127111,48sleep1370-21snmpd08:13:451
1785627112,22sleep10-21swapper/109:02:531
1300127115,40sleep212607-21/usr/sbin/munin07:07:562
1300127115,40sleep212607-21/usr/sbin/munin07:07:562
1178227115,42sleep011789-21df_inode12:12:490
660327019,35sleep26757-21latency_hist11:57:332
391527023,31sleep23964-21irqstats11:47:532
2286927027,25sleep10-21swapper/107:38:021
112462709,49sleep00-21swapper/012:10:150
104427020,30sleep20-21swapper/209:52:512
682326912,41sleep06873-21cpuspeed11:57:480
3228126911,41sleep132317-21ntp_states09:48:011
2349126920,19sleep00-21swapper/011:02:590
953526820,25sleep10-21swapper/108:37:351
498826811,40sleep15006-21munin-run11:52:321
441726822,21sleep00-21swapper/008:22:320
2460226813,31sleep124638-21ntp_states07:43:021
2118726824,25sleep021321-21apt-key10:57:340
1794426822,26sleep10-21swapper/112:32:311
1388726813,39sleep113898-21modprobe07:12:311
1388726813,39sleep113898-21modprobe07:12:311
340726714,30sleep10-21swapper/111:47:311
978926615,28sleep09808-21apt-get08:37:420
978926615,28sleep09808-21apt-get08:37:420
2964026621,30sleep029684-21wc11:27:330
2741926618,30sleep10-21swapper/107:52:511
1836026619,27sleep20-21swapper/210:47:562
165412669,29sleep00-21swapper/010:42:450
761326510,40sleep27614-21sshd08:31:142
3080526511,39sleep230841-21ntpq08:02:572
2324826512,39sleep1370-21snmpd11:02:521
715726419,31sleep17173-21awk10:12:431
715726419,31sleep17173-21awk10:12:431
2539626412,21sleep20-21swapper/211:08:052
1207526413,36sleep011841-21/usr/sbin/munin10:27:560
416126318,19sleep00-21swapper/010:02:490
416126318,19sleep00-21swapper/010:02:490
2262826318,18sleep10-21swapper/109:17:511
129512638,11sleep10-21swapper/110:30:151
1060926318,27sleep110753-21uptime08:38:281
1060926318,27sleep110753-21uptime08:38:281
943126215,31sleep09530-21apt-config08:37:340
74726223,22sleep00-21swapper/011:37:500
493326211,35sleep04896-21apt-get08:23:050
1827826212,31sleep118288-21/usr/sbin/munin12:32:521
1652926219,23sleep00-21swapper/008:58:040
1652926219,23sleep00-21swapper/008:58:040
1593426219,18sleep00-21swapper/007:17:340
105526225,19sleep20-21swapper/208:12:332
345026127,21sleep20-21swapper/211:47:322
3219526115,28sleep032189-21apt-get08:07:470
2598926119,23sleep20-21swapper/207:47:512
229726122,16sleep00-21swapper/009:57:330
1779926123,22sleep20-21swapper/207:22:492
15592615,11sleep10-21swapper/111:38:401
1147526120,19sleep10-21swapper/112:12:341
844826021,21sleep00-21swapper/010:17:300
844826021,21sleep00-21swapper/010:17:300
256126016,24sleep02589-21munin-run08:17:320
2414626019,26sleep20-21swapper/207:42:432
206112606,10sleep00-21swapper/009:11:480
206112606,10sleep00-21swapper/009:11:480
475625921,18sleep20-21swapper/210:03:042
475625921,18sleep20-21swapper/210:03:042
3166525918,25sleep131721-21systemd-run08:07:321
2591625915,20sleep20-21swapper/209:28:012
2370225916,21sleep00-21swapper/007:42:320
2254425918,24sleep222614-21/usr/sbin/munin09:17:472
2104225917,21sleep10-21swapper/109:12:501
1492725913,27sleep20-21swapper/208:52:572
1492725913,27sleep20-21swapper/208:52:572
1239025916,24sleep211687-21/usr/sbin/munin12:13:032
1031825917,24sleep110388-21/usr/sbin/munin10:22:481
311112586,40sleep10-21swapper/111:31:331
2987325815,27sleep229867-21apt-get11:27:432
2764625811,33sleep127653-21sh09:33:001
2764625811,33sleep127653-21sh09:33:001
267512586,10sleep10-21swapper/109:30:381
236482588,13sleep20-21swapper/211:03:012
1944925818,26sleep00-21swapper/012:37:310
1813425812,28sleep218226-21/usr/sbin/munin09:03:062
1811325815,24sleep018148-21ntp_kernel_pll_07:22:550
1798025819,16sleep20-21swapper/212:32:332
1644325820,23sleep00-21swapper/007:17:550
737325717,17sleep20-21swapper/211:58:022
2734325719,19sleep20-21swapper/207:52:482
976725615,23sleep19935-21gpgv12:07:331
976725615,23sleep19935-21gpgv12:07:331
876825614,24sleep18761-21apt-get10:17:481
876825614,24sleep18761-21apt-get10:17:481
3052825616,24sleep030579-21irqstats09:42:520
3052825616,24sleep030579-21irqstats09:42:520
2968525614,28sleep129836-21latency_hist11:27:341
267125617,20sleep10-21swapper/109:57:501
267125617,20sleep10-21swapper/109:57:501
2632425616,24sleep126373-21irqstats11:12:531
2489825612,25sleep224990-21users09:23:042
2089325615,25sleep120920-21munin-run10:57:311
1963625617,23sleep019630-21apt-get10:52:460
188522565,37sleep00-21swapper/009:04:130
1566225618,21sleep20-21swapper/210:38:032
1038925617,23sleep210395-21df_inode10:22:492
1846025518,21sleep00-21swapper/010:47:580
1573425513,22sleep015761-21munin-run08:57:310
1573425513,22sleep015761-21munin-run08:57:310
1556825519,16sleep10-21swapper/110:38:021
926325415,26sleep29338-21apt-config08:37:312
85272549,22sleep20-21swapper/210:17:322
85272549,22sleep20-21swapper/210:17:322
3204825418,23sleep032090-21cat09:47:560
2975125411,29sleep20-21swapper/207:58:042
2618525410,30sleep226080-21/usr/sbin/munin11:12:502
2576425418,22sleep025793-21df07:47:450
1849825412,19sleep10-21swapper/110:47:591
1815225413,25sleep118170-21ntp_states07:22:561
136932548,17sleep00-21swapper/010:32:560
2095625313,23sleep020950-21apt-get09:12:450
1702125312,25sleep217074-21users07:18:052
1067025315,19sleep00-21swapper/010:22:550
60025216,18sleep20-21swapper/211:37:442
311325213,25sleep23107-21apt-get08:17:462
203762525,36sleep10-21swapper/109:08:471
203762525,36sleep10-21swapper/109:08:471
18702525,30sleep10-21swapper/109:53:061
1535325213,24sleep115371-21irqstats12:22:591
1197025216,22sleep111983-21irqstats12:12:541
1184025214,19sleep20-21swapper/210:27:462
776525114,15sleep00-21swapper/008:32:330
1491525112,21sleep114926-21ntp_states08:52:561
1491525112,21sleep114926-21ntp_states08:52:561
1416725116,18sleep20-21swapper/207:12:362
83812509,16sleep00-21swapper/012:02:480
83812509,16sleep00-21swapper/012:02:480
601625012,17sleep10-21swapper/110:07:591
2809325015,19sleep20-21swapper/211:17:582
2809325015,19sleep20-21swapper/211:17:582
2222625015,20sleep20-21swapper/209:17:312
1949525012,22sleep219499-21diskstats09:07:552
1949525012,22sleep219499-21diskstats09:07:552
564124914,21sleep05670-21cpu10:07:500
3040824913,20sleep00-21swapper/008:02:460
206452495,34sleep20-21swapper/210:54:172
1654324911,14sleep20-21swapper/210:42:472
1010124911,15sleep20-21swapper/210:22:332
1010124911,15sleep20-21swapper/210:22:332
694024812,22sleep26985-21latency_hist10:12:322
324012474,13sleep20-21swapper/211:33:162
3174224714,16sleep00-21swapper/011:32:530
2170124610,21sleep121730-21irqstats10:57:591
2173724415,15sleep20-21swapper/207:33:142
2173724415,15sleep20-21swapper/207:33:142
291012385,14sleep20-21swapper/211:22:342
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional