You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 08:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Nov 22, 2024 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9208249412,468sleep09217-21ntpq19:18:030
904623767,352sleep19128-21/usr/sbin/munin19:18:001
170582319157,148sleep317078-21latency_hist21:27:383
144532306165,85sleep30-21swapper/321:18:053
2243822868,265sleep322462-21apt-get23:27:523
201612276126,132sleep30-21swapper/319:53:053
201612276126,132sleep30-21swapper/319:53:053
24782224270,38sleep34991-21cyclictest21:52:363
28971222818,190sleep128980-21unin-run20:22:381
30984221950,30sleep331002-21fschecks_count23:57:543
1483922086,191sleep314882-21ntpq23:03:023
26740219414,155sleep326832-21fschecks_time21:57:573
21145219130,25sleep30-21swapper/323:23:043
15347217716,140sleep315388-21ntp_kernel_err19:38:043
17807216816,22sleep30-21swapper/323:12:593
1566321657,16sleep20-21swapper/221:22:402
206621619,113sleep32117-21fschecks_count22:22:553
1358721589,135sleep313732-21unixbench_singl22:58:063
12703215815,125sleep312747-21ntpq21:13:013
21002215514,119sleep30-21swapper/321:38:073
8482215416,20sleep10-21swapper/122:42:581
648521519,128sleep36529-21ntp_offset20:52:593
5690215110,17sleep30-21swapper/319:08:053
5690215110,17sleep30-21swapper/319:08:053
9427214820,104sleep19467-21cut00:32:551
190372145113,18sleep30-21swapper/321:32:543
11056214414,116sleep311091-21latency21:08:003
23645214310,21sleep30-21swapper/320:03:103
23645214310,21sleep30-21swapper/320:03:103
17821214317,108sleep117837-21latency_hist19:47:381
28504213124,78sleep028640-21grep23:48:070
7667212759,34sleep30-21swapper/320:57:513
2094212444,15sleep30-21swapper/300:08:113
26047212111,94sleep225570-21apt-get20:12:422
12024211918,27sleep312175-21head22:53:053
5376211814,88sleep15457-21memory22:33:021
5376211814,88sleep15457-21memory22:33:021
937211656,50sleep3370-21snmpd00:03:233
9211211540,13sleep30-21swapper/322:44:173
4690211414,24sleep30-21swapper/300:17:523
1815921148,68sleep318249-21perl19:47:563
23293211314,84sleep323447-21latency_hist21:47:403
22212211239,16sleep30-21swapper/319:59:273
1962221128,12sleep20-21swapper/223:18:062
508211039,45sleep30-21swapper/322:17:543
508211039,45sleep30-21swapper/322:17:543
4371211068,19sleep30-21swapper/300:17:363
4371211068,19sleep30-21swapper/300:17:363
1647121106,67sleep316534-21mailstats23:08:023
4296210957,38sleep34364-21latency_hist20:47:383
26295210917,71sleep226441-21grep23:38:082
10336210952,37sleep30-21swapper/322:48:093
28779210837,11sleep30-21swapper/320:20:423
2985210711,16sleep20-21swapper/220:42:492
28381210768,23sleep30-21swapper/323:48:063
2847721057,15sleep30-21swapper/322:03:013
9595210414,70sleep19201-21/usr/sbin/munin21:03:021
32565210314,69sleep132594-21munin-run22:17:361
13179210363,24sleep313336-21sed19:32:393
3150421028,15sleep30-21swapper/322:12:583
1775721019,15sleep20-21swapper/223:12:582
8009210045,38sleep30-21swapper/300:28:013
3089521009,26sleep330969-21cut20:27:573
29850210016,22sleep329937-21grep22:07:573
29850210016,22sleep329937-21grep22:07:573
585329911,16sleep30-21swapper/320:52:383
792229848,34sleep30-21swapper/322:42:363
306592989,74sleep330759-21latency_hist23:57:383
2043129830,46sleep120403-21/usr/sbin/munin21:37:531
434929633,34sleep04517-21sed19:03:060
3242329610,67sleep10-21swapper/120:32:551
634529563,21sleep36375-21munin-run22:37:363
634529563,21sleep36375-21munin-run22:37:363
622629549,31sleep30-21swapper/300:22:483
2446929543,35sleep30-21swapper/323:33:043
328829410,16sleep00-21swapper/000:12:560
328829410,16sleep00-21swapper/000:12:560
1731329411,18sleep10-21swapper/121:27:561
1264829415,62sleep012749-21sed22:57:380
353629361,20sleep33531-21apt-get22:27:553
122222936,13sleep00-21swapper/019:28:000
893629218,54sleep08959-21awk21:02:370
87592927,15sleep30-21swapper/319:17:503
2689629212,63sleep026956-21latency_hist23:42:380
1325129214,58sleep012858-21/usr/sbin/munin22:57:590
33532919,65sleep00-21swapper/020:42:590
3172829114,60sleep00-21swapper/022:13:030
2664829115,61sleep226649-21cpu21:57:522
126429117,59sleep01282-21modprobe00:07:360
2092229017,56sleep221001-21sendmail_mailtr21:38:062
1587529042,26sleep115909-21fschecks_count21:23:201
1340129040,35sleep30-21swapper/319:32:513
1125029014,56sleep110762-21/usr/sbin/munin21:08:061
2663628947,10sleep30-21swapper/323:39:593
177782898,20sleep317781-21cron19:47:363
1068728911,59sleep30-21swapper/300:37:373
102428940,18sleep30-21swapper/320:36:203
3245928817,21sleep10-21swapper/100:02:531
934128725,37sleep09335-21apt-get00:32:500
2566928722,45sleep225693-21munin-run20:12:382
930428639,32sleep30-21swapper/321:02:563
2718228640,11sleep30-21swapper/323:43:463
192012866,61sleep30-21swapper/323:17:553
1444428617,54sleep114462-21cpu23:02:521
80232858,58sleep30-21swapper/319:13:103
2822228517,24sleep20-21swapper/220:18:072
2722528543,10sleep30-21swapper/320:15:583
1727328540,10sleep30-21swapper/321:27:423
1443028515,54sleep214452-21pluginstate21:18:042
931228425,33sleep00-21swapper/021:02:570
301842849,17sleep3370-21snmpd20:23:533
362428321,37sleep10-21swapper/122:28:011
1153028339,12sleep30-21swapper/319:27:103
2620828240,10sleep30-21swapper/321:56:033
2128228225,33sleep021345-21cut19:57:590
1379128216,43sleep013375-21/usr/sbin/munin19:33:000
930128111,58sleep0370-21snmpd22:45:280
2361228110,49sleep123692-21tune2fs21:47:581
806828012,54sleep08123-21cat00:28:020
2955228016,45sleep229586-21/usr/sbin/munin23:52:572
2447228012,53sleep024494-21/usr/sbin/munin23:33:050
1424128019,34sleep10-21swapper/121:17:591
1533927920,34sleep015514-21cmp21:22:380
2930127821,42sleep129345-21fschecks_count20:22:541
1941327818,28sleep20-21swapper/219:52:402
770327716,31sleep00-21swapper/019:13:060
3202427745,17sleep20-21swapper/220:32:362
400827625,30sleep10-21swapper/119:02:591
329327617,39sleep23422-21latency_hist22:27:382
211932769,17sleep20-21swapper/219:57:522
47602756,12sleep20-21swapper/222:32:142
25792755,12sleep30-21swapper/320:41:043
2207727517,35sleep20-21swapper/223:27:392
1708227514,45sleep017241-21latency_hist21:27:390
825927410,50sleep18273-21awk00:28:071
755627418,26sleep00-21swapper/000:27:370
674927425,30sleep20-21swapper/222:37:552
655227427,24sleep10-21swapper/119:12:391
655227427,24sleep10-21swapper/119:12:391
2753527427,22sleep00-21swapper/023:47:380
2018427416,29sleep20-21swapper/221:37:382
1070527417,41sleep210748-21ntpq19:23:012
654727310,50sleep26585-21ps20:53:022
52127322,27sleep10-21swapper/122:17:561
52127322,27sleep10-21swapper/122:17:561
515227316,27sleep20-21swapper/200:18:032
2480327328,29sleep30-21swapper/320:08:073
799727220,32sleep18152-21latency_hist22:42:381
676627227,29sleep10-21swapper/120:53:051
500227215,40sleep25039-21proc_pri20:48:052
1609027223,32sleep20-21swapper/223:07:532
569427119,36sleep05777-21wc22:33:090
569427119,36sleep05777-21wc22:33:090
3014827121,28sleep00-21swapper/022:08:030
3014827121,28sleep00-21swapper/022:08:030
2777927110,11sleep00-21swapper/022:00:520
2407227118,28sleep00-21swapper/020:07:390
2407227118,28sleep00-21swapper/020:07:390
202827111,31sleep20-21swapper/200:08:102
1661027116,30sleep00-21swapper/019:42:570
970927016,35sleep29317-21/usr/sbin/munin00:33:002
823927023,22sleep20-21swapper/220:58:062
796527023,28sleep00-21swapper/022:42:370
2515327014,40sleep025182-21fschecks_count21:52:530
2193027016,33sleep00-21swapper/023:27:370
411326916,27sleep00-21swapper/022:28:120
3090226912,25sleep00-21swapper/023:57:510
1800526920,23sleep10-21swapper/123:13:051
131526918,21sleep20-21swapper/200:07:382
490326820,29sleep34931-21munin-run19:07:373
2803226812,24sleep00-21swapper/020:18:010
245842687,49sleep00-21swapper/021:50:070
2197826821,28sleep10-21swapper/123:27:381
1958526817,33sleep119619-21ps23:18:051
1786826815,33sleep00-21swapper/023:13:000
1365126811,42sleep113670-21irqstats19:32:571
681626716,26sleep10-21swapper/100:23:061
296592674,13sleep10-21swapper/122:07:391
296592674,13sleep10-21swapper/122:07:391
2007726719,24sleep10-21swapper/119:53:031
2007726719,24sleep10-21swapper/119:53:031
54326623,24sleep00-21swapper/020:33:060
520426618,32sleep25206-21/usr/sbin/munin19:07:482
520426618,32sleep25206-21/usr/sbin/munin19:07:482
3147226618,24sleep20-21swapper/223:58:052
735026521,28sleep07384-21awk20:57:360
275126517,25sleep00-21swapper/020:42:390
2327626517,28sleep10-21swapper/121:47:371
1628426517,24sleep20-21swapper/219:42:392
1404326518,25sleep20-21swapper/219:33:062
1260426512,21sleep10-21swapper/121:12:591
651826413,21sleep20-21swapper/200:22:592
1731426412,21sleep20-21swapper/221:27:572
827426316,28sleep28339-21find00:28:082
2785426321,23sleep20-21swapper/223:47:522
2275126312,24sleep20-21swapper/220:02:532
2275126312,24sleep20-21swapper/220:02:532
122212637,10sleep10-21swapper/122:53:111
825426224,24sleep1227-21systemd-journal19:15:131
229982629,10sleep20-21swapper/221:44:362
1315326218,24sleep20-21swapper/219:32:382
445126116,22sleep10-21swapper/100:17:381
445126116,22sleep10-21swapper/100:17:381
319226122,20sleep10-21swapper/120:42:561
2641126115,22sleep20-21swapper/221:57:392
2374326122,23sleep10-21swapper/120:04:131
2374326122,23sleep10-21swapper/120:04:131
2259926113,21sleep00-21swapper/023:28:090
225026113,22sleep10-21swapper/100:08:131
216626121,21sleep00-21swapper/000:08:120
1702526115,26sleep20-21swapper/219:43:072
2982326012,19sleep00-21swapper/020:23:050
29392609,11sleep00-21swapper/022:23:440
2555626019,23sleep00-21swapper/020:12:360
203442605,11sleep00-21swapper/023:22:080
1979926022,24sleep20-21swapper/219:52:562
1979926022,24sleep20-21swapper/219:52:562
488525917,23sleep20-21swapper/222:32:382
488525917,23sleep20-21swapper/222:32:382
480025917,24sleep00-21swapper/020:47:590
31882596,40sleep20-21swapper/200:12:542
31882596,40sleep20-21swapper/200:12:542
3057625915,23sleep10-21swapper/123:57:361
2940325911,34sleep029492-21fschecks_time23:52:550
2497825920,21sleep10-21swapper/120:08:091
244952595,11sleep10-21swapper/123:33:071
180125914,18sleep10-21swapper/120:38:001
1191125913,23sleep20-21swapper/222:53:042
1055025914,29sleep210621-21cat21:07:392
382425823,18sleep20-21swapper/219:02:562
3270925821,20sleep20-21swapper/220:33:002
2095425818,19sleep00-21swapper/019:57:380
2095425818,19sleep00-21swapper/019:57:380
1825125815,20sleep00-21swapper/019:47:580
1231025816,22sleep20-21swapper/221:12:482
468825714,26sleep14683-21apt-get00:17:501
2352825720,21sleep223598-21df21:47:552
1444325712,27sleep014437-21apt-get23:02:510
107152576,12sleep00-21swapper/022:48:420
81125613,22sleep20-21swapper/222:18:012
81125613,22sleep20-21swapper/222:18:012
1937925615,17sleep10-21swapper/119:52:391
108162569,11sleep20-21swapper/222:50:022
1013125617,22sleep10-21swapper/122:48:031
829825518,15sleep20-21swapper/222:42:552
2820325514,23sleep228197-21apt-get22:02:522
2119825515,23sleep121265-21df19:57:551
1598725514,25sleep015982-21apt-get23:07:470
1341525514,24sleep113486-21sensors_temp22:58:041
1333525517,23sleep20-21swapper/222:58:012
462125414,26sleep14641-21fschecks_count20:47:561
2619225411,15sleep10-21swapper/123:38:071
2272725412,17sleep20-21swapper/223:28:112
2011225412,27sleep020150-21sendmail21:37:360
1028625415,23sleep110304-21apt-get19:22:481
896425314,24sleep18986-21munin-run21:02:381
703325314,25sleep26551-21munin-node19:12:442
3138525312,26sleep131448-21awk23:58:041
2689725319,17sleep10-21swapper/120:13:131
2296625319,15sleep10-21swapper/123:28:181
2224125314,19sleep00-21swapper/021:42:560
1662725314,18sleep10-21swapper/123:08:041
1332825313,23sleep013325-21apt-get21:13:590
1099825313,23sleep211055-21irqstats21:07:592
1078925313,23sleep010780-21apt-get21:07:530
320122526,11sleep00-21swapper/020:32:260
2409125210,18sleep20-21swapper/223:32:552
88825111,23sleep0894-21ntp_states22:18:030
88825111,23sleep0894-21ntp_states22:18:030
3207625114,20sleep10-21swapper/120:32:391
2827825114,22sleep128287-21diskstats22:02:551
278592514,16sleep10-21swapper/122:01:541
2776825111,24sleep127761-21apt-get23:47:471
1864225112,19sleep018796-21cut21:32:390
1524825112,23sleep114954-21/usr/sbin/munin19:38:021
1490425118,19sleep20-21swapper/223:03:052
2983725012,16sleep20-21swapper/222:07:562
2983725012,16sleep20-21swapper/222:07:562
2934625010,18sleep20-21swapper/220:22:552
2857225011,23sleep028601-21/usr/sbin/munin22:03:020
2594725010,15sleep00-21swapper/020:12:400
2580725011,17sleep10-21swapper/121:53:061
143625013,22sleep21433-21apt-get20:37:482
572724913,23sleep05805-21sendmail_mailtr19:08:060
572724913,23sleep05805-21sendmail_mailtr19:08:060
3142024917,16sleep20-21swapper/222:12:572
2015524913,23sleep11-21systemd21:37:371
922724815,15sleep20-21swapper/219:18:052
204202487,17sleep00-21swapper/021:37:480
1039424814,19sleep00-21swapper/019:22:550
1623424714,17sleep00-21swapper/019:42:370
274832469,15sleep20-21swapper/223:47:362
131132467,15sleep10-21swapper/119:32:371
55472449,17sleep00-21swapper/000:18:090
297524311,16sleep10-21swapper/122:24:141
2745524111,16sleep10-21swapper/123:47:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional