You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 03:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 02, 2025 00:41:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2136052995041,5cyclictest2256624-21sendmail23:20:012
2136051994538,5cyclictest2136049-21cyclictest00:27:381
213605199451,36cyclictest2192072-21ssh21:42:081
2136050994537,5cyclictest453-21systemd-logind22:45:020
2136051994436,5cyclictest2222841-21systemd-run22:30:011
2136051994435,6cyclictest422-21dbus-daemon19:45:011
2136050994336,4cyclictest2220406-21systemd-run22:25:010
2136050994335,5cyclictest2207198-21systemd-run22:05:020
2136051994234,5cyclictest2283393-21cmp00:00:011
2136051994234,5cyclictest2190323-21latency_hist21:40:021
2136051994234,5cyclictest2190323-21latency_hist21:40:021
2136050994235,4cyclictest421-21avahi-daemon23:12:090
2136051994133,5cyclictest2238456-21unixbench-2d22:52:151
2136051994130,8cyclictest2270059-21systemd-run23:40:021
2136051994130,8cyclictest2270059-21systemd-run23:40:021
2136052994032,5cyclictest2155786-21systemd-run20:20:022
2136052994028,9cyclictest266-21systemd-journal22:41:552
2136050994034,4cyclictest2152497-21cron20:05:010
2136050994034,4cyclictest2152497-21cron20:05:010
2136052993932,4cyclictest957-21mta-sts-daemon19:47:032
2136052993932,4cyclictest2145007-21munin-run19:40:012
2136051993930,5cyclictest2255380-21grep23:17:181
2136050993931,5cyclictest2283664-21cups-browsed00:16:500
2136052993833,3cyclictest0-21swapper/223:15:012
2136052993830,5cyclictest2249409-21munin-plugin-st23:10:012
2136051993830,5cyclictest2294595-21awk00:17:161
2136050993832,3cyclictest0-21swapper/021:11:590
2136050993831,4cyclictest2209633-21systemd-run22:10:020
2136052993729,5cyclictest2216779-21cron22:20:012
2136051993729,5cyclictest2276202-21diskmemload23:50:021
2136051993728,6cyclictest349911-21cups-browsed21:52:221
2136051993728,6cyclictest2218156-21fschecks_count22:21:581
213605199371,30cyclictest2214945-21ssh22:17:081
2136050993729,6cyclictest2145009-21munin-plugin-st19:40:010
2136050993729,5cyclictest349911-21cups-browsed23:03:400
2136052993631,2cyclictest0-21swapper/200:00:052
2136052993629,4cyclictest2137550-21kthreadd19:15:012
2136052993628,6cyclictest2182934-21/usr/sbin/munin21:30:022
2136052993625,8cyclictest422-21dbus-daemon21:22:072
2136051993629,4cyclictest2264036-21fschecks_count23:31:521
2136051993628,5cyclictest2230117-21cron22:40:011
2136051993627,7cyclictest2157262-21munin-plugin-st20:25:011
2136051993627,5cyclictest2252931-21sshd23:14:551
2136051993625,8cyclictest2224171-21fschecks_time22:31:531
2136050993630,4cyclictest2260301-21ls23:25:010
2136050993628,5cyclictest421-21avahi-daemon22:39:460
2136050993628,5cyclictest349911-21cups-browsed23:28:450
2136052993530,2cyclictest0-21swapper/222:40:012
2136052993528,4cyclictest2174926-21fake-hwclock21:17:012
2136052993527,5cyclictest2152773-21sh20:10:012
2136051993528,4cyclictest698-21cron00:35:011
2136051993527,5cyclictest2231638-21ssh22:41:591
2136051993526,5cyclictest2186719-21latency_hist21:35:031
2136050993529,4cyclictest2286939-21sh00:05:010
2136050993526,7cyclictest2243383-21munin-run23:00:010
2136050993526,7cyclictest2243383-21munin-run23:00:010
2136052993429,2cyclictest0-21swapper/222:25:012
2136052993428,3cyclictest0-21swapper/200:25:012
2136052993426,5cyclictest2225200-21unixbench_singl22:32:122
2136051993427,4cyclictest2262708-21cron23:30:011
2136051993426,6cyclictest0-21swapper/100:02:211
2136051993426,5cyclictest2298147-21unixbench-2d00:22:151
2136051993425,7cyclictest2167995-21/usr/sbin/munin21:05:021
213605199342,27cyclictest2178667-21ssh21:22:041
2136050993426,5cyclictest349911-21cups-browsed23:55:130
2136050993423,9cyclictest0-21swapper/023:50:010
2136050993423,4cyclictest2249409-21munin-plugin-st23:10:010
2136052993328,2cyclictest0-21swapper/200:00:012
2136052993327,4cyclictest2136049-21cyclictest23:01:522
2136052993327,4cyclictest2136049-21cyclictest21:52:032
2136052993327,3cyclictest0-21swapper/221:35:012
2136052993327,3cyclictest0-21swapper/200:27:392
2136052993326,5cyclictest2207181-21ssh22:05:012
2136052993326,3cyclictest0-21swapper/222:57:132
2136052993326,3cyclictest0-21swapper/222:57:132
2136052993325,5cyclictest2303373-21/usr/sbin/munin00:32:112
2136052993325,5cyclictest2295951-21unin-run00:20:012
2136052993322,7cyclictest2264450-21ntp_kernel_pll_23:31:592
2136052993314,4cyclictest171rcu_preempt21:49:212
2136051993327,3cyclictest0-21swapper/122:00:011
2136051993326,5cyclictest2136049-21cyclictest22:02:101
2136050993329,2cyclictest0-21swapper/021:51:550
2136050993324,7cyclictest2190184-21/usr/sbin/munin21:40:010
2136050993324,7cyclictest2190184-21/usr/sbin/munin21:40:010
2136050993324,6cyclictest349911-21cups-browsed21:23:340
2136050993323,7cyclictest2203503-21sh22:00:010
2136050993320,10cyclictest2157263-21cut20:25:010
2136052993224,5cyclictest2289998-21/usr/sbin/munin00:12:022
2136052993224,5cyclictest2289998-21/usr/sbin/munin00:12:022
2136052993224,5cyclictest2143518-21cut19:35:012
213605299322,17cyclictest436-21gpm19:42:082
2136051993225,4cyclictest956-21mta-sts-daemon21:47:321
2136050993225,4cyclictest2268652-21timerwakeupswit23:37:180
2136050993225,4cyclictest2268652-21timerwakeupswit23:37:180
2136050993225,4cyclictest2264422-21ntp_kernel_err23:31:580
2136050993225,4cyclictest2154191-21head20:12:120
213605299314,12cyclictest2169484-21diskmemload23:20:232
2136052993126,2cyclictest0-21swapper/200:40:012
2136052993124,3cyclictest0-21swapper/223:40:022
2136052993124,3cyclictest0-21swapper/223:40:022
2136052993123,5cyclictest956-21mta-sts-daemon21:41:532
2136052993123,5cyclictest2201939-21cut21:57:142
2136052993116,4cyclictest2169484-21diskmemload22:52:012
2136051993124,4cyclictest2149499-21munin-plugin-st19:55:011
2136051993123,5cyclictest453-21systemd-logind23:45:011
2136051993121,7cyclictest2138384-21ntp_kernel_err19:17:031
2136050993127,2cyclictest0-21swapper/022:55:010
2136050993125,4cyclictest433-21polkitd00:12:250
2136050993125,4cyclictest433-21polkitd00:12:250
2136050993125,4cyclictest2158934-21unixbench_singl20:32:100
2136050993124,4cyclictest2184297-21fschecks_time21:31:580
2136050993123,6cyclictest2273690-21munin-plugin-st23:45:010
2136050993121,7cyclictest2306200-21missed_timers00:37:060
2136050993121,7cyclictest2256621-21grep23:20:020
2136050993121,7cyclictest151ktimers/020:17:060
2136050993119,9cyclictest2169484-21diskmemload23:52:050
2136052993023,5cyclictest2139034-21munin-run19:20:012
2136052993022,5cyclictest2277413-21fschecks_time23:51:522
2136052993019,8cyclictest2211920-21unixbench_singl22:12:162
2136051993025,2cyclictest0-21swapper/123:02:151
2136051993023,3cyclictest0-21swapper/121:16:551
2136051993022,5cyclictest955-21mta-sts-daemon22:15:031
2136051993022,5cyclictest349911-21cups-browsed22:55:571
2136051993022,5cyclictest349911-21cups-browsed22:55:571
2136051993022,5cyclictest1-21systemd23:55:011
2136051993020,7cyclictest2260324-21sh23:25:021
2136050993025,2cyclictest0-21swapper/019:20:010
2136050993023,4cyclictest421-21avahi-daemon21:02:110
2136050993023,4cyclictest2299370-21kworker/u8:5-xprtiod00:31:560
2136050993023,4cyclictest2222847-21systemd-run22:30:010
2136050993022,5cyclictest349911-21cups-browsed22:11:540
2136050993021,6cyclictest2283664-21cups-browsed00:08:280
2136052992923,4cyclictest2136049-21cyclictest20:30:022
2136052992922,4cyclictest959-21mta-sts-daemon23:29:492
2136052992922,4cyclictest959-21mta-sts-daemon00:09:442
2136052992922,4cyclictest2128109-21kworker/u8:14-xprtiod20:32:552
2136052992921,6cyclictest2164280-21missed_timers20:52:062
2136052992919,7cyclictest2271168-21aten_r2power_fr23:41:522
2136051992923,3cyclictest0-21swapper/123:10:011
2136051992923,3cyclictest0-21swapper/121:10:011
2136051992920,6cyclictest2148021-21systemd-run19:50:011
2136051992919,7cyclictest2165173-21latency_hist20:55:021
2136051992917,9cyclictest2290483-21latency00:11:561
2136051992917,9cyclictest2290483-21latency00:11:561
213605099293,12cyclictest2169484-21diskmemload21:47:200
2136050992921,5cyclictest2143531-21true19:35:010
2136050992920,7cyclictest2150310-21needreboot19:57:010
2136050992920,7cyclictest2150310-21needreboot19:57:010
2136050992920,6cyclictest421-21avahi-daemon00:20:110
2136050992919,8cyclictest2163497-21idleruntime-cro20:50:010
2136050992919,6cyclictest422-21dbus-daemon22:46:420
2136050992914,12cyclictest349911-21cups-browsed21:42:010
213605299283,11cyclictest2169484-21diskmemload22:47:042
2136052992822,3cyclictest0-21swapper/221:37:092
2136052992822,3cyclictest0-21swapper/221:37:092
2136052992821,4cyclictest2171986-21unixbench_multi21:12:182
2136052992820,4cyclictest0-21swapper/219:27:122
2136051992822,4cyclictest957-21mta-sts-daemon00:36:171
2136051992821,4cyclictest2209809-21latency_hist22:10:031
2136051992820,5cyclictest437-21mta-sts-daemon22:48:251
2136051992820,5cyclictest2173263-21munin-run21:15:011
2136051992820,5cyclictest2137529-21cron19:15:011
2136051992819,5cyclictest2151030-21systemd-run20:00:011
2136051992819,5cyclictest2151030-21systemd-run20:00:011
2136050992822,4cyclictest2160488-21sh20:40:010
2136052992720,4cyclictest0-21swapper/221:00:012
2136052992719,6cyclictest2163567-21latency_hist20:50:022
2136051992722,2cyclictest0-21swapper/120:41:551
2136051992721,3cyclictest0-21swapper/121:29:551
2136050992722,3cyclictest2215572-21unixbench_singl22:17:200
2136050992719,5cyclictest2146534-21systemd-run19:45:010
2136050992718,7cyclictest453-21systemd-logind19:10:020
2136050992716,8cyclictest12450irq/32-eth021:17:050
213605299263,11cyclictest2169484-21diskmemload22:07:272
2136052992621,2cyclictest0-21swapper/222:30:012
2136052992620,4cyclictest2136049-21cyclictest21:07:102
2136052992619,4cyclictest959-21mta-sts-daemon23:46:042
2136052992617,7cyclictest2156917-21switchtime20:22:142
2136051992620,3cyclictest2133528-21kworker/u8:3-xprtiod19:26:551
2136051992619,5cyclictest0-21swapper/120:16:551
2136051992618,5cyclictest2142555-21fschecks_time19:32:001
2136051992617,7cyclictest2139575-21fschecks_time19:21:571
2136050992620,4cyclictest2148537-21fschecks_time19:52:020
2136050992620,3cyclictest959-21mta-sts-daemon22:33:310
2136050992618,5cyclictest0-21swapper/021:10:010
2136052992517,5cyclictest2167376-21ntp_kernel_pll_21:02:032
2136052992515,7cyclictest2148853-21ntp_kernel_err19:52:082
2136051992518,4cyclictest2166467-21cron21:00:011
213605199251,19cyclictest2145007-21munin-run19:40:011
2136050992520,2cyclictest0-21swapper/000:25:550
2136050992518,5cyclictest151ktimers/019:22:030
213605299243,14cyclictest486-21gmain20:42:052
2136052992418,3cyclictest955-21mta-sts-daemon20:03:552
2136052992418,3cyclictest955-21mta-sts-daemon20:03:552
2136051992417,4cyclictest2152494-21cron20:05:011
2136051992417,4cyclictest2152494-21cron20:05:011
2136051992417,3cyclictest0-21swapper/100:09:091
2136051992416,5cyclictest2153531-21missed_timers20:11:571
2136050992419,2cyclictest0-21swapper/019:12:000
2136050992418,4cyclictest474-21in:imuxsock21:00:010
2136050992417,3cyclictest0-21swapper/021:29:370
213605299232,16cyclictest2159850-21ntpq20:37:022
2136052992317,3cyclictest0-21swapper/219:22:162
2136052992316,4cyclictest2148031-21kworker/u8:3-xprtiod20:00:012
2136052992316,4cyclictest2148031-21kworker/u8:3-xprtiod20:00:012
2136052992314,6cyclictest349911-21cups-browsed20:11:582
2136051992317,3cyclictest0-21swapper/120:35:031
2136050992316,5cyclictest2160742-21/usr/sbin/munin20:41:540
2136050992314,7cyclictest2157509-21latency_hist20:30:020
2136050992312,7cyclictest266-21systemd-journal19:26:550
213605199228,4cyclictest271ktimers/120:47:141
213605099222,15cyclictest2164440-21ntpq20:52:120
2136052992114,4cyclictest2128117-21kworker/u8:24-xprtiod19:10:022
2136051992115,3cyclictest0-21swapper/120:40:011
2136051992115,3cyclictest0-21swapper/119:10:021
213605099212,14cyclictest2147449-21ntpq19:47:050
213605399203,11cyclictest0-21swapper/323:12:093
213605099209,7cyclictest0-21swapper/020:10:020
213605399192,10cyclictest0-21swapper/322:42:063
213605199192,13cyclictest2157514-21munin-run20:30:011
213605399182,8cyclictest0-21swapper/323:21:563
213605399182,10cyclictest0-21swapper/323:37:043
213605399182,10cyclictest0-21swapper/323:37:043
213605399182,10cyclictest0-21swapper/323:31:523
213605399181,9cyclictest0-21swapper/322:17:143
213605399181,7cyclictest0-21swapper/321:02:103
2136051991810,5cyclictest2152754-21cron20:10:001
213605399174,7cyclictest0-21swapper/322:57:123
213605399174,7cyclictest0-21swapper/322:57:123
213605399173,8cyclictest0-21swapper/300:32:123
213605399172,9cyclictest0-21swapper/300:12:013
213605399172,9cyclictest0-21swapper/300:12:013
213605399171,9cyclictest0-21swapper/300:17:063
213605399171,8cyclictest0-21swapper/323:42:023
213605399171,10cyclictest0-21swapper/321:42:073
213605399163,7cyclictest0-21swapper/321:56:543
213605399162,8cyclictest0-21swapper/300:02:113
213605399161,9cyclictest0-21swapper/321:17:043
213605399161,9cyclictest0-21swapper/321:12:073
213605399161,8cyclictest0-21swapper/300:37:003
213605399161,7cyclictest0-21swapper/320:12:003
213605399154,6cyclictest0-21swapper/322:21:593
213605399152,8cyclictest0-21swapper/323:50:023
213605399152,6cyclictest0-21swapper/320:57:053
213605399151,8cyclictest0-21swapper/321:37:063
213605399151,8cyclictest0-21swapper/321:37:063
213605399151,8cyclictest0-21swapper/321:22:043
213605399143,6cyclictest0-21swapper/323:51:513
213605399143,6cyclictest0-21swapper/321:30:023
213605399143,5cyclictest0-21swapper/321:06:553
213605399143,5cyclictest0-21swapper/320:22:103
213605399142,7cyclictest0-21swapper/322:12:033
213605399142,7cyclictest0-21swapper/321:51:573
213605399142,7cyclictest0-21swapper/300:25:553
213605399142,7cyclictest0-21swapper/300:22:063
213605399142,6cyclictest0-21swapper/322:05:553
213605399141,9cyclictest0-21swapper/323:56:533
213605399141,9cyclictest0-21swapper/322:50:033
213605399141,9cyclictest0-21swapper/321:31:543
213605399141,8cyclictest0-21swapper/323:10:023
213605399141,8cyclictest0-21swapper/322:36:583
213605399141,8cyclictest0-21swapper/322:31:583
213605399141,8cyclictest0-21swapper/322:02:093
213605399141,7cyclictest0-21swapper/322:30:023
213605399141,6cyclictest0-21swapper/319:57:073
213605399141,6cyclictest0-21swapper/319:57:073
213605399141,6cyclictest0-21swapper/319:55:013
213605399132,6cyclictest0-21swapper/320:10:003
213605399132,6cyclictest0-21swapper/319:40:013
213605399132,6cyclictest0-21swapper/319:22:023
213605399131,9cyclictest0-21swapper/323:02:023
213605399131,8cyclictest0-21swapper/323:16:593
213605399131,7cyclictest0-21swapper/319:27:103
213605399131,7cyclictest0-21swapper/319:12:113
213605399131,7cyclictest0-21swapper/300:06:593
213605399131,6cyclictest0-21swapper/320:47:033
213605399131,6cyclictest0-21swapper/320:42:053
213605399131,6cyclictest0-21swapper/320:37:023
213605399131,6cyclictest0-21swapper/320:17:093
213605399131,6cyclictest0-21swapper/319:32:063
213605399131,6cyclictest0-21swapper/319:17:023
213605399122,8cyclictest0-21swapper/322:52:033
213605399113,5cyclictest0-21swapper/320:31:553
213605399112,5cyclictest0-21swapper/321:50:023
213605399111,6cyclictest0-21swapper/320:30:013
213605399111,5cyclictest0-21swapper/323:30:033
213605399102,5cyclictest0-21swapper/320:01:543
213605399102,5cyclictest0-21swapper/320:01:543
213605399101,6cyclictest0-21swapper/319:46:563
213605399101,5cyclictest0-21swapper/319:41:543
21360539984,3cyclictest0-21swapper/320:51:553
21360539982,5cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional