You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 03, 2025 00:41:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4029172996046,10cyclictest2283664-21cups-browsed21:37:081
4029171995547,5cyclictest4151507-21latency_hist23:25:020
4029173995433,3cyclictest171rcu_preempt22:12:042
4029171995347,3cyclictest956-21mta-sts-daemon20:37:100
4029172995142,5cyclictest4172983-21cron00:00:011
4029171995142,4cyclictest0-21swapper/021:57:060
4029171995134,14cyclictest4029193-21latency_hist19:10:020
4029173994941,5cyclictest4062064-21awk21:00:012
4029173994941,5cyclictest4062064-21awk21:00:012
4029171994839,6cyclictest4113756-21grep22:25:010
4029172994739,5cyclictest4062062-21cron21:00:011
4029172994739,5cyclictest4062062-21cron21:00:011
4029172994739,5cyclictest1-21systemd19:10:021
4029171994732,12cyclictest4128696-21idleruntime-cro22:50:010
4029173994636,7cyclictest2283664-21cups-browsed22:20:502
4029173994436,5cyclictest4110225-21latency_hist22:20:012
4029173994435,6cyclictest2283664-21cups-browsed20:22:042
402917399441,36cyclictest4171296-21sh23:57:062
4029173994335,5cyclictest266-21systemd-journal00:27:512
4029173994334,6cyclictest4185075-21systemd-run00:20:022
4029173994334,6cyclictest4185075-21systemd-run00:20:022
4029173994334,6cyclictest1-21systemd21:55:022
4029171994336,4cyclictest4169596-21grep23:55:010
4029171994335,5cyclictest4119546-21systemd-run22:35:020
4029173994233,6cyclictest4074399-21unixbench_singl21:22:182
4029172994236,3cyclictest0-21swapper/121:12:031
4029172994233,6cyclictest4073380-21fschecks_count21:21:581
4029171994230,8cyclictest4148047-21systemd-run23:20:020
4029171994230,8cyclictest4148047-21systemd-run23:20:020
4029171994228,11cyclictest4135598-21ssh23:00:020
402917199421,35cyclictest4073783-21ssh21:22:060
4029173994133,5cyclictest4122983-21munin-run22:40:012
402917299413,31cyclictest4092111-21ssh21:51:591
402917299411,31cyclictest4086625-21ssh21:42:131
402917299411,31cyclictest4086625-21ssh21:42:131
4029171994136,3cyclictest433-21polkitd22:30:010
4029171994136,3cyclictest433-21polkitd22:30:010
4029171994135,4cyclictest4110224-21idleruntime-cro22:20:010
402917199411,33cyclictest4171222-21ssh23:57:030
4029173994032,5cyclictest422-21dbus-daemon21:15:012
4029173994032,5cyclictest4060571-21sh20:55:012
4029173994032,5cyclictest4039639-21munin-run19:45:012
4029172994035,2cyclictest0-21swapper/123:25:011
402917299401,33cyclictest4162397-21ssh23:42:081
4029171994026,12cyclictest4143463-21sh23:12:120
4029173993932,4cyclictest956-21mta-sts-daemon19:47:032
4029173993932,4cyclictest1965-21latency_hist00:40:012
4029172993931,5cyclictest4188080-21munin-plugin-st00:25:011
4029172993931,5cyclictest4188080-21munin-plugin-st00:25:011
4029172993930,5cyclictest0-21swapper/122:52:021
4029171993925,9cyclictest12450irq/32-eth023:37:030
4029173993830,5cyclictest4072159-21latency_hist21:20:022
4029173993830,5cyclictest4031188-21fschecks_time19:17:022
4029173993829,6cyclictest453-21systemd-logind19:10:022
4029173993826,9cyclictest4137762-21timerwakeupswit23:02:182
4029172993830,5cyclictest4136801-21cpuspeed23:01:581
4029172993830,5cyclictest4042633-21systemd-run19:55:011
4029172993829,6cyclictest0-21swapper/100:40:011
4029172993829,5cyclictest4148011-21sendmail23:20:011
4029172993829,5cyclictest4148011-21sendmail23:20:011
4029172993818,10cyclictest4065076-21diskmemload23:12:011
4029171993831,4cyclictest4030666-21systemd-run19:15:020
4029171993824,4cyclictest4084797-21latency_hist21:40:030
4029173993731,3cyclictest4077747-21kworker/u8:13-rpciod22:00:012
4029173993730,3cyclictest0-21swapper/223:45:022
4029173993729,5cyclictest4158440-21cut23:36:572
4029173993729,5cyclictest4084868-21latency_hist21:40:032
4029173993728,4cyclictest0-21swapper/222:42:052
4029173993725,8cyclictest4169628-21systemd23:55:022
4029173993723,11cyclictest4030463-21timerwakeupswit19:12:152
4029172993731,3cyclictest0-21swapper/120:00:011
4029172993729,5cyclictest422-21dbus-daemon00:00:051
4029172993729,5cyclictest4122994-21awk22:40:011
402917199373,26cyclictest4191461-21ssh00:32:060
4029171993731,3cyclictest0-21swapper/021:42:080
4029171993731,3cyclictest0-21swapper/021:42:080
4029171993729,4cyclictest0-21swapper/023:32:050
4029171993728,6cyclictest4032243-21latency_hist19:20:020
4029171993722,8cyclictest12450irq/32-eth023:02:090
402917399363,8cyclictest171rcu_preempt22:02:092
4029173993628,5cyclictest4153854-21latency_hist23:30:022
4029173993627,5cyclictest4055115-21fschecks_count20:36:572
4029173993627,4cyclictest0-21swapper/223:32:072
4029173993625,9cyclictest4130153-21missed_timers22:51:552
402917399360,7cyclictest4133766-21ssh22:57:012
4029172993629,4cyclictest0-21swapper/122:30:001
4029172993629,4cyclictest0-21swapper/122:30:001
4029172993628,5cyclictest4169598-21cut23:55:011
4029172993628,5cyclictest4160592-21cron23:40:011
4029172993628,5cyclictest4112534-21unixbench-2d22:22:221
4029172993627,6cyclictest4036652-21systemd-run19:35:011
4029171993630,3cyclictest4126491-21kworker/u8:9-xprtiod22:45:010
4029171993627,7cyclictest4099687-21timerwakeupswit22:02:240
4029171993617,13cyclictest151ktimers/021:52:040
4029173993528,4cyclictest0-21swapper/223:20:012
4029173993528,4cyclictest0-21swapper/223:20:012
4029173993527,4cyclictest0-21swapper/200:31:592
4029173993526,5cyclictest4036102-21ntp_states19:32:092
402917399352,24cyclictest4149719-21ssh23:22:052
4029172993526,6cyclictest4155037-21idleruntime23:31:571
4029172993521,11cyclictest437-21mta-sts-daemon00:17:081
4029172993521,11cyclictest437-21mta-sts-daemon00:17:081
4029171993526,6cyclictest4044056-21unixbench_singl19:57:130
4029173993427,3cyclictest0-21swapper/221:42:112
4029173993427,3cyclictest0-21swapper/221:42:112
4029173993426,5cyclictest4174320-21fschecks_time00:02:002
4029173993423,8cyclictest4063571-21cron21:05:012
4029172993427,4cyclictest4191867-21unixbench_multi00:32:111
4029172993426,5cyclictest4072143-21systemctl21:20:011
4029171993422,10cyclictest453-21systemd-logind00:25:010
4029171993422,10cyclictest453-21systemd-logind00:25:010
4029171993421,10cyclictest4189939-21ssh00:30:020
402917199340,28cyclictest4104893-21ssh22:11:590
4029173993326,4cyclictest4044110-21cut20:00:012
4029173993326,3cyclictest0-21swapper/222:30:022
4029173993326,3cyclictest0-21swapper/222:30:022
4029173993325,3cyclictest0-21swapper/200:22:112
4029173993325,3cyclictest0-21swapper/200:22:112
4029173993324,5cyclictest4042639-21systemd-run19:55:022
4029173993324,5cyclictest0-21swapper/221:32:052
4029173993323,8cyclictest4037539-21ntp_kernel_pll_19:37:062
4029172993328,3cyclictest0-21swapper/120:20:011
4029172993327,3cyclictest0-21swapper/122:05:011
4029172993325,5cyclictest4077761-21ssh21:30:021
402917299332,24cyclictest4095758-21ssh21:57:071
4029171993327,3cyclictest4174975-21sendmail_mailtr00:02:150
4029171993324,6cyclictest421-21avahi-daemon00:13:350
4029171993316,12cyclictest12450irq/32-eth023:42:030
4029173993224,5cyclictest4178664-21latency_hist00:10:032
402917299323,22cyclictest4105030-21ssh22:12:051
4029172993224,5cyclictest4065073-21munin-plugin-st21:10:011
4029172993219,10cyclictest4165521-21sh23:48:131
4029171993226,3cyclictest4051621-21munin-plugin-st20:25:010
4029171993225,4cyclictest4057587-21cron20:45:010
402917199321,26cyclictest4130288-21ssh22:51:590
402917399315,22cyclictest4180199-21ssh00:12:062
402917399313,21cyclictest4117731-21ssh22:32:012
4029173993123,5cyclictest4057027-21ntp_kernel_pll_20:42:062
4029172993125,3cyclictest0-21swapper/122:45:011
4029172993124,3cyclictest0-21swapper/122:57:581
4029172993123,5cyclictest4079624-21sendmail_mailqu21:32:121
4029172993123,5cyclictest4045620-21cron20:05:011
4029172993121,7cyclictest4065076-21diskmemload22:35:011
4029172993120,8cyclictest698-21cron19:20:011
4029171993124,4cyclictest4078861-21aten_r2power_vo21:31:550
4029171993123,6cyclictest4067199-21sh21:12:150
4029171993123,4cyclictest0-21swapper/023:26:070
4029171993122,6cyclictest266-21systemd-journal21:30:010
4029171993119,8cyclictest4059895-21needreboot20:52:040
402917199311,25cyclictest4070446-21ssh21:17:050
4029173993023,4cyclictest0-21swapper/220:16:592
4029173993023,3cyclictest0-21swapper/223:12:122
4029173993023,3cyclictest0-21swapper/222:10:022
4029173993021,6cyclictest4110271-21kworker/u8:1+xprtiod22:50:022
4029173993021,6cyclictest4059100-21systemd-run20:50:012
4029172993024,3cyclictest0-21swapper/120:15:011
4029172993024,3cyclictest0-21swapper/119:37:091
4029172993023,4cyclictest0-21swapper/123:10:011
4029172993022,5cyclictest422-21dbus-daemon00:26:281
4029172993022,5cyclictest4038451-21aten_r2power_en19:41:531
4029171993025,2cyclictest0-21swapper/019:46:590
4029171993024,2cyclictest0-21swapper/022:10:010
4029171993019,5cyclictest4065072-21munin-run21:10:010
4029171993019,4cyclictest4121369-21sh22:37:110
4029173992922,4cyclictest4166249-21true23:50:022
4029172992922,4cyclictest4041130-21munin-plugin-st19:50:011
4029172992922,3cyclictest0-21swapper/122:50:011
4029172992921,5cyclictest422-21dbus-daemon20:10:011
4029172992920,6cyclictest2283664-21cups-browsed23:26:381
4029172992919,7cyclictest4108492-21ntp_offset22:17:081
4029171992910,15cyclictest2283664-21cups-browsed20:47:030
402917399284,12cyclictest4065076-21diskmemload21:27:342
4029173992819,6cyclictest4047138-21systemd-run20:10:012
4029173992819,5cyclictest956-21mta-sts-daemon21:08:272
4029172992823,3cyclictest0-21swapper/122:10:011
4029172992820,5cyclictest4060468-21unixbench_multi20:52:181
4029171992824,2cyclictest0-21swapper/000:35:590
4029171992823,2cyclictest0-21swapper/020:27:140
402917199282,19cyclictest4183494-21ssh00:17:070
402917199282,19cyclictest4183494-21ssh00:17:070
4029171992814,11cyclictest2283664-21cups-browsed19:32:100
4029173992713,10cyclictest351ktimers/220:27:092
4029172992720,4cyclictest0-21swapper/100:12:081
4029172992720,3cyclictest0-21swapper/120:47:031
4029171992719,6cyclictest4038676-21fschecks_time19:41:570
4029171992719,6cyclictest2283664-21cups-browsed23:09:390
4029171992718,6cyclictest0-21swapper/019:52:180
4029171992717,8cyclictest1-21systemd23:49:110
4029171992715,9cyclictest266-21systemd-journal19:21:580
4029173992620,3cyclictest0-21swapper/221:50:012
4029173992620,3cyclictest0-21swapper/221:50:012
4029173992619,4cyclictest959-21mta-sts-daemon20:32:012
4029173992619,4cyclictest957-21mta-sts-daemon23:07:322
4029173992617,6cyclictest474-21in:imuxsock20:01:012
4029173992617,6cyclictest4048622-21systemd-run20:15:012
4029173992617,6cyclictest4033551-21unixbench_multi19:22:172
4029172992620,3cyclictest956-21mta-sts-daemon19:27:031
4029172992617,6cyclictest2283664-21cups-browsed21:02:081
4029172992617,6cyclictest2283664-21cups-browsed19:24:081
4029171992621,3cyclictest4034195-21fschecks_time19:26:570
4029171992621,2cyclictest0-21swapper/020:31:560
4029171992619,5cyclictest4017968-21kworker/u8:6-xprtiod19:37:010
4029171992619,3cyclictest0-21swapper/021:48:050
4029171992619,3cyclictest0-21swapper/021:48:050
4029171992618,5cyclictest0-21swapper/021:00:010
4029171992618,5cyclictest0-21swapper/021:00:010
4029172992517,5cyclictest2283664-21cups-browsed20:23:461
402917299251,19cyclictest4177025-21ssh00:06:111
4029171992519,4cyclictest2283664-21cups-browsed20:07:240
4029173992418,3cyclictest956-21mta-sts-daemon19:25:572
4029171992419,3cyclictest959-21mta-sts-daemon20:02:110
4029171992419,3cyclictest956-21mta-sts-daemon00:07:070
4029171992416,6cyclictest4048267-21switchtime20:12:160
4029172992314,6cyclictest684-21munin-node21:47:111
4029172992314,6cyclictest684-21munin-node21:47:111
4029172992314,6cyclictest4054962-21aten_r2power_po20:36:541
402917299231,18cyclictest4053601-21df_inode20:31:591
4029171992318,3cyclictest4049432-21needreboot20:17:020
4029171992318,3cyclictest0-21swapper/021:01:550
4029172992218,2cyclictest0-21swapper/120:45:011
4029172992217,3cyclictest957-21mta-sts-daemon20:27:081
402917499201,9cyclictest0-21swapper/322:02:193
402917299202,13cyclictest4030035-21ntpq19:12:041
402917499172,9cyclictest0-21swapper/323:37:043
402917499164,6cyclictest0-21swapper/321:37:063
402917499163,8cyclictest0-21swapper/321:12:133
402917499162,9cyclictest0-21swapper/323:32:053
402917499162,9cyclictest0-21swapper/323:22:093
402917499162,8cyclictest0-21swapper/321:32:053
402917499162,8cyclictest0-21swapper/321:22:043
402917499162,7cyclictest0-21swapper/322:12:033
402917499161,9cyclictest0-21swapper/322:57:013
402917499161,8cyclictest0-21swapper/321:42:123
402917499161,8cyclictest0-21swapper/321:42:123
402917499154,5cyclictest0-21swapper/323:11:583
402917499153,7cyclictest0-21swapper/323:51:593
402917499153,7cyclictest0-21swapper/300:37:023
402917499153,6cyclictest0-21swapper/323:27:113
402917499153,5cyclictest0-21swapper/320:32:123
402917499152,9cyclictest0-21swapper/323:16:513
402917499152,9cyclictest0-21swapper/323:16:513
402917499152,8cyclictest0-21swapper/322:31:593
402917499152,8cyclictest0-21swapper/300:00:033
402917499152,7cyclictest0-21swapper/321:17:093
402917499151,9cyclictest0-21swapper/300:12:083
402917499151,7cyclictest0-21swapper/320:22:053
402917499151,7cyclictest0-21swapper/319:32:063
402917499151,7cyclictest0-21swapper/319:22:063
402917499151,7cyclictest0-21swapper/300:17:073
402917499151,7cyclictest0-21swapper/300:17:073
402917499144,6cyclictest0-21swapper/322:42:033
402917499143,5cyclictest0-21swapper/322:22:063
402917499143,5cyclictest0-21swapper/319:37:073
402917499142,8cyclictest0-21swapper/300:21:583
402917499142,8cyclictest0-21swapper/300:21:583
402917499142,7cyclictest0-21swapper/323:41:553
402917499142,7cyclictest0-21swapper/322:37:063
402917499142,7cyclictest0-21swapper/320:02:083
402917499142,6cyclictest0-21swapper/319:57:003
402917499141,8cyclictest0-21swapper/323:02:123
402917499141,7cyclictest0-21swapper/321:51:503
402917499141,6cyclictest0-21swapper/321:07:033
402917499133,4cyclictest0-21swapper/320:37:053
402917499132,8cyclictest0-21swapper/300:31:533
402917499132,7cyclictest0-21swapper/322:52:013
402917499132,6cyclictest0-21swapper/322:17:053
402917499132,6cyclictest0-21swapper/322:10:013
402917499132,6cyclictest0-21swapper/319:52:083
402917499132,6cyclictest0-21swapper/319:47:063
402917499132,6cyclictest0-21swapper/300:10:013
402917499131,8cyclictest0-21swapper/322:25:533
402917499131,8cyclictest0-21swapper/322:25:533
402917499131,7cyclictest0-21swapper/319:42:023
402917499131,7cyclictest0-21swapper/300:02:063
402917499124,6cyclictest0-21swapper/321:55:393
402917499123,6cyclictest0-21swapper/320:57:073
402917499123,6cyclictest0-21swapper/320:57:073
402917499122,6cyclictest0-21swapper/320:47:043
402917499121,6cyclictest0-21swapper/323:10:013
402917499121,6cyclictest0-21swapper/320:17:103
402917499121,6cyclictest0-21swapper/319:17:153
402917499121,5cyclictest0-21swapper/322:50:023
402917499116,3cyclictest0-21swapper/321:50:013
402917499116,3cyclictest0-21swapper/321:50:013
402917499112,6cyclictest0-21swapper/319:12:103
402917499112,5cyclictest0-21swapper/323:50:023
402917499112,5cyclictest0-21swapper/320:26:573
402917499111,6cyclictest0-21swapper/321:29:353
402917499111,4cyclictest0-21swapper/320:15:023
402917499102,5cyclictest0-21swapper/300:30:023
402917499101,6cyclictest0-21swapper/320:41:583
402917499101,6cyclictest0-21swapper/319:27:033
402917499101,6cyclictest0-21swapper/319:10:023
402917499101,5cyclictest0-21swapper/320:52:003
40291749995,3cyclictest0-21swapper/320:07:013
40291749992,4cyclictest0-21swapper/321:01:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional