You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Sat Jul 27, 2024 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
993321997233,34cyclictest275-21dbus-daemon00:00:001
993326996920,10cyclictest30-21ksoftirqd/223:25:012
993326996917,11cyclictest30-21ksoftirqd/223:55:012
993326996816,13cyclictest30-21ksoftirqd/219:10:012
99332199684,4cyclictest131rcu_preempt00:20:031
99332199684,4cyclictest131rcu_preempt00:20:031
99332199672,28cyclictest996335-21munin-run20:09:591
993321996625,10cyclictest23-21ksoftirqd/122:25:011
993321996625,10cyclictest23-21ksoftirqd/122:25:001
993321996428,12cyclictest1003163-21sed22:20:021
993326996113,10cyclictest30-21ksoftirqd/219:35:002
993330995838,15cyclictest1-21systemd19:30:003
993326995815,8cyclictest30-21ksoftirqd/221:15:012
993326995719,10cyclictest209-21jbd2/mmcblk1p2-00:00:012
99333099556,6cyclictest131rcu_preempt21:17:183
99331699558,31cyclictest19350irq/34-mmc119:40:020
993330995413,6cyclictest131rcu_preempt23:00:003
993326995414,28cyclictest993311-21cyclictest21:20:022
993326995312,8cyclictest30-21ksoftirqd/219:55:012
993321995336,13cyclictest1-21systemd23:40:011
993316995312,8cyclictest131rcu_preempt20:00:000
993330995234,14cyclictest237-21systemd-journal22:40:003
99332199513,3cyclictest131rcu_preempt22:45:031
99331699514,18cyclictest131rcu_preempt19:45:020
993316995137,10cyclictest1-21systemd22:10:010
993316995137,10cyclictest1-21systemd22:10:010
99333099508,5cyclictest131rcu_preempt00:40:013
99333099496,5cyclictest131rcu_preempt19:20:003
993326994910,7cyclictest30-21ksoftirqd/220:45:002
993321994935,10cyclictest312-21in:imuxsock23:05:031
99331699492,16cyclictest1003690-21grep22:30:010
993326994829,12cyclictest275-21dbus-daemon20:30:002
993321994836,7cyclictest1008662-21munin-plugin-st00:10:001
993330994722,21cyclictest361rcuc/321:30:013
993326994727,14cyclictest275-21dbus-daemon00:15:012
993316994730,13cyclictest237-21systemd-journal21:55:010
99333099463,40cyclictest361rcuc/321:40:023
993330994627,16cyclictest275-21dbus-daemon20:00:013
993326994630,12cyclictest1005035-21sshd22:55:172
993326994628,14cyclictest1009416-21sshd00:21:212
993326994627,15cyclictest1003010-21sshd22:19:182
993321994628,14cyclictest275-21dbus-daemon22:50:011
993321994627,14cyclictest275-21dbus-daemon20:40:011
993321994625,14cyclictest237-21systemd-journal20:30:001
99332199462,10cyclictest999973-21modprobe21:20:011
993316994631,10cyclictest275-21dbus-daemon19:25:010
993316994627,15cyclictest1-21systemd19:15:000
99333099454,6cyclictest131rcu_preempt00:00:003
993330994532,10cyclictest275-21dbus-daemon20:35:023
99332699453,7cyclictest30-21ksoftirqd/219:50:012
99332699453,7cyclictest30-21ksoftirqd/219:50:002
993326994525,16cyclictest368-21snmpd22:25:572
993321994531,10cyclictest23-21ksoftirqd/119:45:001
993321994530,12cyclictest1006136-21systemctl23:20:011
993316994534,7cyclictest312-21in:imuxsock20:50:000
993316994511,23cyclictest12-21ksoftirqd/023:30:010
993316994510,6cyclictest131rcu_preempt22:14:380
993330994431,10cyclictest37-21ksoftirqd/322:02:173
993330994428,12cyclictest275-21dbus-daemon00:10:013
993330994419,20cyclictest37-21ksoftirqd/321:50:013
993326994429,11cyclictest1001733-21sshd21:53:182
993326994426,13cyclictest1003304-21munin-run22:25:012
993326994426,13cyclictest1003304-21munin-run22:25:002
993326994418,21cyclictest275-21dbus-daemon21:05:012
993321994410,6cyclictest131rcu_preempt21:42:181
993316994436,5cyclictest1004048-21sendmail22:40:000
993316994432,9cyclictest12-21ksoftirqd/000:05:010
993316994430,9cyclictest999644-21cron21:15:010
993316994429,10cyclictest12-21ksoftirqd/021:20:000
993316994428,11cyclictest1-21systemd21:35:020
993316994428,11cyclictest1-21systemd21:35:010
99331699442,38cyclictest999190-21sed21:05:010
993330994331,8cyclictest312-21in:imuxsock23:20:033
993330994324,15cyclictest1-21systemd23:55:013
993326994330,10cyclictest368-21snmpd23:48:152
993326994328,11cyclictest275-21dbus-daemon21:17:202
993326994327,12cyclictest1009664-21sshd00:26:222
993321994330,10cyclictest23-21ksoftirqd/122:55:011
99331699436,33cyclictest12-21ksoftirqd/023:15:000
993316994336,5cyclictest19350irq/34-mmc119:50:020
993316994336,5cyclictest19350irq/34-mmc119:50:020
993316994330,9cyclictest12-21ksoftirqd/020:55:020
993316994330,9cyclictest12-21ksoftirqd/020:55:010
993316994324,4cyclictest12-21ksoftirqd/021:45:000
99333099426,7cyclictest37-21ksoftirqd/321:55:033
993330994225,12cyclictest237-21systemd-journal23:30:003
993330994224,13cyclictest237-21systemd-journal00:05:003
99333099421,5cyclictest131rcu_preempt23:45:003
993330994215,18cyclictest361rcuc/323:15:013
993326994230,9cyclictest1001224-21sshd21:40:182
993326994228,10cyclictest1002537-21munin-run22:10:022
993326994228,10cyclictest1002537-21munin-run22:10:012
993326994226,11cyclictest275-21dbus-daemon23:20:012
99332199424,5cyclictest131rcu_preempt20:59:591
993321994230,10cyclictest23-21ksoftirqd/120:05:021
993321994230,10cyclictest23-21ksoftirqd/120:05:011
993321994229,10cyclictest1007793-21sed23:50:021
993321994227,10cyclictest312-21in:imuxsock21:39:591
993321994226,12cyclictest1-21systemd23:44:591
993321994226,11cyclictest275-21dbus-daemon00:30:011
99332199421,12cyclictest1008970-21latency_hist00:15:021
99331699422,9cyclictest1008165-40mandb00:00:010
993316994225,13cyclictest1000959-21sshd21:37:170
993316994223,14cyclictest275-21dbus-daemon23:40:010
993330994130,7cyclictest1008880-21sshd00:10:203
993330994129,9cyclictest1006059-21sshd23:16:183
993330994123,14cyclictest275-21dbus-daemon23:40:013
993326994129,9cyclictest997838-21sshd20:38:172
993326994129,9cyclictest995040-21sshd19:41:192
993326994128,10cyclictest997625-21munin-run20:35:022
993326994128,10cyclictest1005810-21sshd23:11:172
993326994127,11cyclictest998610-21sshd20:53:182
993326994127,11cyclictest998610-21sshd20:53:172
993326994127,10cyclictest993515-21munin-run19:15:012
993326994122,15cyclictest1004798-21sshd22:53:182
993326994120,17cyclictest368-21snmpd21:36:312
993321994130,8cyclictest1-21systemd22:35:011
993321994130,7cyclictest237-21systemd-journal00:25:001
993316994127,10cyclictest996312-21sshd20:07:180
993316994127,10cyclictest1-21systemd21:25:010
993316994120,5cyclictest12-21ksoftirqd/019:19:590
99331699411,37cyclictest0-21swapper/020:05:010
99331699411,37cyclictest0-21swapper/020:05:010
993330994027,10cyclictest1-21systemd23:05:013
993330994027,10cyclictest1-21systemd23:05:013
99333099402,5cyclictest1-21systemd00:30:013
993330994019,16cyclictest1-21systemd23:35:003
993326994031,6cyclictest996084-21munin-plugin-st20:05:012
993326994031,6cyclictest996084-21munin-plugin-st20:05:002
993326994026,10cyclictest996583-21munin-run20:15:012
993326994026,10cyclictest993814-21munin-run19:20:002
993326994025,11cyclictest1000735-21munin-run21:35:012
993326994025,11cyclictest1000735-21munin-run21:35:002
993326994025,10cyclictest1003786-21cron22:34:592
993326994021,12cyclictest30-21ksoftirqd/220:20:012
993326994021,12cyclictest30-21ksoftirqd/220:20:002
993326994015,21cyclictest1004699-21date22:50:012
99332199407,8cyclictest23-21ksoftirqd/122:10:031
99332199407,8cyclictest23-21ksoftirqd/122:10:031
99332199405,7cyclictest23-21ksoftirqd/100:05:011
993321994027,10cyclictest275-21dbus-daemon19:20:011
993321994026,10cyclictest996856-21tr20:20:011
993321994026,10cyclictest996856-21tr20:20:001
99331699406,6cyclictest131rcu_preempt22:25:010
99331699406,6cyclictest131rcu_preempt22:25:010
99331699402,7cyclictest998137-21munin-run20:45:000
993316994021,4cyclictest12-21ksoftirqd/020:57:170
99333099395,6cyclictest131rcu_preempt23:48:163
993330993929,6cyclictest993574-21latency_hist19:15:013
993330993926,9cyclictest997003-21wc20:20:033
993330993926,9cyclictest997003-21wc20:20:033
993330993924,11cyclictest1003313-21sh22:25:013
993330993924,11cyclictest1003313-21sh22:25:013
99333099391,5cyclictest131rcu_preempt22:29:593
993326993928,8cyclictest1009951-21munin-run00:35:012
993326993928,7cyclictest995565-21grep19:55:002
993326993927,8cyclictest1006638-21latency_hist23:30:012
993326993926,10cyclictest994767-21sshd19:37:192
993326993925,11cyclictest1007396-21systemd-run23:45:002
993326993925,10cyclictest994051-21munin-run19:25:012
993326993925,10cyclictest1010194-21munin-run00:40:002
993326993924,11cyclictest994310-21munin-run19:30:022
993326993923,12cyclictest368-21snmpd22:44:392
99332199397,9cyclictest23-21ksoftirqd/121:25:161
993321993928,8cyclictest1005815-21sshd23:12:171
993321993921,12cyclictest275-21dbus-daemon23:05:011
993321993921,12cyclictest275-21dbus-daemon23:05:001
993316993927,9cyclictest12-21ksoftirqd/000:20:010
993316993927,9cyclictest12-21ksoftirqd/000:20:010
993316993923,12cyclictest1001474-21sshd21:45:170
993330993826,9cyclictest37-21ksoftirqd/320:30:023
993330993826,10cyclictest37-21ksoftirqd/300:25:003
99333099382,5cyclictest131rcu_preempt00:20:033
99333099382,5cyclictest131rcu_preempt00:20:033
993330993822,11cyclictest275-21dbus-daemon22:10:013
993330993822,11cyclictest275-21dbus-daemon22:10:003
993330993821,12cyclictest275-21dbus-daemon19:45:003
993330993821,12cyclictest1-21systemd23:10:003
99333099381,6cyclictest131rcu_preempt19:40:003
993330993816,4cyclictest37-21ksoftirqd/320:05:023
993330993816,4cyclictest37-21ksoftirqd/320:05:023
99333099381,5cyclictest131rcu_preempt21:15:013
993326993825,9cyclictest368-21snmpd21:46:152
993326993825,9cyclictest274-21cron22:00:002
993326993824,11cyclictest237-21systemd-journal00:17:002
993326993824,11cyclictest237-21systemd-journal00:17:002
993326993824,10cyclictest1007157-21munin-run23:40:012
993321993827,7cyclictest237-21systemd-journal21:30:031
993321993827,7cyclictest237-21systemd-journal21:30:031
99332199382,5cyclictest131rcu_preempt20:25:011
99331699389,4cyclictest131rcu_preempt19:35:010
99331699388,5cyclictest131rcu_preempt20:40:000
99331699385,7cyclictest131rcu_preempt23:05:010
99331699385,7cyclictest131rcu_preempt23:05:010
993316993831,4cyclictest16950irq/224-800000020:20:300
993316993828,6cyclictest12-21ksoftirqd/021:10:020
993316993827,8cyclictest1003781-21sshd22:34:170
993316993826,9cyclictest1007602-21date23:45:010
993316993823,12cyclictest237-21systemd-journal00:34:590
993330993725,9cyclictest37-21ksoftirqd/322:45:013
993330993725,9cyclictest37-21ksoftirqd/322:20:013
993330993725,10cyclictest37-21ksoftirqd/320:55:023
993330993725,10cyclictest37-21ksoftirqd/320:55:013
99333099371,5cyclictest131rcu_preempt21:35:023
99333099371,5cyclictest131rcu_preempt21:35:013
99333099371,5cyclictest131rcu_preempt19:35:023
993330993713,6cyclictest37-21ksoftirqd/320:10:003
993326993729,4cyclictest1005342-21munin-run23:05:012
993326993729,4cyclictest1005342-21munin-run23:05:002
99332699372,6cyclictest999406-21modprobe21:10:002
993326993725,8cyclictest30-21ksoftirqd/222:05:022
993326993719,13cyclictest368-21snmpd00:00:482
99332199377,9cyclictest23-21ksoftirqd/119:45:451
99332199377,9cyclictest23-21ksoftirqd/119:45:441
993321993724,9cyclictest23-21ksoftirqd/122:13:161
993321993721,12cyclictest23-21ksoftirqd/120:35:021
993321993720,13cyclictest275-21dbus-daemon23:55:011
99331699375,22cyclictest993320-21latency_hist19:10:020
993316993731,4cyclictest16950irq/224-800000022:47:170
993316993726,8cyclictest1006201-21sed23:20:020
993316993722,11cyclictest312-21in:imuxsock00:15:010
993316993720,13cyclictest275-21dbus-daemon19:30:000
993316993714,19cyclictest275-21dbus-daemon23:30:010
993330993628,5cyclictest1004555-21cron22:50:003
99333099362,5cyclictest131rcu_preempt22:00:003
993330993625,8cyclictest998567-21sed20:50:023
993330993624,9cyclictest37-21ksoftirqd/321:20:033
993330993624,6cyclictest312-21in:imuxsock19:50:013
993330993624,6cyclictest312-21in:imuxsock19:50:013
99333099361,6cyclictest131rcu_preempt20:45:003
99333099361,5cyclictest131rcu_preempt00:34:593
993330993612,5cyclictest131rcu_preempt21:10:013
99332699367,23cyclictest998890-21modprobe21:00:002
993326993626,6cyclictest312-21in:imuxsock23:35:012
993326993623,9cyclictest368-21snmpd20:48:262
993326993623,10cyclictest368-21snmpd00:09:322
993326993621,11cyclictest368-21snmpd20:05:492
993326993618,14cyclictest368-21snmpd23:05:462
993326993612,20cyclictest1-21systemd21:30:012
99332199363,3cyclictest131rcu_preempt19:39:591
993321993626,7cyclictest23-21ksoftirqd/100:40:011
993321993625,7cyclictest993513-21munin-plugin-st19:15:001
993321993623,10cyclictest994425-21sed19:30:021
993321993618,13cyclictest1-21systemd22:05:011
993321993616,14cyclictest237-21systemd-journal21:50:011
99332199361,3cyclictest131rcu_preempt21:15:001
993316993627,6cyclictest12-21ksoftirqd/023:55:020
993316993625,7cyclictest274-21cron22:19:590
993316993624,9cyclictest12-21ksoftirqd/020:30:010
993316993624,8cyclictest237-21systemd-journal20:10:010
99331699362,29cyclictest1006151-21kworker/0:200:06:190
99331699362,29cyclictest0-21swapper/021:30:020
993316993620,12cyclictest275-21dbus-daemon20:20:010
993316993620,12cyclictest275-21dbus-daemon20:20:010
993316993617,13cyclictest275-21dbus-daemon23:25:000
99333099353,5cyclictest131rcu_preempt21:00:163
99333099352,7cyclictest131rcu_preempt22:35:013
993330993522,5cyclictest37-21ksoftirqd/320:40:003
99333099351,5cyclictest131rcu_preempt22:55:013
993321993527,5cyclictest274-21cron20:55:011
993321993527,5cyclictest274-21cron20:55:011
993321993525,8cyclictest23-21ksoftirqd/120:50:011
993321993525,8cyclictest23-21ksoftirqd/120:00:021
99332199352,4cyclictest131rcu_preempt22:39:591
993316993527,6cyclictest12-21ksoftirqd/022:05:020
993316993522,9cyclictest995537-21sshd19:51:180
993316993520,11cyclictest12-21ksoftirqd/000:30:010
993316993519,12cyclictest1004299-21sshd22:43:160
99331699351,30cyclictest0-21swapper/022:55:490
99331699351,30cyclictest0-21swapper/000:20:040
99333099344,4cyclictest131rcu_preempt19:10:023
99333099343,4cyclictest131rcu_preempt20:22:183
993330993427,4cyclictest996579-21latency_hist20:15:013
993330993422,9cyclictest275-21dbus-daemon21:45:013
993330993419,12cyclictest413-21lldpd22:10:183
99333099341,4cyclictest131rcu_preempt21:00:003
993326993422,9cyclictest368-21snmpd20:23:372
99332199348,6cyclictest131rcu_preempt19:23:181
99332199345,5cyclictest23-21ksoftirqd/123:35:021
99332199344,6cyclictest131rcu_preempt21:05:001
99332199344,3cyclictest131rcu_preempt20:15:021
993321993426,5cyclictest1003547-21latency_hist22:30:011
993321993425,6cyclictest998138-21grep20:45:001
993316993417,13cyclictest237-21systemd-journal21:55:030
99331699341,31cyclictest0-21swapper/022:54:010
99331699341,29cyclictest0-21swapper/023:06:000
993330993323,6cyclictest37-21ksoftirqd/319:54:593
993330993318,12cyclictest1-21systemd19:25:013
993326993320,9cyclictest368-21snmpd22:35:362
99332199333,4cyclictest131rcu_preempt23:25:171
993321993322,8cyclictest23-21ksoftirqd/123:25:011
993321993318,12cyclictest1001972-21sshd21:55:171
99332199331,4cyclictest131rcu_preempt19:55:011
99332199331,3cyclictest131rcu_preempt23:00:001
993316993328,3cyclictest0-21swapper/020:30:200
993316993325,5cyclictest1007386-21sh23:45:000
993321993226,4cyclictest1001752-21latency_hist21:55:021
993321993223,5cyclictest999404-21munin-plugin-st21:10:011
993321993222,8cyclictest23-21ksoftirqd/121:25:011
993321993215,14cyclictest1009911-21sshd00:31:211
99332199321,3cyclictest131rcu_preempt19:10:011
993326993117,11cyclictest1002750-21sshd22:13:172
99332199312,4cyclictest131rcu_preempt19:35:021
99331699311,4cyclictest0-21swapper/000:35:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional