You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-23 - 20:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Nov 23, 2024 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
544692991241,101cyclictest0-21swapper/011:55:020
544695997528,12cyclictest1-21systemd08:10:011
54469299744,6cyclictest555796-21cat10:40:030
54470399681,57cyclictest341irq_work/309:05:023
544695996529,10cyclictest550880-21systemd-run09:10:001
544695996519,10cyclictest0-21swapper/108:49:591
54470399628,33cyclictest560787-21modprobe12:20:003
544703996139,17cyclictest1-21systemd10:30:013
544703996139,17cyclictest1-21systemd10:30:013
544703996135,16cyclictest235-21systemd-journal08:55:003
544703996038,14cyclictest550867-21cron09:10:013
544703995940,16cyclictest37-21ksoftirqd/307:25:003
544703995814,27cyclictest551383-21modprobe09:20:003
544703995635,14cyclictest545153-21systemctl07:19:593
544703995536,13cyclictest37-21ksoftirqd/309:00:013
544703995536,13cyclictest37-21ksoftirqd/309:00:003
54469999544,4cyclictest131rcu_preempt08:25:012
54469299545,44cyclictest131rcu_preempt11:35:020
544703995339,11cyclictest37-21ksoftirqd/310:40:023
544703995235,13cyclictest1-21systemd08:20:003
544703995233,13cyclictest306-21in:imuxsock10:30:023
544695995242,6cyclictest1-21systemd09:35:011
54469299522,3cyclictest131rcu_preempt11:45:020
544703995115,6cyclictest131rcu_preempt09:40:013
544703995115,6cyclictest131rcu_preempt09:40:013
544695995113,7cyclictest131rcu_preempt08:00:011
54470399509,14cyclictest37-21ksoftirqd/307:50:013
544703995023,23cyclictest37-21ksoftirqd/311:00:013
544703995023,23cyclictest37-21ksoftirqd/311:00:003
54469299503,4cyclictest131rcu_preempt07:54:590
544692995034,9cyclictest0-21swapper/007:10:000
54469599493,3cyclictest131rcu_preempt11:05:011
544692994930,14cyclictest560803-21systemctl12:20:000
54470399485,23cyclictest561816-21modprobe12:40:013
54470399485,23cyclictest561816-21modprobe12:40:013
544703994811,6cyclictest131rcu_preempt07:39:593
54469999485,4cyclictest131rcu_preempt10:25:022
54469999484,4cyclictest131rcu_preempt09:55:022
54469999484,4cyclictest131rcu_preempt09:55:012
54469999483,4cyclictest131rcu_preempt09:49:592
54469999482,4cyclictest131rcu_preempt12:35:022
54469299485,38cyclictest0-21swapper/011:20:000
54470399474,3cyclictest131rcu_preempt12:15:003
544703994711,7cyclictest131rcu_preempt10:50:003
54469999476,6cyclictest131rcu_preempt12:30:022
54469999476,4cyclictest131rcu_preempt09:30:012
544699994731,12cyclictest272-21dbus-daemon10:00:012
544699994731,12cyclictest272-21dbus-daemon10:00:012
544699994731,12cyclictest235-21systemd-journal09:00:012
544699994731,12cyclictest235-21systemd-journal09:00:002
544695994734,9cyclictest557661-21cron11:19:591
544695994731,12cyclictest1-21systemd10:40:011
54469299477,4cyclictest131rcu_preempt12:34:590
54469299474,4cyclictest131rcu_preempt11:35:020
544703994632,7cyclictest306-21in:imuxsock10:20:013
544703994628,13cyclictest1-21systemd11:55:013
544703994619,17cyclictest235-21systemd-journal10:55:013
54469999464,3cyclictest131rcu_preempt09:15:032
54469999464,3cyclictest131rcu_preempt09:15:022
544699994630,12cyclictest272-21dbus-daemon07:15:002
544695994630,12cyclictest235-21systemd-journal07:10:021
54469299461,4cyclictest131rcu_preempt12:30:010
54469299461,4cyclictest131rcu_preempt10:20:000
544703994529,8cyclictest552174-21munin-plugin-st09:34:593
544703994520,16cyclictest272-21dbus-daemon08:50:013
544703994519,17cyclictest272-21dbus-daemon08:10:003
54469999454,4cyclictest131rcu_preempt12:05:012
54469999454,4cyclictest131rcu_preempt08:20:022
54469999453,4cyclictest131rcu_preempt12:00:032
544699994526,15cyclictest272-21dbus-daemon12:10:012
54469999452,4cyclictest131rcu_preempt11:00:022
544695994527,14cyclictest272-21dbus-daemon07:30:011
544695994527,14cyclictest272-21dbus-daemon07:30:001
544692994529,12cyclictest235-21systemd-journal12:15:000
544692994526,14cyclictest561054-21sshd12:24:290
544692994516,6cyclictest131rcu_preempt07:40:000
54470399443,31cyclictest555872-21munin-run10:45:013
544703994429,7cyclictest306-21in:imuxsock08:00:013
544703994426,11cyclictest306-21in:imuxsock11:05:013
544703994425,13cyclictest235-21systemd-journal12:10:003
544703994421,16cyclictest272-21dbus-daemon11:25:003
544703994421,15cyclictest272-21dbus-daemon08:30:013
544703994421,15cyclictest272-21dbus-daemon08:30:003
544703994418,7cyclictest131rcu_preempt07:45:023
544703994410,6cyclictest131rcu_preempt11:20:013
54469999446,3cyclictest131rcu_preempt11:30:012
54469999445,4cyclictest131rcu_preempt08:00:012
54469999445,4cyclictest131rcu_preempt07:53:302
54469999445,3cyclictest131rcu_preempt07:45:012
54469999443,4cyclictest131rcu_preempt10:30:032
54469999443,4cyclictest131rcu_preempt10:30:022
54469599446,8cyclictest23-21ksoftirqd/111:38:321
54469599444,6cyclictest131rcu_preempt10:15:011
544695994429,10cyclictest235-21systemd-journal07:25:001
54469599441,3cyclictest131rcu_preempt12:35:021
544695994411,6cyclictest131rcu_preempt11:50:301
54469299443,3cyclictest131rcu_preempt08:15:020
54469299443,11cyclictest0-21swapper/008:05:020
544692994428,12cyclictest1-21systemd07:50:010
54469299441,3cyclictest131rcu_preempt09:10:010
54470399434,32cyclictest559758-21sendmail12:00:013
54470399434,20cyclictest547195-21kworker/u8:308:20:023
544703994322,16cyclictest1-21systemd11:45:013
54469999433,4cyclictest131rcu_preempt09:40:012
54469999433,4cyclictest131rcu_preempt09:40:002
54469999433,3cyclictest131rcu_preempt10:20:002
54469999433,3cyclictest131rcu_preempt07:50:002
54469999431,5cyclictest131rcu_preempt10:09:282
54469599438,6cyclictest131rcu_preempt12:05:001
544695994331,9cyclictest1-21systemd09:15:011
544695994331,9cyclictest1-21systemd09:15:011
544695994330,10cyclictest553905-21cat10:05:021
544695994329,11cyclictest23-21ksoftirqd/112:11:301
544695994329,10cyclictest1-21systemd10:35:011
54469599431,5cyclictest131rcu_preempt08:39:591
544692994336,4cyclictest16950irq/224-800000012:05:000
544692994329,12cyclictest12-21ksoftirqd/010:25:000
54469299432,4cyclictest131rcu_preempt09:40:010
54469299432,4cyclictest131rcu_preempt09:40:010
544692994319,20cyclictest16850irq/134-800000007:32:300
54469299431,3cyclictest131rcu_preempt11:00:010
54469299431,3cyclictest131rcu_preempt11:00:000
54469299431,3cyclictest131rcu_preempt09:50:020
54469299431,3cyclictest131rcu_preempt09:25:020
54469299431,3cyclictest131rcu_preempt07:45:010
544703994226,9cyclictest554298-21grep10:15:003
544703994226,12cyclictest235-21systemd-journal10:05:293
54469999424,3cyclictest131rcu_preempt08:34:592
54469999424,3cyclictest131rcu_preempt08:05:012
54469999423,4cyclictest131rcu_preempt11:00:022
54469999423,3cyclictest131rcu_preempt11:25:022
54469999423,3cyclictest131rcu_preempt07:10:012
544699994229,9cyclictest272-21dbus-daemon10:05:002
544695994230,8cyclictest545193-21grep07:20:001
544695994228,11cyclictest554029-21cron10:09:591
544695994228,11cyclictest23-21ksoftirqd/108:20:011
544695994220,9cyclictest23-21ksoftirqd/109:00:031
544695994220,9cyclictest23-21ksoftirqd/109:00:021
54469299423,3cyclictest131rcu_preempt09:45:000
544692994229,10cyclictest555310-21sshd10:31:040
54469299422,3cyclictest131rcu_preempt11:30:030
54469299421,4cyclictest557961-21true11:25:000
54469299421,4cyclictest131rcu_preempt09:35:010
54469299421,3cyclictest131rcu_preempt09:00:020
54469299421,3cyclictest131rcu_preempt09:00:010
54470399419,5cyclictest131rcu_preempt11:35:003
544703994130,8cyclictest554810-21sshd10:22:303
544703994129,8cyclictest37-21ksoftirqd/311:30:033
544703994128,7cyclictest306-21in:imuxsock12:05:003
544703994127,9cyclictest553448-21cron10:00:003
544703994127,9cyclictest553448-21cron09:59:593
544703994122,14cyclictest235-21systemd-journal11:05:003
544703994118,6cyclictest131rcu_preempt09:15:023
544703994118,6cyclictest131rcu_preempt09:15:013
544703994111,21cyclictest361rcuc/307:10:013
54469999413,4cyclictest131rcu_preempt12:25:022
544699994130,8cyclictest551346-21sshd09:15:302
54469599415,5cyclictest131rcu_preempt09:25:021
544695994128,10cyclictest546002-21sed07:35:021
544695994125,12cyclictest272-21dbus-daemon09:05:011
544695994124,14cyclictest1-21systemd08:15:001
544695994110,6cyclictest131rcu_preempt10:20:001
544692994131,7cyclictest12-21ksoftirqd/008:25:010
544692994129,6cyclictest306-21in:imuxsock08:45:010
544692994129,6cyclictest306-21in:imuxsock08:45:000
544692994128,10cyclictest272-21dbus-daemon08:40:010
54469299412,6cyclictest549828-21munin-plugin-st08:50:000
544692994126,12cyclictest561964-21sed12:40:020
544692994126,12cyclictest561964-21sed12:40:020
544692994126,11cyclictest544854-21sshd07:11:300
54469299412,4cyclictest131rcu_preempt09:30:020
54469299412,3cyclictest131rcu_preempt10:55:020
54469299412,3cyclictest131rcu_preempt08:55:010
54470399406,11cyclictest37-21ksoftirqd/307:32:303
544703994029,8cyclictest561042-21sshd12:21:313
544703994021,13cyclictest235-21systemd-journal09:25:003
54469999404,4cyclictest131rcu_preempt12:15:012
544699994032,5cyclictest549323-21sendmail-msp08:40:012
544699994028,8cyclictest30-21ksoftirqd/210:55:012
54469999402,3cyclictest131rcu_preempt08:10:022
54469599405,3cyclictest131rcu_preempt12:30:021
54469599404,3cyclictest131rcu_preempt12:10:021
544695994028,9cyclictest1-21systemd08:05:011
544695994027,9cyclictest561813-21cron12:40:011
544695994027,9cyclictest561813-21cron12:40:001
544695994027,9cyclictest1-21systemd11:35:011
544695994027,10cyclictest1-21systemd12:00:011
544695994025,12cyclictest23-21ksoftirqd/111:00:021
544695994025,12cyclictest23-21ksoftirqd/111:00:021
544695994018,9cyclictest547507-21kworker/1:108:35:021
544695994016,5cyclictest23-21ksoftirqd/107:44:181
54469599401,3cyclictest131rcu_preempt11:45:001
54469299404,4cyclictest131rcu_preempt07:30:010
54469299404,4cyclictest131rcu_preempt07:30:000
544692994021,13cyclictest272-21dbus-daemon08:10:000
544692994018,9cyclictest235-21systemd-journal09:20:010
54469299401,3cyclictest131rcu_preempt09:55:020
54469299401,3cyclictest131rcu_preempt09:55:010
544703993929,6cyclictest553195-21cron09:55:013
544703993929,6cyclictest553195-21cron09:55:003
54469999395,30cyclictest548103-21latency_hist08:15:012
54469999394,3cyclictest131rcu_preempt12:20:022
544699993930,7cyclictest30-21ksoftirqd/210:45:012
544699993930,5cyclictest306-21in:imuxsock08:45:012
544699993930,5cyclictest306-21in:imuxsock08:45:002
54469599396,3cyclictest131rcu_preempt08:25:021
54469599394,4cyclictest131rcu_preempt09:50:011
544695993931,5cyclictest552431-21sendmail09:40:011
544695993931,5cyclictest552431-21sendmail09:40:001
544695993929,5cyclictest306-21in:imuxsock07:45:021
544695993928,7cyclictest235-21systemd-journal10:00:001
544695993928,7cyclictest235-21systemd-journal09:59:591
544695993912,7cyclictest131rcu_preempt09:43:301
54469299393,3cyclictest131rcu_preempt12:10:020
54469299393,2cyclictest0-21swapper/010:49:590
544692993932,4cyclictest16950irq/224-800000007:25:000
544692993927,9cyclictest553427-21sshd09:56:000
544692993927,9cyclictest553427-21sshd09:55:590
54469299392,3cyclictest131rcu_preempt10:15:000
544692993920,13cyclictest272-21dbus-daemon10:10:010
544692993918,18cyclictest16850irq/134-800000011:56:310
54469299391,4cyclictest131rcu_preempt09:00:030
54469299391,3cyclictest131rcu_preempt10:45:010
54469299391,3cyclictest131rcu_preempt10:30:020
54469299391,3cyclictest131rcu_preempt10:30:020
54470399385,21cyclictest553735-21modprobe10:05:003
54470399384,5cyclictest131rcu_preempt09:45:013
54470399384,4cyclictest131rcu_preempt08:40:013
544703993826,8cyclictest549014-21sshd08:30:403
544703993824,9cyclictest546983-21latency_hist07:55:013
54469999384,4cyclictest131rcu_preempt11:35:002
544699993827,7cyclictest552803-21cat09:45:022
544699993826,8cyclictest561806-21sshd12:38:302
544699993826,8cyclictest561806-21sshd12:38:292
544699993819,8cyclictest545293-21sed07:20:022
54469599383,4cyclictest131rcu_preempt09:29:591
544695993827,8cyclictest551623-21cat09:20:021
544695993825,9cyclictest553227-21wc09:55:021
544695993825,9cyclictest553227-21wc09:55:011
54469299384,3cyclictest131rcu_preempt08:35:010
544692993831,4cyclictest16950irq/224-800000011:15:020
544692993830,5cyclictest556907-21munin-plugin-st11:05:000
544692993812,6cyclictest131rcu_preempt07:19:590
544703993726,9cyclictest37-21ksoftirqd/307:30:013
544703993726,9cyclictest37-21ksoftirqd/307:30:003
544703993726,6cyclictest37-21ksoftirqd/308:45:023
544703993726,6cyclictest37-21ksoftirqd/308:45:013
54469999373,4cyclictest131rcu_preempt07:25:022
544699993726,6cyclictest306-21in:imuxsock09:05:002
544699993724,9cyclictest235-21systemd-journal11:20:002
544699993724,10cyclictest30-21ksoftirqd/208:55:002
54469999371,3cyclictest131rcu_preempt11:50:022
544695993728,5cyclictest306-21in:imuxsock12:20:001
544695993724,10cyclictest558393-21cut11:30:031
54469599372,3cyclictest131rcu_preempt12:25:021
54469599372,3cyclictest131rcu_preempt10:55:011
54469599371,4cyclictest131rcu_preempt11:50:011
54469599371,3cyclictest131rcu_preempt10:25:001
54469299371,33cyclictest551104-21sshd09:11:300
54469299371,33cyclictest551104-21sshd09:11:290
54470399363,23cyclictest547497-21grep08:05:013
54470399363,10cyclictest37-21ksoftirqd/311:40:013
54470399362,6cyclictest131rcu_preempt12:35:003
544703993625,9cyclictest37-21ksoftirqd/309:30:013
544703993624,6cyclictest559345-21wc11:50:013
54469999363,29cyclictest557405-21modprobe11:15:002
54469999363,29cyclictest555723-21latency_hist10:40:012
54469999362,4cyclictest131rcu_preempt09:10:012
54469999361,32cyclictest272-21dbus-daemon08:30:012
54469999361,32cyclictest272-21dbus-daemon08:30:002
54469999361,11cyclictest30-21ksoftirqd/211:36:492
544695993624,9cyclictest23-21ksoftirqd/108:55:011
544695993623,9cyclictest272-21dbus-daemon10:50:011
54469599361,4cyclictest131rcu_preempt11:10:001
54469599361,4cyclictest131rcu_preempt08:45:021
54469599361,4cyclictest131rcu_preempt08:45:021
54469599361,3cyclictest131rcu_preempt07:39:591
544692993628,5cyclictest19050irq/34-mmc111:45:250
54469299362,4cyclictest131rcu_preempt08:30:020
54469299362,4cyclictest131rcu_preempt08:30:010
54469299361,3cyclictest131rcu_preempt11:10:010
54470399359,5cyclictest131rcu_preempt07:15:023
54470399355,6cyclictest131rcu_preempt09:50:023
544703993512,6cyclictest131rcu_preempt12:30:033
54469999355,4cyclictest131rcu_preempt09:25:002
544699993524,8cyclictest30-21ksoftirqd/211:05:022
544699993522,8cyclictest235-21systemd-journal07:40:002
544699993521,10cyclictest30-21ksoftirqd/210:47:312
544699993520,10cyclictest0-21swapper/207:30:012
544699993520,10cyclictest0-21swapper/207:30:012
54469999351,4cyclictest131rcu_preempt11:10:012
54469599355,3cyclictest131rcu_preempt07:10:021
54469599353,3cyclictest131rcu_preempt08:25:031
54469599353,3cyclictest131rcu_preempt08:25:021
54469599352,4cyclictest131rcu_preempt10:41:051
544695993523,9cyclictest547040-21cut07:55:021
544695993522,9cyclictest0-21swapper/110:30:021
54469299352,4cyclictest131rcu_preempt08:00:010
54469299352,3cyclictest0-21swapper/008:20:000
54469299351,5cyclictest553926-21sed10:05:020
54470399343,7cyclictest0-21swapper/308:15:013
54469999345,3cyclictest131rcu_preempt10:35:012
54469999344,3cyclictest131rcu_preempt10:11:022
544699993424,7cyclictest30-21ksoftirqd/207:35:012
54469999342,3cyclictest131rcu_preempt08:50:002
544699993423,8cyclictest272-21dbus-daemon09:35:012
54469599342,4cyclictest131rcu_preempt10:30:021
544703993322,7cyclictest1-21systemd11:15:013
54469999333,4cyclictest131rcu_preempt11:54:292
544695993221,8cyclictest23-21ksoftirqd/111:25:021
54469599312,3cyclictest131rcu_preempt11:15:011
544699993019,8cyclictest559079-21cat11:45:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional