You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 04:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Wed Apr 02, 2025 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4078011997227,13cyclictest4093193-21cat00:05:011
4078011997123,16cyclictest4078861-21cat19:25:021
4078021996924,14cyclictest4086378-21cut21:50:023
4078021996823,13cyclictest4088891-21cat22:40:033
4078011996722,14cyclictest4091721-21cut23:35:021
4078011996646,16cyclictest4080766-21sshd20:04:231
4078021996443,17cyclictest4083559-21sshd20:57:243
4078021996242,17cyclictest4094883-21sshd00:39:223
4078021996241,17cyclictest4078739-21munin-run19:25:003
4078011996245,14cyclictest4079460-21sshd19:38:221
4078011996242,17cyclictest4092062-21sshd23:41:231
4078011996240,15cyclictest306-21in:imuxsock21:05:001
4078021996142,13cyclictest4087985-21munin-run22:25:003
4078021996140,17cyclictest4086646-21sshd21:56:233
4078021996140,17cyclictest4080021-21munin-run19:50:013
4078021996138,11cyclictest4080786-21munin-plugin-st20:05:003
4078019996133,10cyclictest30-21ksoftirqd/221:46:222
4078011996147,10cyclictest4080223-21sshd19:50:231
4078011996147,10cyclictest4080223-21sshd19:50:221
407800799616,9cyclictest12-21ksoftirqd/019:30:010
407800799616,9cyclictest12-21ksoftirqd/019:30:010
4078007996131,11cyclictest12-21ksoftirqd/022:40:010
4078021996044,12cyclictest4085373-21sshd21:31:233
4078021996040,15cyclictest4092322-21sshd23:49:243
4078021996037,19cyclictest4081801-21cron20:25:003
4078021996037,19cyclictest4081801-21cron20:25:003
4078011996048,8cyclictest4084082-21sshd21:06:241
4078011996048,8cyclictest4084082-21sshd21:06:231
4078011996046,11cyclictest4087968-21sshd22:24:241
4078011996046,10cyclictest4080482-21sshd19:57:231
4078011996045,11cyclictest4091821-21munin-run23:40:011
4078011996044,13cyclictest4082534-21sshd20:36:221
4078011996044,13cyclictest4082534-21sshd20:36:221
4078011996044,12cyclictest4083562-21sshd20:58:221
4078011996044,12cyclictest4082839-21munin-run20:45:011
4078021995944,11cyclictest4078964-21sshd19:28:233
4078021995944,11cyclictest4078964-21sshd19:28:223
4078021995936,17cyclictest272-21dbus-daemon21:20:003
4078011995946,10cyclictest4089023-21munin-run22:45:001
4078011995944,12cyclictest4094166-21munin-run00:25:011
4078011995944,11cyclictest4086409-21sshd21:54:241
4078011995944,11cyclictest4086409-21sshd21:54:231
4078011995943,12cyclictest4089765-21/usr/sbin/munin23:00:001
4078011995943,11cyclictest4079001-21munin-run19:30:011
4078011995943,11cyclictest4079001-21munin-run19:30:011
4078011995942,14cyclictest4091290-21sshd23:26:241
4078011995941,15cyclictest4088463-21sshd22:33:231
4078011995941,14cyclictest48-21kcompactd021:55:301
4078011995940,14cyclictest363-21snmpd20:27:271
4078011995938,15cyclictest4085915-21cron21:44:591
4078021995845,9cyclictest4079451-21sshd19:35:243
4078021995843,10cyclictest48-21kcompactd022:11:043
4078021995840,13cyclictest4085916-21cron21:44:593
4078021995834,19cyclictest4082314-21grep20:35:003
4078021995830,23cyclictest4090209-21grep23:05:033
4078011995845,10cyclictest4093343-21sshd00:06:231
4078011995845,10cyclictest4090763-21sshd23:17:221
4078011995845,10cyclictest4085387-21sshd21:34:221
4078011995844,11cyclictest4081519-21sshd20:19:231
4078011995843,11cyclictest4089236-21sshd22:48:241
4078011995842,13cyclictest4090264-21sshd23:09:221
4078011995842,12cyclictest4081031-21munin-run20:10:001
4078011995842,12cyclictest4081031-21munin-run20:10:001
4078021995745,9cyclictest4080475-21sshd19:56:243
4078021995743,11cyclictest4094630-21sshd00:33:243
4078021995741,13cyclictest4082545-21sshd20:39:243
4078021995741,13cyclictest4082545-21sshd20:39:233
4078021995741,12cyclictest4091285-21sshd23:25:233
4078011995742,12cyclictest4092605-21munin-run23:55:021
4078011995742,12cyclictest4079991-21sshd19:49:221
4078011995742,11cyclictest4084372-21munin-run21:14:591
4078011995741,12cyclictest4078203-21munin-run19:15:011
4078011995741,11cyclictest4088772-21systemd-run22:40:021
4078021995643,9cyclictest4092057-21sshd23:40:233
4078021995642,11cyclictest4084089-21sshd21:08:243
4078021995642,11cyclictest4084089-21sshd21:08:233
4078021995642,11cyclictest4083056-21sshd20:45:243
4078021995641,11cyclictest48-21kcompactd022:19:573
4078021995641,11cyclictest4092556-21sshd23:50:233
4078021995641,11cyclictest4079735-21sshd19:42:233
4078021995641,11cyclictest4079735-21sshd19:42:233
4078021995636,16cyclictest363-21snmpd22:54:093
4078021995633,17cyclictest235-21systemd-journal20:30:003
4078011995648,4cyclictest4089485-21sshd22:53:231
4078011995644,9cyclictest4087686-21sshd22:18:231
4078011995643,10cyclictest4087179-21sshd22:08:231
4078011995643,10cyclictest4085137-21sshd21:29:231
4078011995643,10cyclictest4081780-21sshd20:20:241
4078011995643,10cyclictest4081780-21sshd20:20:231
4078011995642,10cyclictest4093842-21cron00:17:001
4078011995640,12cyclictest4094664-21/usr/sbin/munin00:35:011
4078011995640,12cyclictest363-21snmpd00:26:311
4078011995640,11cyclictest4090506-21sshd23:12:241
4078011995639,12cyclictest4092870-40mandb00:00:011
4078021995540,11cyclictest4082852-21munin-run20:45:003
4078021995537,12cyclictest1-21systemd21:05:003
4078021995536,15cyclictest4093377-21systemd-run00:10:003
4078011995541,9cyclictest4086190-21munin-run21:50:011
4078011995540,11cyclictest4082327-21munin-run20:35:011
4078011995537,13cyclictest4084616-21cron21:20:011
4078021995445,5cyclictest4089232-21sshd22:47:233
4078021995444,6cyclictest4093602-21sshd00:14:233
4078021995442,9cyclictest4078185-21sshd19:14:233
4078021995441,10cyclictest4094373-21sshd00:26:233
4078021995441,10cyclictest4078442-21sshd19:19:243
4078021995440,10cyclictest48-21kcompactd023:39:173
4078021995439,11cyclictest4085154-21munin-run21:30:013
4078021995438,13cyclictest4080240-21sshd19:54:233
4078021995438,13cyclictest4080240-21sshd19:54:233
4078021995438,12cyclictest4085694-21munin-run21:40:023
4078021995438,12cyclictest4085694-21munin-run21:40:013
4078021995438,12cyclictest4084906-21munin-run21:25:013
4078021995438,12cyclictest4084906-21munin-run21:25:003
4078021995437,12cyclictest4079235-21munin-run19:35:003
4078021995435,13cyclictest4089742-21cron22:59:593
4078021995434,15cyclictest308-21rs:main3
4078021995432,18cyclictest4083323-21cron20:55:003
4078021995427,22cyclictest4088997-21cron22:44:593
4078011995445,5cyclictest4083054-21sshd20:45:231
4078011995442,9cyclictest4088202-21sshd22:25:231
4078011995442,9cyclictest4084889-21sshd21:24:241
4078011995442,9cyclictest4084889-21sshd21:24:231
4078011995434,14cyclictest4094894-21modprobe00:40:011
4078021995341,9cyclictest4091049-21sshd23:24:233
4078021995339,11cyclictest48-21kcompactd023:08:143
4078021995339,10cyclictest363-21snmpd00:00:053
4078021995338,11cyclictest4088497-21/usr/sbin/munin22:35:013
4078021995338,11cyclictest4081503-21cron20:17:013
4078021995337,10cyclictest1-21systemd22:05:003
4078021995333,14cyclictest4092850-21run-parts00:00:003
4078019995311,12cyclictest30-21ksoftirqd/219:50:002
4078011995339,9cyclictest363-21snmpd22:10:181
4078011995339,11cyclictest48-21kcompactd023:00:281
4078011995338,11cyclictest235-21systemd-journal22:02:231
4078011995337,12cyclictest363-21snmpd19:34:481
4078011995334,15cyclictest4092346-21true23:50:001
4078011995334,14cyclictest363-21snmpd19:16:591
407800799534,7cyclictest1-21systemd19:40:010
4078021995233,14cyclictest4081169-21latency_hist20:10:023
4078021995233,14cyclictest4081169-21latency_hist20:10:023
4078021995229,18cyclictest4086414-21cron21:55:013
4078021995229,18cyclictest4086414-21cron21:55:003
4078011995233,14cyclictest4085680-21sendmail-msp21:40:011
4078011995233,14cyclictest4085680-21sendmail-msp21:40:001
4078011995231,15cyclictest4091068-21grep23:25:001
4078011995219,10cyclictest150-21pr/ttyS219:42:371
4078011995219,10cyclictest150-21pr/ttyS219:42:361
407800799524,12cyclictest12-21ksoftirqd/021:25:010
407800799524,12cyclictest12-21ksoftirqd/021:25:000
4078007995235,14cyclictest1-21systemd21:15:000
4078021995142,5cyclictest4090800-21munin-run23:20:013
4078021995141,6cyclictest4090533-21munin-run23:14:593
4078021995139,9cyclictest4081247-21sshd20:10:233
4078021995136,11cyclictest48-21kcompactd022:27:443
407801999515,9cyclictest30-21ksoftirqd/200:40:012
407801999515,7cyclictest30-21ksoftirqd/221:59:592
4078007995144,4cyclictest16950irq/224-800000022:00:000
4078007995136,12cyclictest1-21systemd19:25:000
4078007995136,11cyclictest1-21systemd22:20:010
4078021995030,16cyclictest363-21snmpd22:09:093
407801999505,10cyclictest30-21ksoftirqd/221:10:012
407801999505,10cyclictest30-21ksoftirqd/221:10:012
4078019995012,7cyclictest30-21ksoftirqd/223:45:002
407800799506,8cyclictest12-21ksoftirqd/021:35:000
4078021994942,4cyclictest4084384-21munin-run21:15:003
407801999497,6cyclictest30-21ksoftirqd/219:40:002
4078011994927,18cyclictest4078054-21latency_hist19:10:011
407800799497,9cyclictest12-21ksoftirqd/023:25:000
407800799494,7cyclictest12-21ksoftirqd/023:25:230
4078021994835,9cyclictest48-21kcompactd000:22:343
407801999487,9cyclictest30-21ksoftirqd/219:25:012
407801999487,6cyclictest30-21ksoftirqd/222:30:012
4078019994836,9cyclictest291rcuc/221:40:022
4078019994836,9cyclictest291rcuc/221:40:022
4078011994835,9cyclictest363-21snmpd20:51:281
4078011994832,11cyclictest4081244-21modprobe20:10:031
4078011994824,18cyclictest4093622-21modprobe00:15:001
4078007994841,4cyclictest16950irq/224-800000020:44:590
4078007994829,14cyclictest272-21dbus-daemon20:35:000
4078019994733,11cyclictest235-21systemd-journal00:08:232
4078019994732,11cyclictest272-21dbus-daemon19:20:002
407801999472,11cyclictest30-21ksoftirqd/220:34:242
407800799477,7cyclictest12-21ksoftirqd/000:30:020
4078007994737,8cyclictest12-21ksoftirqd/020:26:240
4078007994735,9cyclictest12-21ksoftirqd/022:15:020
4078007994734,10cyclictest1-21systemd00:20:010
4078007994731,13cyclictest12-21ksoftirqd/022:35:000
4078007994730,7cyclictest19050irq/34-mmc100:00:010
4078021994627,15cyclictest4091774-21latency_hist23:35:033
407801999464,6cyclictest30-21ksoftirqd/219:55:032
407801999464,6cyclictest30-21ksoftirqd/219:55:022
407801999464,11cyclictest30-21ksoftirqd/220:10:022
407801999464,11cyclictest30-21ksoftirqd/220:10:012
407800799468,4cyclictest131rcu_preempt23:41:230
407800799466,7cyclictest12-21ksoftirqd/020:00:020
407800799461,7cyclictest12-21ksoftirqd/021:50:010
4078007994614,22cyclictest19050irq/34-mmc119:55:020
4078007994614,22cyclictest19050irq/34-mmc119:55:020
4078021994532,10cyclictest4078118-21latency_hist19:10:023
4078019994510,5cyclictest131rcu_preempt22:40:002
407800799456,11cyclictest12-21ksoftirqd/022:45:000
407800799454,6cyclictest12-21ksoftirqd/019:17:000
4078007994538,4cyclictest16950irq/224-800000023:15:000
4078007994525,8cyclictest12-21ksoftirqd/022:05:020
407801999448,5cyclictest131rcu_preempt22:52:232
407801999446,10cyclictest30-21ksoftirqd/200:15:492
407801999444,12cyclictest30-21ksoftirqd/223:16:232
4078019994431,9cyclictest4081547-21systemctl20:20:002
4078019994430,10cyclictest272-21dbus-daemon22:35:012
407801999442,5cyclictest30-21ksoftirqd/221:20:222
407801999442,5cyclictest30-21ksoftirqd/221:20:222
407801999442,4cyclictest131rcu_preempt23:45:242
4078019994414,10cyclictest30-21ksoftirqd/223:51:152
407800799444,9cyclictest12-21ksoftirqd/020:25:010
407800799444,9cyclictest12-21ksoftirqd/020:25:010
407800799444,7cyclictest12-21ksoftirqd/023:40:000
4078007994436,6cyclictest12-21ksoftirqd/019:50:010
4078007994433,8cyclictest12-21ksoftirqd/021:05:010
407800799442,38cyclictest48-21kcompactd023:15:270
407801999436,8cyclictest30-21ksoftirqd/219:12:232
407801999435,10cyclictest30-21ksoftirqd/220:14:232
4078019994333,7cyclictest30-21ksoftirqd/223:31:222
4078019994331,9cyclictest4087683-21sshd22:17:242
407800799436,6cyclictest12-21ksoftirqd/021:37:240
407800799436,6cyclictest12-21ksoftirqd/021:37:230
407800799431,14cyclictest12-21ksoftirqd/021:44:590
407801999427,7cyclictest30-21ksoftirqd/200:02:242
407801999427,5cyclictest131rcu_preempt20:52:242
407801999427,5cyclictest131rcu_preempt19:41:242
407801999427,5cyclictest131rcu_preempt19:41:232
4078019994216,7cyclictest30-21ksoftirqd/222:59:402
407800799425,8cyclictest12-21ksoftirqd/020:55:010
407800799424,35cyclictest151rcuc/022:25:000
4078007994232,6cyclictest16950irq/224-800000019:15:000
4078007994228,10cyclictest4093398-21cat00:10:010
4078007994228,10cyclictest1-21systemd23:35:010
4078007994224,13cyclictest1-21systemd19:35:000
407800799421,37cyclictest4079737-21sshd19:42:230
407800799421,37cyclictest4079737-21sshd19:42:230
407801999419,5cyclictest131rcu_preempt20:41:232
407801999412,3cyclictest131rcu_preempt23:12:242
4078019994114,17cyclictest131rcu_preempt21:42:242
407800799419,5cyclictest131rcu_preempt21:20:000
407800799414,7cyclictest12-21ksoftirqd/022:55:030
4078007994130,7cyclictest306-21in:imuxsock22:59:590
4078007994112,25cyclictest12-21ksoftirqd/023:10:010
407801999408,4cyclictest131rcu_preempt20:38:242
407801999408,4cyclictest131rcu_preempt20:38:232
407801999407,4cyclictest131rcu_preempt20:20:242
407801999407,4cyclictest131rcu_preempt20:20:232
407801999406,6cyclictest30-21ksoftirqd/221:51:242
407801999406,6cyclictest30-21ksoftirqd/221:51:242
407801999406,5cyclictest30-21ksoftirqd/220:01:232
407801999406,5cyclictest131rcu_preempt22:05:022
407801999405,4cyclictest131rcu_preempt22:45:222
407801999404,5cyclictest131rcu_preempt21:34:232
407801999402,4cyclictest131rcu_preempt21:01:242
4078019994023,11cyclictest272-21dbus-daemon23:30:002
407801999401,10cyclictest30-21ksoftirqd/221:30:002
407800799406,5cyclictest12-21ksoftirqd/020:20:030
407800799403,9cyclictest12-21ksoftirqd/000:35:010
4078007994033,4cyclictest16950irq/224-800000021:54:240
4078007994033,4cyclictest16950irq/224-800000021:54:230
4078007994019,16cyclictest16850irq/134-800000020:01:220
407800799401,7cyclictest235-21systemd-journal21:10:010
407800799401,7cyclictest235-21systemd-journal21:10:000
407801999394,6cyclictest30-21ksoftirqd/200:30:012
4078019993930,5cyclictest4087438-21cron22:14:592
407801999392,7cyclictest30-21ksoftirqd/219:55:232
407801999391,9cyclictest30-21ksoftirqd/221:20:002
4078007993929,7cyclictest12-21ksoftirqd/019:10:020
407800799391,6cyclictest12-21ksoftirqd/020:10:000
407800799391,6cyclictest12-21ksoftirqd/020:10:000
407801999387,5cyclictest131rcu_preempt23:02:232
407801999385,7cyclictest30-21ksoftirqd/223:08:232
407801999385,5cyclictest30-21ksoftirqd/219:10:022
4078019993829,4cyclictest4088997-21cron22:45:002
407800799387,7cyclictest12-21ksoftirqd/000:12:390
407800799386,5cyclictest131rcu_preempt22:06:230
407800799386,4cyclictest131rcu_preempt20:40:030
407800799386,4cyclictest131rcu_preempt20:40:020
407800799385,11cyclictest12-21ksoftirqd/023:45:320
4078007993832,4cyclictest16950irq/224-800000023:50:230
4078007993828,7cyclictest12-21ksoftirqd/023:05:020
407801999371,9cyclictest0-21swapper/221:10:482
407801999371,8cyclictest0-21swapper/223:21:312
407800799377,4cyclictest131rcu_preempt20:15:000
407800799376,4cyclictest131rcu_preempt22:45:220
407800799374,8cyclictest12-21ksoftirqd/000:25:010
4078007993726,6cyclictest235-21systemd-journal21:25:380
407800799372,5cyclictest0-21swapper/000:00:030
4078007993725,8cyclictest12-21ksoftirqd/000:40:000
407801999366,5cyclictest131rcu_preempt20:56:232
407801999363,5cyclictest30-21ksoftirqd/200:35:012
4078019993628,5cyclictest291rcuc/200:25:022
4078019993627,7cyclictest30-21ksoftirqd/200:00:022
4078019993621,11cyclictest272-21dbus-daemon20:25:502
4078019993621,11cyclictest235-21systemd-journal19:25:242
4078019993621,11cyclictest235-21systemd-journal19:25:232
407800799361,7cyclictest12-21ksoftirqd/022:30:010
407801999353,8cyclictest30-21ksoftirqd/222:25:002
407801999351,9cyclictest0-21swapper/200:13:342
407801999351,8cyclictest30-21ksoftirqd/220:49:232
407801999351,8cyclictest0-21swapper/219:31:522
4078007993525,7cyclictest12-21ksoftirqd/020:50:010
407801999342,9cyclictest30-21ksoftirqd/222:07:432
4078019993425,6cyclictest4091790-21sshd23:37:242
4078007993320,10cyclictest408-21lldpd20:57:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional