You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-11 - 22:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Fri Apr 11, 2025 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1058671998277,2cyclictest0-21swapper/010:25:020
1058676998146,26cyclictest23-21ksoftirqd/107:15:021
105867699741,40cyclictest1061109-21cut07:55:031
105867699731,38cyclictest1075340-21latency_hist12:35:021
105867699721,38cyclictest1066755-21cat09:45:021
105867699711,39cyclictest1059945-21munin-run07:35:011
105867699661,61cyclictest1064532-21sshd09:01:221
105867699643,57cyclictest1069947-21sshd10:49:231
105867699642,57cyclictest1071782-21munin-run11:25:011
1058681996355,5cyclictest0-21swapper/309:50:023
105867699622,55cyclictest1063796-21munin-run08:50:001
105867699622,54cyclictest1061457-21sshd08:03:231
105867699622,54cyclictest1061457-21sshd08:03:231
1058676996222,37cyclictest1065672-21cat09:25:011
105867699621,57cyclictest1072249-21sshd11:34:241
105867699621,57cyclictest1064016-21sshd08:54:241
105867699621,57cyclictest1060222-21munin-run07:40:011
105867699621,56cyclictest1074007-21sed12:05:021
105867699613,54cyclictest1075571-21sshd12:38:231
105867699612,54cyclictest1068690-21munin-run10:25:001
105867699612,54cyclictest1064836-21munin-run09:10:021
105867699612,54cyclictest1064836-21munin-run09:10:011
105867699612,54cyclictest1062277-21munin-run08:20:001
1058676996123,34cyclictest1067642-21sshd10:04:231
105867699611,56cyclictest1059117-21sshd07:16:231
105867699611,55cyclictest1074530-21sshd12:16:231
105867699611,55cyclictest1060470-21munin-run07:45:011
1058671996153,4cyclictest12-21ksoftirqd/008:50:010
1058676996027,27cyclictest1064269-21cron09:00:011
1058676996027,27cyclictest1064269-21cron09:00:001
105867699602,53cyclictest1072522-21munin-run11:40:011
105867699602,53cyclictest1072522-21munin-run11:40:001
105867699602,53cyclictest1062513-21munin-run08:25:011
105867699601,55cyclictest1071467-21sshd11:17:241
105867699601,54cyclictest363-21snmpd10:25:161
105867699601,53cyclictest48-21kcompactd010:59:301
105867899598,7cyclictest30-21ksoftirqd/212:30:012
105867699592,53cyclictest1070952-21sshd11:05:231
105867699592,53cyclictest1063009-21munin-run08:35:001
105867699592,52cyclictest1068157-21munin-run10:15:001
105867699592,52cyclictest1060718-21munin-run07:50:011
105867699591,54cyclictest48-21kcompactd010:08:241
105867699591,54cyclictest1069693-21sshd10:43:241
105867699591,54cyclictest1066091-21sshd09:34:241
105867699591,54cyclictest1065837-21sshd09:28:231
105867699591,53cyclictest48-21kcompactd011:58:501
105867699591,52cyclictest48-21kcompactd011:04:281
105867699591,52cyclictest1059422-21cron07:25:001
1058671995922,33cyclictest12-21ksoftirqd/010:33:220
1058678995813,9cyclictest30-21ksoftirqd/207:50:022
105867699581,52cyclictest48-21kcompactd010:16:121
105867699581,52cyclictest48-21kcompactd010:16:111
105867699581,52cyclictest48-21kcompactd009:56:441
105867699573,49cyclictest363-21snmpd10:52:231
105867699572,50cyclictest363-21snmpd11:47:541
105867699572,50cyclictest1069170-21sendmail-mta10:34:131
105867699571,53cyclictest48-21kcompactd012:05:301
105867699571,52cyclictest363-21snmpd09:46:031
105867699571,52cyclictest363-21snmpd07:58:091
105867699571,52cyclictest1062731-21sshd08:26:231
105867699571,52cyclictest1062731-21sshd08:26:231
105867699571,51cyclictest48-21kcompactd011:53:181
105867699571,51cyclictest48-21kcompactd011:25:331
105867699571,51cyclictest48-21kcompactd010:35:041
1058678995615,8cyclictest30-21ksoftirqd/211:25:022
105867699562,49cyclictest1063267-21munin-run08:40:001
105867699561,50cyclictest48-21kcompactd011:44:581
105867699561,50cyclictest363-21snmpd08:10:241
105867699561,50cyclictest363-21snmpd08:10:241
105867699561,49cyclictest48-21kcompactd012:22:081
105867699553,47cyclictest363-21snmpd08:06:151
105867699552,48cyclictest363-21snmpd11:10:561
105867699552,48cyclictest363-21snmpd11:10:561
1058678995413,9cyclictest30-21ksoftirqd/208:45:002
1058678995413,9cyclictest30-21ksoftirqd/208:45:002
105867699542,47cyclictest363-21snmpd09:12:231
105867699542,47cyclictest363-21snmpd07:28:121
105867699541,49cyclictest363-21snmpd09:35:441
105867699541,48cyclictest48-21kcompactd012:12:421
105867699541,48cyclictest363-21snmpd09:18:321
1058671995436,14cyclictest1-21systemd09:50:000
105867899538,8cyclictest30-21ksoftirqd/210:25:002
105867899535,16cyclictest1059938-21systemd-run07:35:012
1058678995212,7cyclictest30-21ksoftirqd/211:10:012
1058678995211,9cyclictest30-21ksoftirqd/212:40:002
105867699521,47cyclictest363-21snmpd09:54:421
105867699521,46cyclictest363-21snmpd12:29:281
1058671995229,18cyclictest1060207-21sendmail-msp07:40:000
105867899517,12cyclictest30-21ksoftirqd/212:25:022
105867699512,44cyclictest1063502-21modprobe08:40:241
105867699512,44cyclictest1063502-21modprobe08:40:231
1058671995144,5cyclictest19050irq/34-mmc109:30:020
1058671995117,30cyclictest12-21ksoftirqd/012:29:230
1058681995035,11cyclictest1071501-21systemctl11:20:003
1058678995036,10cyclictest1-21systemd07:25:012
1058678995013,8cyclictest30-21ksoftirqd/209:10:012
1058678995013,8cyclictest30-21ksoftirqd/209:10:002
1058671995015,6cyclictest1071529-21systemd-run11:20:000
1058678994930,14cyclictest235-21systemd-journal11:55:002
1058671994933,11cyclictest235-21systemd-journal09:15:000
1058681994832,8cyclictest1064053-21latency_hist08:55:013
105867899486,9cyclictest30-21ksoftirqd/209:00:022
105867899486,9cyclictest30-21ksoftirqd/209:00:022
1058671994840,4cyclictest16950irq/224-800000012:20:010
1058671994831,12cyclictest306-21in:imuxsock12:35:020
1058671994831,11cyclictest306-21in:imuxsock11:00:010
1058681994730,13cyclictest272-21dbus-daemon09:19:593
1058678994712,7cyclictest0-21swapper/210:45:222
1058671994740,4cyclictest16950irq/224-800000008:00:010
1058671994736,7cyclictest1059140-21cron07:19:590
1058671994734,9cyclictest19050irq/34-mmc112:24:590
1058671994734,9cyclictest12-21ksoftirqd/010:00:000
1058678994631,10cyclictest1074556-21cron12:20:002
105867899462,9cyclictest30-21ksoftirqd/212:00:002
1058671994628,13cyclictest235-21systemd-journal08:45:020
1058671994628,13cyclictest235-21systemd-journal08:45:010
1058681994526,13cyclictest235-21systemd-journal09:00:003
1058681994526,13cyclictest235-21systemd-journal09:00:003
1058678994528,14cyclictest30-21ksoftirqd/210:35:012
1058678994526,14cyclictest235-21systemd-journal07:15:002
1058678994511,6cyclictest0-21swapper/208:37:412
1058671994536,7cyclictest19050irq/34-mmc110:30:000
1058671994535,7cyclictest19050irq/34-mmc109:05:000
1058671994527,14cyclictest1075322-21sshd12:33:230
1058681994428,12cyclictest272-21dbus-daemon09:45:013
1058681994427,12cyclictest272-21dbus-daemon07:40:003
1058671994430,10cyclictest235-21systemd-journal07:46:230
1058671994426,12cyclictest272-21dbus-daemon09:40:010
1058671994411,4cyclictest208-21jbd2/mmcblk1p2-10:06:230
1058681994330,10cyclictest37-21ksoftirqd/312:18:233
1058681994329,9cyclictest272-21dbus-daemon11:55:003
1058681994327,12cyclictest1-21systemd08:50:003
1058681994325,14cyclictest272-21dbus-daemon08:05:013
1058681994325,14cyclictest272-21dbus-daemon08:05:013
105867899439,7cyclictest0-21swapper/209:45:552
105867899435,8cyclictest30-21ksoftirqd/212:35:032
1058678994325,14cyclictest272-21dbus-daemon11:20:002
105867699438,30cyclictest1058822-21latency_hist07:10:021
105867199433,7cyclictest1068425-21munin-run10:20:000
105867199433,7cyclictest1068425-21munin-run10:20:000
105867199433,7cyclictest1064816-21munin-run09:10:000
105867199433,7cyclictest1064816-21munin-run09:10:000
1058671994336,4cyclictest16950irq/224-800000011:50:000
1058671994326,11cyclictest272-21dbus-daemon09:00:020
1058671994326,11cyclictest272-21dbus-daemon09:00:010
1058671994325,12cyclictest272-21dbus-daemon08:20:000
1058681994231,6cyclictest1064796-21cron09:10:003
1058681994231,6cyclictest1064796-21cron09:10:003
1058681994229,10cyclictest235-21systemd-journal08:37:233
1058681994228,11cyclictest37-21ksoftirqd/309:15:003
1058678994229,9cyclictest1072527-21systemctl11:40:012
1058678994229,9cyclictest1072527-21systemctl11:40:002
1058678994219,17cyclictest235-21systemd-journal11:04:592
1058671994236,4cyclictest16950irq/224-800000010:05:010
105867199422,36cyclictest1071210-21sshd11:12:230
105867199422,36cyclictest1071210-21sshd11:12:230
1058681994129,6cyclictest306-21in:imuxsock12:30:013
1058681994127,10cyclictest272-21dbus-daemon10:45:003
1058681994123,12cyclictest272-21dbus-daemon11:15:023
1058681994123,12cyclictest272-21dbus-daemon11:15:013
105867899419,8cyclictest0-21swapper/211:26:492
105867899417,6cyclictest0-21swapper/209:15:082
1058678994130,7cyclictest1061188-21cron08:00:002
1058678994125,12cyclictest272-21dbus-daemon09:40:002
1058678994123,11cyclictest1070447-21cron11:00:002
1058671994134,4cyclictest16950irq/224-800000009:55:000
1058671994133,4cyclictest1058673-21latency_hist07:10:010
1058671994131,6cyclictest306-21in:imuxsock09:45:010
1058671994126,12cyclictest12-21ksoftirqd/007:35:020
1058671994119,15cyclictest1072501-21cron11:40:010
1058671994119,15cyclictest1072501-21cron11:40:000
1058681994029,7cyclictest1072764-21sshd11:41:233
1058681994025,11cyclictest37-21ksoftirqd/308:20:003
1058681994023,13cyclictest0-21swapper/308:00:013
1058681994022,12cyclictest1069424-21cron10:40:003
1058681994021,13cyclictest272-21dbus-daemon09:35:003
105868199401,3cyclictest131rcu_preempt07:25:023
105867899407,6cyclictest0-21swapper/212:01:202
105867899404,7cyclictest30-21ksoftirqd/209:04:232
1058678994026,10cyclictest30-21ksoftirqd/210:10:022
1058678994023,11cyclictest30-21ksoftirqd/210:40:002
1058671994031,6cyclictest19050irq/34-mmc111:54:120
1058671994023,13cyclictest12-21ksoftirqd/008:55:020
1058681993930,6cyclictest1060480-21kworker/3:010:15:033
1058681993925,10cyclictest306-21in:imuxsock10:05:233
1058681993922,13cyclictest1-21systemd12:40:003
105867899399,5cyclictest0-21swapper/208:23:552
105867899398,6cyclictest0-21swapper/212:06:582
105867899397,7cyclictest0-21swapper/207:39:242
1058678993931,5cyclictest1068937-21munin-plugin-st10:30:012
1058678993927,8cyclictest271-21cron08:20:002
1058678993924,9cyclictest1061477-21munin-plugin-st08:05:012
1058678993924,9cyclictest1061477-21munin-plugin-st08:05:002
1058678993910,5cyclictest0-21swapper/209:35:002
105867199395,13cyclictest12-21ksoftirqd/009:30:010
1058671993930,6cyclictest12-21ksoftirqd/011:30:020
1058671993927,8cyclictest12-21ksoftirqd/008:10:000
1058671993925,11cyclictest12-21ksoftirqd/008:35:010
1058671993924,11cyclictest12-21ksoftirqd/011:05:010
1058671993924,11cyclictest1071099-21sed11:10:010
1058671993922,13cyclictest1065583-21sshd09:22:240
105868199385,5cyclictest131rcu_preempt11:40:003
105868199385,5cyclictest131rcu_preempt11:40:003
1058681993827,8cyclictest1061735-21systemd-run08:10:003
1058681993827,5cyclictest306-21in:imuxsock12:15:003
105868199382,3cyclictest131rcu_preempt08:25:023
1058678993828,8cyclictest30-21ksoftirqd/209:55:012
1058678993826,9cyclictest1059658-21sshd07:27:232
1058678993826,8cyclictest30-21ksoftirqd/208:55:012
1058678993825,9cyclictest30-21ksoftirqd/209:15:022
1058678993825,9cyclictest30-21ksoftirqd/207:55:002
1058678993825,10cyclictest30-21ksoftirqd/210:05:002
1058678993821,13cyclictest1072237-21sshd11:31:222
1058678993820,14cyclictest30-21ksoftirqd/208:35:002
1058678993820,13cyclictest1-21systemd08:30:012
1058678993820,13cyclictest1-21systemd08:30:012
105867199383,5cyclictest1074059-21latency_hist12:10:020
1058671993832,3cyclictest0-21swapper/007:45:010
1058671993826,8cyclictest235-21systemd-journal08:40:000
1058671993814,20cyclictest151rcuc/010:55:010
105868199377,21cyclictest361rcuc/309:30:013
105868199376,6cyclictest131rcu_preempt12:00:013
1058681993723,11cyclictest37-21ksoftirqd/309:25:013
1058681993719,14cyclictest235-21systemd-journal11:25:023
1058678993728,7cyclictest30-21ksoftirqd/208:50:022
1058678993725,9cyclictest30-21ksoftirqd/212:15:022
1058671993730,4cyclictest16950irq/224-800000012:00:030
1058671993717,15cyclictest272-21dbus-daemon07:15:010
105867199371,3cyclictest0-21swapper/008:05:000
105867199371,3cyclictest0-21swapper/008:05:000
1058681993626,8cyclictest37-21ksoftirqd/311:05:013
1058681993625,6cyclictest306-21in:imuxsock07:15:013
1058681993621,8cyclictest37-21ksoftirqd/309:55:003
105868199361,32cyclictest37-21ksoftirqd/307:35:023
105867899362,7cyclictest30-21ksoftirqd/210:17:242
105867899362,7cyclictest30-21ksoftirqd/210:17:232
1058678993626,7cyclictest30-21ksoftirqd/207:10:012
1058678993626,5cyclictest306-21in:imuxsock08:06:232
1058678993622,9cyclictest0-21swapper/208:15:022
1058678993622,9cyclictest0-21swapper/208:15:022
1058678993622,11cyclictest30-21ksoftirqd/211:45:002
1058678993616,16cyclictest272-21dbus-daemon09:30:022
1058671993627,5cyclictest271-21cron10:39:590
1058671993625,9cyclictest12-21ksoftirqd/010:45:010
1058671993623,9cyclictest235-21systemd-journal08:14:230
1058671993623,9cyclictest235-21systemd-journal08:14:230
105867199362,30cyclictest0-21swapper/010:12:570
1058671993622,10cyclictest12-21ksoftirqd/010:50:020
1058671993611,23cyclictest19050irq/34-mmc111:25:020
1058681993528,4cyclictest1074831-21sshd12:24:233
1058681993528,4cyclictest1068935-21latency_hist10:30:003
1058681993525,8cyclictest37-21ksoftirqd/307:30:023
1058681993525,7cyclictest1061947-21sshd08:11:243
1058681993525,7cyclictest1061947-21sshd08:11:233
1058681993525,6cyclictest306-21in:imuxsock11:00:003
1058681993522,8cyclictest1075345-21grep12:35:003
1058681993520,11cyclictest1064566-21systemd-run09:05:013
1058681993518,13cyclictest0-21swapper/309:40:013
1058681993517,13cyclictest272-21dbus-daemon12:10:013
1058678993524,8cyclictest30-21ksoftirqd/211:15:012
1058678993524,8cyclictest30-21ksoftirqd/211:15:002
1058678993524,8cyclictest1065689-21sed09:25:012
1058678993522,10cyclictest30-21ksoftirqd/210:15:002
1058678993518,13cyclictest30-21ksoftirqd/207:16:232
105867899351,7cyclictest0-21swapper/207:44:452
105867199358,4cyclictest131rcu_preempt08:25:000
1058671993527,6cyclictest12-21ksoftirqd/007:25:010
1058671993526,6cyclictest19050irq/34-mmc112:12:120
1058671993523,9cyclictest12-21ksoftirqd/012:02:100
1058671993520,11cyclictest12-21ksoftirqd/008:30:020
1058671993520,11cyclictest12-21ksoftirqd/008:30:020
105867199351,4cyclictest0-21swapper/007:55:000
1058681993426,6cyclictest37-21ksoftirqd/311:35:013
1058681993423,8cyclictest37-21ksoftirqd/311:50:023
1058681993421,9cyclictest235-21systemd-journal10:52:233
1058681993421,9cyclictest0-21swapper/310:00:023
1058681993421,10cyclictest37-21ksoftirqd/307:55:003
1058681993420,9cyclictest306-21in:imuxsock10:20:013
1058681993420,9cyclictest306-21in:imuxsock10:20:013
1058681993420,11cyclictest37-21ksoftirqd/310:20:243
1058678993423,8cyclictest30-21ksoftirqd/210:00:002
1058678993418,10cyclictest1073030-21modprobe11:50:012
105867199341,30cyclictest0-21swapper/011:40:570
1058671993410,5cyclictest131rcu_preempt11:33:180
105868199335,9cyclictest37-21ksoftirqd/311:06:233
105868199334,3cyclictest131rcu_preempt07:10:023
1058681993322,6cyclictest1073800-21cron12:05:003
1058681993320,11cyclictest37-21ksoftirqd/310:35:003
1058681993319,10cyclictest0-21swapper/307:45:013
105867199331,29cyclictest0-21swapper/007:26:260
1058681993224,6cyclictest1067625-21sshd10:00:243
1058681993220,9cyclictest1-21systemd07:50:013
105867899322,5cyclictest150-21pr/ttyS210:43:462
105868199314,22cyclictest1063554-21latency_hist08:45:023
105868199314,22cyclictest1063554-21latency_hist08:45:013
105868199313,5cyclictest131rcu_preempt10:47:243
1058681993115,12cyclictest272-21dbus-daemon08:30:013
1058681993115,12cyclictest272-21dbus-daemon08:30:013
1058681993114,13cyclictest272-21dbus-daemon07:20:013
105867899313,7cyclictest30-21ksoftirqd/209:45:022
105867899311,3cyclictest0-21swapper/210:52:242
105867199311,3cyclictest0-21swapper/009:18:070
1058681992817,8cyclictest0-21swapper/308:35:013
105868199271,3cyclictest131rcu_preempt11:26:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional