You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 23:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Thu Apr 03, 2025 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
258092235161,23sleep10-21swapper/107:07:491
259312205167,25sleep20-21swapper/207:09:242
256722203165,25sleep30-21swapper/307:06:033
256642202163,26sleep00-21swapper/007:05:540
170052600,3sleep011-21rcuc/009:20:180
298952580,4sleep12607999cyclictest11:30:151
154002560,2sleep20-21swapper/207:55:232
2608599402,27cyclictest16564-21/usr/sbin/munin08:00:132
2608599394,30cyclictest1556-40dnf11:38:402
2608599394,24cyclictest780-21rngd09:00:132
2608599385,28cyclictest32128-40dnf08:37:342
2608599381,27cyclictest21202-21/usr/sbin/munin09:35:212
2608599381,25cyclictest5582-21/usr/sbin/munin07:35:192
2608599381,25cyclictest13297-21gltestperf10:40:162
2608599371,26cyclictest25358-21/usr/sbin/munin08:20:262
2608599371,25cyclictest1042-21/usr/sbin/munin10:10:222
2608599371,22cyclictest8975-21idleruntime10:30:182
2608599363,21cyclictest19425-21chrt10:57:282
2608599361,20cyclictest6768-21latency_hist11:55:022
2608599361,20cyclictest4563-21switchtime10:15:262
2608599354,15cyclictest12050irq/123-ahci[0008:15:022
2608599351,22cyclictest1167-21/usr/sbin/munin07:25:202
2609199344,11cyclictest17464-21/usr/sbin/munin09:20:263
2608599341,24cyclictest24626-21gunzip09:40:242
2608599341,23cyclictest0-21swapper/212:10:222
2608599341,22cyclictest836-21runrttasks10:35:422
2608599341,22cyclictest32010-21timerwakeupswit10:00:262
2608599341,22cyclictest27282-21ntp_states09:50:192
2608599341,20cyclictest12562-21/usr/sbin/munin12:15:282
2608599341,16cyclictest12853-21grep09:10:212
2607499346,13cyclictest27953-21awk08:30:000
2608599335,18cyclictest0-21swapper/212:33:372
2608599333,23cyclictest86850irq/129-enp1s0-09:20:252
2608599332,21cyclictest7781-21/usr/sbin/munin07:40:212
2608599331,28cyclictest21677-40dnf12:39:322
2608599331,18cyclictest26443-21sed11:15:262
2608599321,4cyclictest22625-21latency_hist11:10:022
2608599321,18cyclictest25994-21perf09:50:012
2608599321,18cyclictest25994-21perf09:50:012
2608599321,18cyclictest11578-21diskmemload11:27:252
2608599321,16cyclictest0-21swapper/210:10:022
2608599319,19cyclictest937-21latency_hist07:25:002
2608599319,19cyclictest3445-21cpu07:30:182
2608599315,23cyclictest19010-21/usr/sbin/munin09:30:292
2608599315,15cyclictest6899-21sshd08:56:372
2608599313,17cyclictest11578-21diskmemload11:43:412
2608599311,27cyclictest31481-21/usr/sbin/munin11:30:292
2608599311,27cyclictest23770-21head08:15:162
2608599311,26cyclictest11235-21smart_sda07:45:252
2608599311,20cyclictest9523-21/usr/sbin/munin12:05:022
2608599311,20cyclictest0-21swapper/212:07:392
2608599311,19cyclictest28372-21switchtime11:20:272
2608599311,18cyclictest11578-21diskmemload12:20:102
2608599311,17cyclictest0-21swapper/209:28:532
2607499317,11cyclictest11573-21idleruntime-cro09:10:010
2608599302,25cyclictest16584-21ls10:50:242
2608599302,25cyclictest16584-21ls10:50:242
26085993013,14cyclictest29160-21/usr/sbin/munin07:15:182
2608599301,26cyclictest19881-21/usr/sbin/munin11:00:132
2608599301,26cyclictest12294-21tr07:50:172
2608599301,25cyclictest28234-21latency_hist07:15:022
2608599301,25cyclictest14421-21/usr/sbin/munin09:15:152
2608599301,20cyclictest18765-21/usr/sbin/munin08:05:152
2608599301,20cyclictest0-21swapper/212:28:402
2608599301,13cyclictest0-21swapper/210:30:012
26085993010,17cyclictest28241-21sort08:30:112
26074993011,16cyclictest10241-21munin-run10:35:000
2608599292,23cyclictest6523-21ntp_states10:20:222
2608599291,24cyclictest24678-21ls11:15:022
2608599291,17cyclictest0-21swapper/211:57:292
2608599291,17cyclictest0-21swapper/211:57:292
75952280,1sleep00-21swapper/010:25:010
2608599281,23cyclictest4993-21idleruntime-cro08:55:012
2608599281,23cyclictest28880-21grep09:55:172
26085992812,13cyclictest9613-21/usr/sbin/munin09:05:272
2609199274,13cyclictest19822-21grep12:35:123
2609199270,13cyclictest0-21swapper/308:19:203
2608599271,23cyclictest836-21runrttasks10:48:142
2608599271,23cyclictest32732-21cstates08:40:132
2608599271,23cyclictest32732-21cstates08:40:132
2607999270,18cyclictest4871-21tr11:50:131
2607999270,16cyclictest0-21swapper/109:01:351
2609199267,7cyclictest7404-21/usr/sbin/munin09:00:343
2609199266,7cyclictest0-21swapper/310:50:273
2609199266,7cyclictest0-21swapper/310:50:273
2609199264,8cyclictest86650irq/127-enp1s0-12:10:153
2608599261,22cyclictest836-21runrttasks11:47:272
2608599261,22cyclictest836-21runrttasks08:47:352
2607999262,20cyclictest15247-21munin-run10:50:001
2609199258,8cyclictest30181-21sshd09:57:373
2609199255,10cyclictest12562-21/usr/sbin/munin12:15:123
2609199254,13cyclictest23735-21df09:40:133
2609199252,11cyclictest11578-21diskmemload11:49:503
2609199251,10cyclictest15218-21/usr/sbin/munin09:15:233
2608599251,21cyclictest581-21systemd-journal08:26:372
2607999259,14cyclictest7119-21kworker/u8:212:01:421
2607999257,10cyclictest13125-21sort09:10:241
2607999251,9cyclictest15829-21ntp_states12:20:221
2607999251,18cyclictest0-21swapper/110:01:391
2607999251,12cyclictest12250-21taskset10:37:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional