You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-12-27 - 19:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Dec 27, 2024 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211042238164,60sleep10-21swapper/107:05:321
211042238164,60sleep10-21swapper/107:05:321
211992237163,23sleep30-21swapper/307:06:463
211992237163,23sleep30-21swapper/307:06:453
212402233161,24sleep20-21swapper/207:07:182
212402233161,24sleep20-21swapper/207:07:182
211382231161,56sleep00-21swapper/007:05:590
211382231161,56sleep00-21swapper/007:05:590
1459321400,3sleep32156199cyclictest08:05:153
68502580,2sleep20-21swapper/210:30:012
216122550,4sleep12154999cyclictest12:35:251
294772510,2sleep20-21swapper/210:00:142
113212510,1sleep20-21swapper/212:10:132
2154499367,14cyclictest3930-21munin-run10:20:010
2154499367,13cyclictest25028-21munin-run08:30:010
2154999321,16cyclictest0-21swapper/108:06:551
2154499313,13cyclictest26878-21munin-run09:55:020
2154499308,11cyclictest22836-21awk08:25:010
21544993012,15cyclictest24065-21munin-run09:45:010
123152300,5sleep121-21rcuc/110:40:201
241842290,2sleep00-21swapper/007:15:020
2154999291,17cyclictest0-21swapper/109:18:191
2154999291,17cyclictest0-21swapper/109:18:191
2154499292,12cyclictest6757-21perf10:30:010
2154999280,11cyclictest0-21swapper/110:15:571
2156199279,8cyclictest0-21swapper/307:24:313
2156199271,9cyclictest0-21swapper/312:00:273
2154999271,10cyclictest0-21swapper/110:21:471
2154999271,10cyclictest0-21swapper/110:21:471
2154499279,15cyclictest27757-21idleruntime-cro11:25:010
2156199261,10cyclictest0-21swapper/310:05:173
2154999266,12cyclictest0-21swapper/109:32:491
2154999261,13cyclictest0-21swapper/108:44:351
2154999261,12cyclictest0-21swapper/112:16:451
2154999261,10cyclictest0-21swapper/111:04:051
220202251,20sleep122105-21/usr/sbin/munin08:20:231
2156199257,9cyclictest0-21swapper/309:40:193
2156199257,8cyclictest7728-21cat12:00:023
2156199256,9cyclictest29066-21latency_hist10:00:023
2156199254,8cyclictest28170-40dnf11:27:343
2156199254,8cyclictest28170-40dnf11:27:343
2156199252,3cyclictest86750irq/128-enp1s0-07:45:353
2156199251,11cyclictest0-21swapper/311:40:133
2156199251,11cyclictest0-21swapper/307:58:313
2154999251,14cyclictest0-21swapper/107:52:531
2154999251,13cyclictest0-21swapper/107:24:131
2154999251,10cyclictest10252-21diskmemload11:16:031
2154999251,10cyclictest0-21swapper/111:50:171
184642250,5sleep02154499cyclictest12:30:140
184642250,5sleep02154499cyclictest12:30:140
2156199241,11cyclictest0-21swapper/308:45:193
2156199241,10cyclictest24605-21unixbench_singl08:25:253
2156199240,11cyclictest19528-21missed_timers11:00:193
21556992420,3cyclictest0-21swapper/208:43:422
2154999245,3cyclictest0-21swapper/110:05:391
2154999241,8cyclictest0-21swapper/111:21:011
2154999241,14cyclictest0-21swapper/107:55:171
21549992411,11cyclictest2541-21kworker/u8:009:20:081
2154999240,22cyclictest0-21swapper/110:46:071
2156199234,7cyclictest26140-21unixbench_multi07:15:253
2156199233,5cyclictest0-21swapper/311:50:003
2156199231,9cyclictest0-21swapper/307:27:453
2156199231,6cyclictest0-21swapper/311:18:503
2155699233,4cyclictest0-21swapper/209:16:102
2155699233,4cyclictest0-21swapper/209:16:102
2154999237,4cyclictest0-21swapper/111:32:131
2154999231,4cyclictest25684-21ntpq07:15:181
2154499235,3cyclictest86550irq/126-enp1s0-09:32:510
2154499232,5cyclictest21862-21munin-run09:40:010
2156199229,4cyclictest10878-21cstates12:10:103
2156199229,3cyclictest12050irq/123-ahci[0011:35:023
2156199228,3cyclictest0-21swapper/311:39:513
2156199228,3cyclictest0-21swapper/308:54:553
2156199227,6cyclictest0-21swapper/309:08:113
2156199226,3cyclictest0-21swapper/308:16:593
2156199226,3cyclictest0-21swapper/308:16:593
2156199223,8cyclictest0-21swapper/307:52:533
2156199222,8cyclictest27394-21cat08:35:023
2156199222,10cyclictest780-21rngd07:39:233
2156199221,9cyclictest0-21swapper/308:37:453
2156199221,11cyclictest0-21swapper/310:37:443
2156199221,10cyclictest0-21swapper/309:04:353
2156199221,10cyclictest0-21swapper/308:57:013
21556992218,3cyclictest0-21swapper/207:41:532
2155699221,6cyclictest0-21swapper/211:49:032
2155699221,5cyclictest0-21swapper/209:55:552
2155699221,5cyclictest0-21swapper/209:48:062
2155699221,2cyclictest0-21swapper/210:15:002
2154999229,2cyclictest21322-21timerwakeupswit09:35:271
2154999228,3cyclictest0-21swapper/112:14:331
2154999227,3cyclictest0-21swapper/112:28:391
2154999224,3cyclictest31722-21memory11:35:231
2154999223,3cyclictest0-21swapper/110:25:031
2154999223,2cyclictest86850irq/129-enp1s0-08:15:071
2154999223,2cyclictest86850irq/129-enp1s0-08:15:071
2154999222,11cyclictest6705-21ntp_states11:55:201
2154999221,3cyclictest7143-21ntp_states09:00:191
2154999221,19cyclictest0-21swapper/112:01:331
2154999221,19cyclictest0-21swapper/108:30:021
2154999221,18cyclictest28120-21smartctl08:35:151
2154999221,18cyclictest0-21swapper/112:05:141
2154999221,14cyclictest0-21swapper/111:07:511
2154999220,4cyclictest0-21swapper/108:26:311
2154999220,10cyclictest0-21swapper/110:51:071
123212221,3sleep00-21swapper/012:11:080
237042211,2sleep10-21swapper/109:40:361
2156199217,3cyclictest0-21swapper/309:45:053
2156199215,8cyclictest0-21swapper/308:01:153
2156199214,9cyclictest29643-21sh10:00:163
2156199213,8cyclictest86550irq/126-enp1s0-11:00:023
2156199212,2cyclictest836-21runrttasks11:22:333
2156199212,16cyclictest20951-21ntp_states09:35:223
2156199211,6cyclictest0-21swapper/312:15:233
2156199211,12cyclictest14634-21snmpd07:43:163
2156199211,11cyclictest0-21swapper/309:17:113
2156199211,11cyclictest0-21swapper/309:17:113
21561992110,3cyclictest14634-21snmpd12:33:423
21561992110,3cyclictest14634-21snmpd12:33:423
2156199210,9cyclictest0-21swapper/310:11:433
2155699211,6cyclictest0-21swapper/212:39:132
2154999216,3cyclictest31854-21gltestperf07:30:151
2154999214,4cyclictest0-21swapper/112:30:111
2154999214,4cyclictest0-21swapper/112:30:111
2154999212,3cyclictest0-21swapper/110:10:431
2154999211,17cyclictest0-21swapper/110:30:131
2154999211,15cyclictest0-21swapper/107:40:151
2154999211,14cyclictest27474-21fschecks_count09:55:131
2154999211,14cyclictest0-21swapper/109:27:311
2154999211,13cyclictest0-21swapper/110:01:231
21549992111,3cyclictest0-21swapper/108:49:111
2154999211,12cyclictest0-21swapper/111:10:151
2154999211,11cyclictest0-21swapper/111:27:331
2154999211,11cyclictest0-21swapper/111:27:331
2154999211,11cyclictest0-21swapper/107:10:171
21549992110,3cyclictest18335-21latency_hist11:00:021
2154999210,12cyclictest0-21swapper/109:10:011
2154499215,8cyclictest10252-21diskmemload11:00:110
2154499212,11cyclictest6962-21/usr/sbin/munin10:30:190
2154499211,11cyclictest0-21swapper/011:55:110
108362211,2sleep10-21swapper/110:36:201
2156199209,2cyclictest88250irq/135-enp2s0-08:14:453
2156199206,3cyclictest0-21swapper/309:20:473
2156199205,3cyclictest0-21swapper/307:33:113
2156199202,4cyclictest0-21swapper/310:46:573
2156199201,9cyclictest0-21swapper/310:18:273
2156199201,8cyclictest0-21swapper/312:07:313
2156199201,11cyclictest12028-21tail10:40:163
21561992010,2cyclictest86850irq/129-enp1s0-07:10:173
2156199200,9cyclictest24892-21chrt09:50:043
2156199200,7cyclictest0-21swapper/308:20:413
2155699205,5cyclictest32765-21sh11:40:012
2155699201,7cyclictest0-21swapper/212:34:112
2155699201,7cyclictest0-21swapper/212:34:112
2155699201,4cyclictest0-21swapper/209:25:522
2155699201,3cyclictest0-21swapper/211:30:052
21556992013,4cyclictest22722-21grep09:40:142
2154999208,3cyclictest18579-21cat08:15:011
2154999202,6cyclictest0-21swapper/109:46:031
2154999201,16cyclictest0-21swapper/111:48:571
2154999201,14cyclictest0-21swapper/109:11:091
2154999201,13cyclictest4046-21cstates08:55:131
2154999201,11cyclictest0-21swapper/111:40:071
2154999201,10cyclictest581-21systemd-journal07:50:021
2154499209,3cyclictest3263-21/usr/sbin/munin07:40:180
2154499209,3cyclictest20061-21/usr/sbin/munin12:35:230
2154499206,6cyclictest0-21swapper/010:36:470
2154499206,5cyclictest0-21swapper/011:49:010
2154499206,5cyclictest0-21swapper/011:06:240
2154499203,8cyclictest0-21swapper/012:00:110
2154499202,5cyclictest17488-40dnf12:28:120
2154499201,10cyclictest0-21swapper/011:36:070
32982191,2sleep00-21swapper/008:50:370
2156199194,3cyclictest14822-21gltestperf10:50:133
2156199193,2cyclictest780-21rngd09:10:163
2156199192,3cyclictest0-21swapper/311:14:443
2156199191,9cyclictest86550irq/126-enp1s0-10:30:213
21561991914,3cyclictest20432-21grep12:35:133
2156199191,12cyclictest0-21swapper/312:20:183
2156199191,11cyclictest0-21swapper/310:20:233
2156199191,11cyclictest0-21swapper/310:20:233
2156199191,11cyclictest0-21swapper/308:40:143
2156199191,10cyclictest3442-21/usr/sbin/munin11:50:153
2156199191,10cyclictest0-21swapper/310:27:123
2155699196,4cyclictest1249-21grep11:40:172
2155699191,8cyclictest4947-21ntp_states08:55:212
2155699191,7cyclictest0-21swapper/209:11:412
2155699191,6cyclictest0-21swapper/212:16:072
2155699191,6cyclictest0-21swapper/207:45:152
2155699191,5cyclictest0-21swapper/210:44:582
21556991913,3cyclictest15432-21fschecks_count12:20:162
21556991912,4cyclictest25252-21diskstats09:50:102
2155699191,10cyclictest0-21swapper/211:50:432
2155699191,10cyclictest0-21swapper/211:18:552
2155699191,10cyclictest0-21swapper/211:06:062
2154999197,3cyclictest0-21swapper/107:25:171
21549991915,3cyclictest0-21swapper/112:20:101
2154999190,9cyclictest0-21swapper/109:53:331
2154999190,18cyclictest0-21swapper/108:50:581
2154999190,18cyclictest0-21swapper/107:36:301
2154499199,3cyclictest14043-21awk08:05:010
2154499198,3cyclictest12682-21/usr/sbin/munin09:15:240
2154499198,3cyclictest12682-21/usr/sbin/munin09:15:240
2154499197,6cyclictest0-21swapper/010:08:180
2154499196,5cyclictest0-21swapper/010:44:080
2154499196,4cyclictest1165-21cpuspeed_turbos07:35:140
2154499195,7cyclictest28081-21kworker/u8:111:42:410
2154499193,9cyclictest836-21runrttasks11:27:340
2154499193,9cyclictest836-21runrttasks11:27:340
2154499193,6cyclictest0-21swapper/010:54:380
2154499191,9cyclictest5139-21kworker/u8:112:05:180
2154499191,9cyclictest3780-21df11:50:110
21544991914,3cyclictest18192-21sshd08:12:110
21544991912,4cyclictest31050-21sshd10:02:110
2154499191,13cyclictest0-21swapper/011:16:370
2154499191,10cyclictest31227-21/usr/sbin/munin07:30:180
2154499191,10cyclictest29645-21/usr/sbin/munin08:40:160
2154499191,10cyclictest18651-21/usr/sbin/munin08:15:190
2154499191,10cyclictest18651-21/usr/sbin/munin08:15:190
2154499191,10cyclictest14878-21/usr/sbin/munin09:20:150
2154499191,10cyclictest0-21swapper/010:47:130
2154499191,10cyclictest0-21swapper/009:46:050
2154499191,10cyclictest0-21swapper/009:11:060
2154499190,10cyclictest0-21swapper/010:57:330
164852191,2sleep20-21swapper/209:20:292
55682181,2sleep00-21swapper/008:56:350
29122181,2sleep20-21swapper/207:38:372
235342181,3sleep20-21swapper/207:12:562
2156199185,3cyclictest0-21swapper/309:27:513
2156199184,8cyclictest0-21swapper/311:05:043
2156199180,11cyclictest0-21swapper/309:32:153
2155699182,6cyclictest26768-21cat07:20:022
2155699181,7cyclictest22598-21processes11:10:192
2155699181,7cyclictest0-21swapper/212:08:482
2155699181,7cyclictest0-21swapper/210:58:222
2155699181,7cyclictest0-21swapper/210:08:232
2155699181,7cyclictest0-21swapper/208:29:342
2155699181,4cyclictest0-21swapper/211:22:572
2155699181,3cyclictest0-21swapper/210:32:362
21556991812,4cyclictest28394-21head07:20:302
21556991812,3cyclictest6117-21crond10:25:012
21556991812,3cyclictest6117-21crond10:25:002
2155699180,9cyclictest13405-21cat08:00:252
2154999187,3cyclictest12965-21memory08:00:191
2154499186,3cyclictest0-21swapper/012:24:350
2154499186,3cyclictest0-21swapper/011:13:250
2154499185,5cyclictest0-21swapper/012:17:580
2154499181,9cyclictest0-21swapper/009:55:110
2154499181,8cyclictest4155-21/usr/sbin/munin10:20:140
2154499181,8cyclictest4155-21/usr/sbin/munin10:20:140
2154499181,6cyclictest0-21swapper/010:12:100
2154499181,14cyclictest26950-21cut07:20:120
138062181,2sleep20-21swapper/210:48:242
215832171,2sleep20-21swapper/209:36:362
2155699179,2cyclictest86850irq/129-enp1s0-07:25:202
2155699175,3cyclictest29425-21perf08:40:002
2155699174,3cyclictest758-21sh07:35:002
2155699171,7cyclictest0-21swapper/207:50:212
2155699171,6cyclictest7227-21users11:55:242
2155699171,5cyclictest14634-21snmpd12:03:372
2155699171,5cyclictest0-21swapper/211:00:522
21556991712,3cyclictest3029-21ntp_states10:15:212
21556991712,3cyclictest26324-21sh08:30:192
21556991712,3cyclictest17278-21kernelversion08:10:162
2155699171,15cyclictest0-21swapper/210:36:572
21556991711,3cyclictest1679-21/usr/sbin/munin08:50:122
2154499179,2cyclictest0-21swapper/011:30:180
2154499176,3cyclictest86550irq/126-enp1s0-08:45:270
2154499175,4cyclictest21907-21grep07:10:120
2154499174,4cyclictest10255-21sadc09:10:010
2154499174,3cyclictest0-21swapper/009:29:310
2154499171,9cyclictest0-21swapper/007:55:160
21544991712,3cyclictest1-21systemd08:30:100
2154499171,15cyclictest0-21swapper/007:25:240
204722171,3sleep20-21swapper/208:18:192
204722171,3sleep20-21swapper/208:18:192
160422171,2sleep20-21swapper/208:07:512
116132171,2sleep20-21swapper/207:57:312
227682161,2sleep20-21swapper/208:24:292
2155699161,6cyclictest8620-21df09:05:142
2155699161,4cyclictest4977-1kworker/u9:112:25:192
2155699161,4cyclictest18289-21grep09:30:112
2155699161,4cyclictest0-21swapper/211:25:392
2155699161,4cyclictest0-21swapper/211:25:392
21556991611,3cyclictest6232-21cut09:00:092
21556991610,4cyclictest31860-21bc08:45:112
21556991610,3cyclictest15229-21ntp_offset10:50:172
2154499166,3cyclictest86750irq/128-enp1s0-07:50:170
21544991612,3cyclictest0-21swapper/008:36:500
21544991611,3cyclictest5459-21/usr/sbin/munin07:45:120
21544991610,3cyclictest6302-21expr09:00:100
173372160,3sleep30-21swapper/312:26:163
21544991510,3cyclictest14946-21hwlatdetect08:05:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional