You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 04:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Wed Apr 02, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
87248099295112,2936cyclictest917446-21kworker/2:1+events21:40:182
87248299280612,2789cyclictest870336-21kworker/3:2+events20:00:133
87247599274412,2729cyclictest930663-21kworker/1:1+events22:30:161
87247299268613,2669cyclictest930605-21kworker/0:1+events23:30:120
87247599261912,2603cyclictest890593-21kworker/1:0+events20:10:101
87248299260112,2586cyclictest939332-21kworker/3:2+events23:20:243
8724759925476,2538cyclictest899425-21kworker/1:2+events20:45:201
87247599245512,2439cyclictest875800-21kworker/1:0+events19:25:171
87248299245212,2436cyclictest948448-21kworker/3:1+events23:44:383
87248299244511,2426cyclictest896044-21kworker/3:2+events20:32:413
87248299243812,2423cyclictest870336-21kworker/3:2+events19:36:203
87247599242812,2412cyclictest910274-21kworker/1:1+events21:15:151
87248299242612,2410cyclictest896044-21kworker/3:2+events20:38:253
8724829924128,2399cyclictest870336-21kworker/3:2+events19:20:193
87247299241012,2395cyclictest917566-21kworker/0:0+events21:50:150
87247299240512,2389cyclictest925571-21kworker/0:2+events22:25:190
87247599239210,2379cyclictest881782-21kworker/1:1+events19:46:201
87248099238812,2372cyclictest905528-21kworker/2:0+events21:03:042
87248099234112,2325cyclictest898143-21kworker/2:0+events20:41:282
87247599226411,2250cyclictest899425-21kworker/1:2+events20:40:201
87247599225410,2241cyclictest953839-21kworker/1:2+events00:03:231
8724829922448,2231cyclictest883818-21kworker/3:0+events19:46:423
87248299224110,2227cyclictest870336-21kworker/3:2+events19:58:453
87248299224010,2226cyclictest939332-21kworker/3:2+events23:07:263
87248299223810,2225cyclictest964776-21kworker/3:0+events00:37:183
87248299223810,2224cyclictest959754-21kworker/3:2+events00:23:553
8724829922379,2224cyclictest917346-21kworker/3:1+events21:51:483
8724759922337,2222cyclictest955819-21kworker/1:3+events00:07:591
87247599223310,2220cyclictest925513-21kworker/1:2+events22:12:421
8724829922329,2219cyclictest941692-21kworker/3:0+events23:18:523
87248299223111,2217cyclictest957254-21kworker/3:2+events00:06:513
87248299223111,2216cyclictest946781-21kworker/3:0+events23:38:573
87248299222911,2215cyclictest939332-21kworker/3:2+events23:25:593
87248299222810,2214cyclictest905757-21kworker/3:0+events21:14:063
8724759922279,2214cyclictest919680-21kworker/1:1+events22:08:571
87247599222711,2213cyclictest888289-21kworker/1:0+events19:59:071
87248299222612,2210cyclictest928444-21kworker/3:1+events22:18:493
87247599222611,2211cyclictest875800-21kworker/1:0+events19:22:591
87247599222610,2213cyclictest966348-21kworker/1:2+events00:36:551
87247599222410,2211cyclictest901109-21kworker/1:1+events21:00:491
8724829922236,2214cyclictest870336-21kworker/3:2+events19:30:383
87248299222311,2210cyclictest951695-21kworker/3:2+events23:45:253
8724759922239,2210cyclictest947049-21kworker/1:1+events23:38:341
8724759922237,2213cyclictest901109-21kworker/1:1+events20:57:501
87247599222311,2208cyclictest933867-21kworker/1:1+events22:37:281
87247599222111,2207cyclictest901109-21kworker/1:1+events20:51:031
87248299221911,2205cyclictest905757-21kworker/3:0+events21:46:483
8724759922147,2204cyclictest941372-21kworker/1:2+events23:15:461
8724759922139,2201cyclictest947049-21kworker/1:1+events23:43:521
87248099221212,2197cyclictest868649-21kworker/2:1+events19:22:142
87247599221210,2199cyclictest881782-21kworker/1:1+events19:40:531
87248299221110,2198cyclictest926843-21kworker/3:2+events22:31:023
8724829922079,2194cyclictest870336-21kworker/3:2+events20:08:393
87248299220710,2193cyclictest877306-21kworker/3:1+events19:29:093
87247599220610,2192cyclictest919680-21kworker/1:1+events22:03:151
87248299220510,2192cyclictest963175-21kworker/3:1+events00:31:093
87247599220510,2192cyclictest943028-21kworker/1:1+events23:12:201
87248299220211,2188cyclictest926843-21kworker/3:2+events22:25:413
8724759922027,2191cyclictest946401-21kworker/1:2+events23:29:261
87247599220111,2186cyclictest881782-21kworker/1:1+events19:54:141
87248299220011,2186cyclictest870336-21kworker/3:2+events20:15:553
87248299219911,2185cyclictest870336-21kworker/3:2+events20:12:033
87247599219911,2186cyclictest943028-21kworker/1:1+events23:20:021
87247599219911,2186cyclictest927170-21kworker/1:0+events22:26:031
87247599219810,2185cyclictest927170-21kworker/1:0+events22:41:171
8724759921948,2183cyclictest907109-21kworker/1:0+events21:06:291
87247599218910,2176cyclictest879234-21kworker/1:2+events19:32:301
87248299218710,2174cyclictest905757-21kworker/3:0+events21:02:413
87247599218610,2173cyclictest905489-21kworker/1:2+events21:23:121
87248299218510,2172cyclictest925023-21kworker/3:1+events22:08:343
8724829921848,2173cyclictest933894-21kworker/3:1+events22:48:573
87248299218211,2167cyclictest939332-21kworker/3:2+events23:30:353
8724759921767,2166cyclictest947049-21kworker/1:1+events23:51:341
87248299216910,2155cyclictest905757-21kworker/3:0+events21:42:123
87248299216710,2153cyclictest896044-21kworker/3:2+events20:58:353
8724829921657,2155cyclictest930070-21kworker/3:0+events22:38:373
87248099216012,2145cyclictest898143-21kworker/2:0+events20:39:582
87247299215911,2145cyclictest888562-21kworker/0:0+events20:17:040
87248299215411,2140cyclictest895360-21kworker/3:1+events20:30:003
87248099215411,2140cyclictest917446-21kworker/2:1+events21:45:372
87248099215311,2138cyclictest962559-21kworker/2:1+events00:21:592
87248299214811,2133cyclictest882105-21kworker/3:1+events19:44:243
8724829921476,2137cyclictest948448-21kworker/3:1+events23:55:453
87248099214111,2126cyclictest911966-21kworker/2:2+events21:34:582
87248099214111,2126cyclictest886608-21kworker/2:0+events19:52:442
87248099213910,2126cyclictest957497-21kworker/2:2+events00:07:142
8724759921399,2127cyclictest919680-21kworker/1:1+events21:52:111
87248099213811,2124cyclictest889863-21kworker/2:0+events20:06:452
8724809921349,2122cyclictest878848-21kworker/2:0+events19:34:252
8724759921346,2125cyclictest875800-21kworker/1:0+events19:35:341
8724809921326,2123cyclictest917446-21kworker/2:1+events21:55:342
87248099212211,2107cyclictest940528-21kworker/2:1+events23:09:182
87248099211711,2104cyclictest932366-21kworker/2:1+events22:35:092
87248099211711,2103cyclictest890547-21kworker/2:2+events20:11:172
87248299211611,2103cyclictest957941-21kworker/3:0+events00:25:013
87247599211310,2100cyclictest952234-21kworker/1:0+events23:57:141
87248299211210,2099cyclictest939332-21kworker/3:2+events23:12:433
87248099211010,2097cyclictest946870-21kworker/2:0+events23:28:172
87248099211010,2096cyclictest955589-21kworker/2:0+events00:01:082
87247599210911,2094cyclictest937891-21kworker/1:2+events22:56:061
8724759921079,2094cyclictest919025-21kworker/1:2+events21:47:561
87247299210610,2092cyclictest950322-21kworker/0:0+events00:02:140
8724759920989,2086cyclictest905489-21kworker/1:2+events21:13:221
8724759920968,2085cyclictest919025-21kworker/1:2+events21:41:501
87247599209310,2079cyclictest881782-21kworker/1:1+events20:03:401
8724809920908,2078cyclictest903925-21kworker/2:1+events21:14:302
8724829920879,2074cyclictest905757-21kworker/3:0+events21:27:183
87247599208310,2070cyclictest955819-21kworker/1:3+events00:12:111
8724829920825,2074cyclictest948448-21kworker/3:1+events23:50:483
87248299208210,2069cyclictest937742-21kworker/3:0+events22:57:563
8724809920828,2071cyclictest943001-21kworker/2:2+events23:15:232
87247599208110,2067cyclictest962761-21kworker/1:2+events00:28:301
8724829920804,2073cyclictest873650-21kworker/3:1+events19:17:383
87248299208010,2067cyclictest959754-21kworker/3:2+events00:17:043
8724759920809,2068cyclictest946401-21kworker/1:2+events23:34:231
8724759920809,2068cyclictest913554-21kworker/1:0+events21:27:411
87247599208010,2067cyclictest927170-21kworker/1:0+events22:21:071
8724829920796,2070cyclictest933894-21kworker/3:1+events22:50:503
8724759920797,2069cyclictest962761-21kworker/1:2+events00:24:391
87248299207810,2065cyclictest905757-21kworker/3:0+events21:38:233
87248299207810,2065cyclictest900841-21kworker/3:0+events20:43:013
8724759920789,2066cyclictest925513-21kworker/1:2+events22:19:591
8724829920779,2064cyclictest870336-21kworker/3:2+events20:20:513
8724829920778,2065cyclictest907638-21kworker/3:2+events21:07:163
8724759920779,2066cyclictest890593-21kworker/1:0+events20:25:041
8724759920779,2065cyclictest890593-21kworker/1:0+events20:36:291
87247599207710,2064cyclictest936235-21kworker/1:0+events23:01:201
87247599207710,2064cyclictest921319-21kworker/1:0+events21:55:121
8724829920768,2065cyclictest870336-21kworker/3:2+events19:50:513
87248299207510,2061cyclictest928444-21kworker/3:1+events22:20:213
8724759920759,2062cyclictest941372-21kworker/1:2+events23:05:541
87247599207410,2061cyclictest961149-21kworker/1:1+events00:31:541
8724829920739,2060cyclictest921521-21kworker/3:0+events22:03:383
8724829920739,2060cyclictest905757-21kworker/3:0+events21:30:453
8724829920739,2060cyclictest896044-21kworker/3:2+events20:48:433
8724829920735,2065cyclictest957254-21kworker/3:2+events00:04:313
8724829920732,2068cyclictest913463-21kworker/3:2+events21:24:163
8724829920718,2059cyclictest939332-21kworker/3:2+events23:03:143
87248299207110,2057cyclictest919888-21kworker/3:2+events21:59:463
87247599207111,2057cyclictest935520-21kworker/1:2+events22:45:301
87248299207010,2056cyclictest926843-21kworker/3:2+events22:11:343
8724759920699,2057cyclictest890593-21kworker/1:0+events20:15:301
87247599206910,2056cyclictest864039-21kworker/1:1+events19:14:181
87248299206810,2055cyclictest911756-21kworker/3:1+events21:16:463
87247599206710,2055cyclictest881782-21kworker/1:1+events20:08:161
87247599206710,2053cyclictest951578-21kworker/1:2+events23:45:481
87247599206610,2053cyclictest933867-21kworker/1:1+events22:52:191
8724759920658,2054cyclictest895182-21kworker/1:2+events20:30:471
87247599206510,2052cyclictest957970-21kworker/1:0+events00:18:121
8724759920615,2053cyclictest914262-21kworker/1:1+events21:37:401
8724729920547,2044cyclictest908645-21kworker/0:0+events21:19:460
87247599205110,2039cyclictest873766-21kworker/1:2+events19:15:021
87248099204510,2032cyclictest903925-21kworker/2:1+events21:17:082
87248299204310,2030cyclictest930070-21kworker/3:0+events22:40:303
8724729920439,2031cyclictest897122-21kworker/0:2+events20:38:480
87247299204310,2030cyclictest930605-21kworker/0:1+events23:13:290
8724809920428,2030cyclictest903925-21kworker/2:1+events20:52:112
8724809920405,2032cyclictest878848-21kworker/2:0+events19:42:512
8724809920368,2025cyclictest880477-21kworker/2:1+events19:37:282
8724809920258,2014cyclictest915858-21kworker/2:0+events21:36:072
8724829920218,2010cyclictest948448-21kworker/3:1+events00:12:343
8724759920195,2010cyclictest890593-21kworker/1:0+events20:23:331
8724829920183,2011cyclictest896044-21kworker/3:2+events20:52:583
8724729920129,2000cyclictest928597-21kworker/0:0+events22:20:440
8724829920049,1991cyclictest870336-21kworker/3:2+events19:12:463
8724809920019,1989cyclictest883778-21kworker/2:1+events20:00:352
8724809920019,1988cyclictest873592-21kworker/2:0+events19:15:242
87248099200112,1986cyclictest925297-21kworker/2:0+events22:13:272
8724809920006,1991cyclictest964803-21kworker/2:1+events00:32:182
8724809919994,1991cyclictest925297-21kworker/2:0+events22:08:112
8724809919973,1990cyclictest920959-21kworker/2:2+events21:54:042
8724809919968,1985cyclictest950004-21kworker/2:2+events23:48:302
8724809919968,1985cyclictest940528-21kworker/2:1+events23:10:492
8724809919967,1986cyclictest908735-21kworker/2:2+events21:09:522
87248099199610,1983cyclictest946226-21kworker/2:1+events23:37:242
8724759919969,1984cyclictest905489-21kworker/1:2+events21:34:131
8724809919929,1980cyclictest893757-21kworker/2:2+events20:21:372
87248099198810,1974cyclictest883778-21kworker/2:1+events20:18:112
8724809919877,1977cyclictest952353-21kworker/2:1+events00:13:192
87248099198710,1974cyclictest946870-21kworker/2:0+events23:32:502
87248099198610,1973cyclictest883778-21kworker/2:1+events19:47:042
8724809919855,1976cyclictest928517-21kworker/2:1+events22:24:322
87248099198510,1972cyclictest925297-21kworker/2:0+events22:15:242
8724809919848,1972cyclictest950004-21kworker/2:2+events23:42:432
8724809919839,1971cyclictest930784-21kworker/2:2+events22:32:092
8724809919833,1976cyclictest911966-21kworker/2:2+events21:20:332
87248099198310,1970cyclictest937912-21kworker/2:0+events22:59:052
87248099198310,1970cyclictest932366-21kworker/2:1+events22:41:412
8724809919825,1974cyclictest933966-21kworker/2:2+events22:46:392
8724809919825,1974cyclictest930784-21kworker/2:2+events22:29:532
8724809919813,1974cyclictest963214-21kworker/2:2+events00:27:432
87248099198110,1968cyclictest883778-21kworker/2:1+events19:57:372
8724809919808,1969cyclictest917446-21kworker/2:1+events22:00:562
87248099197810,1964cyclictest950004-21kworker/2:2+events23:58:012
8724809919753,1969cyclictest868649-21kworker/2:1+events19:26:522
87248099197510,1961cyclictest913597-21kworker/2:0+events21:28:272
87248099197410,1961cyclictest940528-21kworker/2:1+events23:02:062
8724809919736,1963cyclictest950004-21kworker/2:2+events23:51:102
8724809919728,1961cyclictest903925-21kworker/2:1+events20:58:592
8724809919646,1954cyclictest933966-21kworker/2:2+events22:53:052
8724729919644,1957cyclictest870531-21kworker/0:0+events19:45:100
8724809919639,1951cyclictest944604-21kworker/2:0+events23:21:092
8724729919633,1957cyclictest895392-21kworker/0:1+events20:30:220
8724729919608,1949cyclictest902354-21kworker/0:1+events21:06:070
87248099195911,1945cyclictest960503-21kworker/2:0+events00:36:312
8724729919599,1947cyclictest944840-21kworker/0:0+events23:23:010
8724729919578,1946cyclictest957283-21kworker/0:1+events00:24:180
87247299195110,1937cyclictest899363-21kworker/0:0+events20:46:290
87247299194610,1934cyclictest870531-21kworker/0:0+events19:35:110
8724809919409,1928cyclictest957497-21kworker/2:2+events00:18:572
87247299193910,1925cyclictest890724-21kworker/0:2+events20:10:530
8724809919369,1923cyclictest898143-21kworker/2:0+events20:47:362
8724809919198,1908cyclictest895385-21kworker/2:1+events20:25:502
8724809919159,1903cyclictest868649-21kworker/2:1+events19:12:032
8724729919089,1896cyclictest912079-21kworker/0:2+events21:40:410
8724729919066,1896cyclictest930605-21kworker/0:1+events23:19:380
87247299190410,1891cyclictest886783-21kworker/0:1+events20:05:130
8724729918843,1878cyclictest888562-21kworker/0:0+events20:24:410
8724729918819,1869cyclictest925571-21kworker/0:2+events22:18:040
8724729918799,1867cyclictest920218-21kworker/0:1+events22:09:420
8724729918299,1817cyclictest947188-21kworker/0:2+events23:40:500
8724729918159,1803cyclictest883779-21kworker/0:2+events19:53:510
8720172187157,25sleep10-21swapper/119:07:561
8719162140110,25sleep00-21swapper/019:06:390
8718402140106,28sleep20-21swapper/219:05:402
8719942139107,27sleep30-21swapper/319:07:383
87247299130127,2cyclictest870531-21kworker/0:0+pm19:45:010
872472998582,2cyclictest870531-21kworker/0:0+pm19:35:010
87247299208,8cyclictest950286-21awk23:40:000
872472992016,3cyclictest0-21swapper/022:00:010
872472991915,3cyclictest658-21in:imuxsock19:15:010
872480991812,4cyclictest897864-21runrttasks20:34:342
87247299178,6cyclictest901084-21perf20:45:000
87247299178,6cyclictest896078-21apt-get20:30:000
87247299177,7cyclictest961324-21plymouth00:20:000
872472991712,3cyclictest875816-21runrttasks19:19:580
872472991712,3cyclictest629-21avahi-daemon21:15:000
87247299170,16cyclictest933784-21taskset22:35:290
87247299170,16cyclictest798-21runrttasks23:02:150
87247299162,13cyclictest955559-21taskset23:59:060
872472991612,2cyclictest941740-21munin-run23:10:000
872472991611,3cyclictest908-10wireplumber22:05:010
87247299160,14cyclictest912947-21/usr/sbin/munin21:20:180
87247299151,7cyclictest920004-21awk21:50:010
872472991513,1cyclictest0-21swapper/021:05:000
872472991511,2cyclictest0-21swapper/021:35:000
87247299151,10cyclictest888479-21munin-run20:00:010
872472991510,2cyclictest966689-21perf00:40:000
87247299150,3cyclictest916809-21/usr/sbin/munin21:35:200
87247299150,14cyclictest937019-21/usr/sbin/munin22:50:120
87247299150,13cyclictest926355-21head22:10:120
87247299147,4cyclictest932588-21awk22:35:000
87247299146,6cyclictest958167-21idleruntime-cro00:09:590
87247299146,6cyclictest0-21swapper/021:30:000
87247299145,6cyclictest947133-21awk23:30:010
87247299143,5cyclictest878617-21users19:25:220
87247299143,3cyclictest953242-21ntpq23:50:120
872472991412,1cyclictest0-21swapper/000:12:180
87247299136,5cyclictest890079-21perf20:05:010
87247299134,5cyclictest876120-21apt-get19:20:120
87247299133,7cyclictest952493-21latency_hist23:50:010
87247299132,9cyclictest963769-21/usr/sbin/munin00:30:130
87247299131,7cyclictest905799-21munin-run20:59:590
87247299131,1cyclictest0-21swapper/000:26:450
87247299130,1cyclictest0-21swapper/022:40:130
87247299130,12cyclictest798-21runrttasks22:49:450
87247299120,2cyclictest902903-21/usr/sbin/munin20:50:280
87247299120,12cyclictest0-21swapper/022:55:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional