You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-25 - 14:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Mon Nov 25, 2024 00:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6617422188159,25sleep00-21swapper/019:07:220
66221999179171,6cyclictest706409-21rsync21:23:571
6618302156116,22sleep30-21swapper/319:08:303
6616222149112,21sleep20-21swapper/219:05:482
6618632142112,23sleep10-21swapper/119:08:561
662224999577,16cyclictest716360-21cc121:31:552
662224999577,16cyclictest716360-21cc121:31:542
662226998981,6cyclictest706409-21rsync21:23:573
8905872790,1chrt890575-21cc100:33:163
662224997365,7cyclictest706409-21rsync21:23:572
662226997268,3cyclictest742035-21kworker/u8:2+events_unbound23:09:563
662216996753,11cyclictest718674-21cc121:33:360
662216996753,11cyclictest718674-21cc121:33:350
662224996657,6cyclictest758679-21latency_hist23:15:012
662219996459,4cyclictest730885-21kworker/u8:5+flush-259:023:10:011
8654892630,1chrt865488-21as00:16:152
662224996357,4cyclictest773183-21kworker/u8:2+events_unbound23:20:062
662226996255,5cyclictest833190-21as23:54:553
662224996050,8cyclictest833272-21cc123:55:022
662219995954,4cyclictest75-21kswapd021:29:341
662216995949,7cyclictest853456-21as00:08:130
8717772580,4chrt871437-21cc100:20:212
662226995855,2cyclictest668808-21kworker/u8:2+flush-259:020:59:553
662219995746,8cyclictest806591-21cc123:38:471
662216995747,7cyclictest879502-21fixdep00:25:210
662226995650,4cyclictest433058-21systemd-journal19:35:013
662226995646,8cyclictest770872-21cc123:18:513
662226995646,8cyclictest770872-21cc123:18:513
662216995650,4cyclictest706409-21rsync21:24:330
662224995553,2cyclictest694896-21kworker/u8:0+flush-259:021:27:192
662226995342,8cyclictest850006-21fixdep00:05:223
662224995351,2cyclictest672455-21kworker/u8:3+flush-259:020:25:212
662224995347,4cyclictest746731-21kworker/u8:0+events_unbound23:48:042
662219995338,13cyclictest836610-21cc123:56:301
662216995341,10cyclictest785069-21cc123:26:470
662226995248,3cyclictest75-21kswapd021:29:083
662226995143,6cyclictest755490-21kworker/u8:3+flush-259:023:25:223
662226995142,7cyclictest717526-21cc121:32:443
662226995142,7cyclictest717526-21cc121:32:433
662224995144,5cyclictest829307-21kworker/u8:2+events_unbound00:07:052
662219995143,6cyclictest829568-21cc123:52:061
662226995048,2cyclictest694896-21kworker/u8:0+flush-259:021:17:203
662226995048,2cyclictest694896-21kworker/u8:0+flush-259:021:17:203
662224995040,8cyclictest814734-21cc123:43:152
662216995041,7cyclictest763823-21as23:17:380
662216995041,7cyclictest763823-21as23:17:380
662226994940,7cyclictest897567-21cc100:38:223
8218132480,2sleep3821814-21gcc23:47:533
662226994839,7cyclictest802754-21as23:36:073
662226994838,8cyclictest880530-21cc100:25:523
662224994841,5cyclictest755490-21kworker/u8:3+events_unbound23:35:522
662219994843,3cyclictest691437-21apt-get20:40:001
662219994838,8cyclictest879895-21cc100:25:261
662219994838,7cyclictest781027-21cc123:24:511
662226994738,7cyclictest814328-21cc123:43:063
662224994740,5cyclictest829307-21kworker/u8:2+events_unbound00:28:422
662224994738,7cyclictest897896-21as00:38:352
662219994741,4cyclictest785319-21kworker/u8:4+flush-259:023:32:151
662219994738,7cyclictest817809-21cc123:45:071
662219994738,7cyclictest809735-21ntpq23:40:211
8695162460,4sleep0869503-21cc100:19:160
662224994641,4cyclictest851185-21kworker/u8:1+events_unbound00:10:142
662224994640,4cyclictest719829-21kworker/u8:1+flush-259:021:35:162
662224994633,10cyclictest845617-21fixdep00:02:552
662219994638,6cyclictest787198-21cc123:28:211
662219994636,8cyclictest896230-21cc100:37:181
662216994638,6cyclictest832585-21as23:54:330
662216994638,6cyclictest790134-21apt-get23:30:160
662224994538,5cyclictest746731-21kworker/u8:0+flush-259:023:53:202
662224994537,6cyclictest891387-21cc100:33:592
662224994534,3cyclictest755490-21kworker/u8:3+flush-259:023:25:552
662219994537,6cyclictest892589-21latency_hist00:35:021
662219994536,7cyclictest866683-21cc100:17:141
662219994532,6cyclictest873279-21cc100:21:151
662216994537,6cyclictest859968-21cc100:12:540
662216994537,6cyclictest781057-21as23:24:520
662216994536,7cyclictest843974-21as00:01:270
662226994434,7cyclictest876738-21as00:23:523
662224994428,14cyclictest799557-21cc123:34:532
662219994437,5cyclictest844112-21cc100:01:371
662219994436,6cyclictest712405-21cc121:30:141
662219994436,6cyclictest712405-21cc121:30:141
662216994441,2cyclictest754301-21xargs23:10:010
662216994435,7cyclictest804221-21as23:36:590
662226994341,2cyclictest754496-21kworker/u8:1+flush-259:023:10:263
662219994332,8cyclictest761090-21cc123:15:501
662219994332,8cyclictest761090-21cc123:15:491
662216994335,6cyclictest822569-21as23:48:170
662216994334,7cyclictest886896-21as00:30:260
662216994333,8cyclictest810917-21as23:40:540
662226994240,2cyclictest719829-21kworker/u8:1+events_unbound21:45:213
662216994234,6cyclictest898379-21as00:38:550
662216994233,7cyclictest720872-21objtool21:35:120
662226994133,6cyclictest859798-21as00:12:333
662226994133,6cyclictest839581-21fixdep23:58:473
662226994132,6cyclictest866296-21fixdep00:16:563
662219994135,4cyclictest753772-21latency_hist23:10:001
662219994134,5cyclictest858965-21cc100:11:481
662216994139,2cyclictest668808-21kworker/u8:2+flush-259:020:45:000
662216994132,7cyclictest837118-21as23:56:560
662226994039,1cyclictest742035-21kworker/u8:2+events_unbound23:00:193
662226994039,1cyclictest668808-21kworker/u8:2+events_unbound19:50:253
662226994032,6cyclictest844156-21objtool00:01:383
662219994034,4cyclictest694896-21kworker/u8:0+flush-259:021:35:151
662219994029,8cyclictest853168-21objtool00:07:561
662226993937,2cyclictest730885-21kworker/u8:5+events_unbound22:05:213
662224993935,3cyclictest746731-21kworker/u8:0+events_unbound23:15:352
662224993935,3cyclictest746731-21kworker/u8:0+events_unbound23:15:342
662216993931,6cyclictest872830-21cc100:20:550
662226993831,5cyclictest799820-21latency_hist23:35:013
662226993829,7cyclictest777197-21cc123:22:253
662226993735,2cyclictest668808-21kworker/u8:2+flush-259:020:09:563
662224993735,2cyclictest646991-21kworker/u8:1+flush-259:019:25:002
662219993730,5cyclictest682520-21ntpq20:10:181
662226993634,2cyclictest678775-21kworker/u8:1+flush-259:020:50:193
662226993626,8cyclictest722130-21cc121:35:343
662224993633,2cyclictest730885-21kworker/u8:5+events_unbound22:50:002
662216993629,5cyclictest709877-21cc121:29:010
662226993533,2cyclictest730885-21kworker/u8:5+events_unbound22:35:183
662224993531,3cyclictest691437-21apt-get20:40:002
662216993529,4cyclictest433058-21systemd-journal23:09:400
662224993431,2cyclictest652082-21kworker/u8:0+events_unbound19:35:022
662226993330,2cyclictest733912-21gpgv22:15:003
662216993327,4cyclictest683263-21latency_hist20:15:010
662226993231,1cyclictest742035-21kworker/u8:2+flush-259:022:54:563
662226993229,2cyclictest729102-21kworker/u8:3+events_unbound22:25:163
662224993227,4cyclictest742767-21ntpq22:40:182
662224993129,2cyclictest694896-21kworker/u8:0+flush-259:021:16:572
662224993129,2cyclictest694896-21kworker/u8:0+flush-259:021:16:572
662224993128,3cyclictest678775-21kworker/u8:1+events_unbound20:40:012
662219993128,2cyclictest698075-21apt-get21:00:011
662226993028,2cyclictest724352-21kworker/u8:2+events_unbound21:58:403
662226993028,2cyclictest646991-21kworker/u8:1+events_unbound19:20:183
662226993028,1cyclictest672455-21kworker/u8:3+flush-259:020:29:563
662224993028,2cyclictest678775-21kworker/u8:1+flush-259:020:30:212
662224993024,4cyclictest685369-21/usr/sbin/munin20:20:142
662216993027,2cyclictest734011-21kworker/u8:1+flush-259:022:35:000
662226992928,1cyclictest730885-21kworker/u8:5+flush-259:022:19:553
662219992926,2cyclictest678775-21kworker/u8:1+flush-259:021:15:011
662216992924,3cyclictest738815-21latency_hist22:30:000
662226992825,2cyclictest678775-21kworker/u8:1+flush-259:020:10:213
662224992822,4cyclictest670353-21apt-get19:35:002
662226992726,1cyclictest742035-21kworker/u8:2+flush-259:022:49:553
662219992723,3cyclictest742035-21kworker/u8:2+flush-259:023:05:011
662219992723,3cyclictest742035-21kworker/u8:2+flush-259:023:00:001
662219992723,3cyclictest678775-21kworker/u8:1+flush-259:020:25:011
662219992721,4cyclictest672455-21kworker/u8:3+flush-259:020:00:001
662226992624,2cyclictest694896-21kworker/u8:0+flush-259:021:10:193
662226992624,2cyclictest668808-21kworker/u8:2+flush-259:019:40:213
662226992624,2cyclictest646991-21kworker/u8:1+events_unbound19:15:003
662224992624,2cyclictest734011-21kworker/u8:1+events_unbound22:20:342
662224992624,2cyclictest672455-21kworker/u8:3+events_unbound20:03:152
662219992620,4cyclictest670354-21latency_hist19:35:011
662226992524,1cyclictest730885-21kworker/u8:5+flush-259:022:44:553
662226992523,2cyclictest678775-21kworker/u8:1+flush-259:020:20:313
662226992523,2cyclictest672455-21kworker/u8:3+events_unbound19:48:123
662226992523,2cyclictest652082-21kworker/u8:0+events_unbound19:25:233
662226992522,2cyclictest698588-21kworker/u8:4+events_unbound21:05:193
662224992520,3cyclictest698075-21apt-get21:00:002
662216992522,2cyclictest742035-21kworker/u8:2+flush-259:023:00:200
662224992422,2cyclictest646991-21kworker/u8:1+events_unbound19:25:182
662216992422,2cyclictest694896-21kworker/u8:0+flush-259:021:17:310
662216992422,2cyclictest694896-21kworker/u8:0+flush-259:021:17:300
662216992419,4cyclictest698588-21kworker/u8:4+flush-259:021:05:160
662226992321,2cyclictest668808-21kworker/u8:2+flush-259:020:00:063
662226992320,2cyclictest727501-21kworker/u8:0+events_unbound22:00:083
662226992319,2cyclictest678775-21kworker/u8:1+events_unbound20:15:153
662224992319,3cyclictest683795-21apt-get20:15:132
662224992318,4cyclictest731470-21ntpq22:05:202
662216992321,2cyclictest668808-21kworker/u8:2+events_unbound19:40:200
662216992321,2cyclictest646991-21kworker/u8:1+events_unbound19:10:200
662226992220,2cyclictest706450-21kworker/u8:4+events_unbound21:41:283
662226992220,2cyclictest668808-21kworker/u8:2+flush-259:020:48:113
662226992220,2cyclictest652082-21kworker/u8:0+flush-259:019:15:153
662219992217,4cyclictest723501-21ntpq21:40:151
662216992220,2cyclictest719829-21kworker/u8:1+flush-259:021:45:220
662216992220,2cyclictest678775-21kworker/u8:1+events_unbound20:30:220
662216992219,2cyclictest729102-21kworker/u8:3+events_unbound22:10:190
662224992119,2cyclictest742035-21kworker/u8:2+events_unbound22:56:522
662224992115,4cyclictest433058-21systemd-journal22:00:002
662219992119,2cyclictest727501-21kworker/u8:0+events_unbound22:00:191
662219992119,2cyclictest724352-21kworker/u8:2+events_unbound21:55:191
662219992117,3cyclictest668808-21kworker/u8:2+flush-259:019:50:201
662219992116,3cyclictest704408-21systemd21:16:511
662219992116,3cyclictest704408-21systemd21:16:511
662216992119,2cyclictest698588-21kworker/u8:4+flush-259:021:10:160
662226992019,1cyclictest652082-21kworker/u8:0+flush-259:019:30:233
662226992018,2cyclictest729102-21kworker/u8:3+flush-259:022:31:013
662226992016,2cyclictest619-21dbus-daemon20:00:003
662224992018,2cyclictest668808-21kworker/u8:2+events_unbound20:53:022
662224992018,2cyclictest668808-21kworker/u8:2+events_unbound20:45:192
662224992017,2cyclictest719829-21kworker/u8:1+events_unbound21:48:272
662219992017,2cyclictest652082-21kworker/u8:0+events_unbound19:15:181
662219992016,3cyclictest433058-21systemd-journal19:13:051
662219992015,4cyclictest697296-21ntpq20:55:171
662216992018,2cyclictest730885-21kworker/u8:5+flush-259:022:25:000
662226991917,2cyclictest742035-21kworker/u8:2+events_unbound22:55:293
662226991916,2cyclictest698588-21kworker/u8:4+events_unbound21:04:563
662224991917,2cyclictest742035-21kworker/u8:2+flush-259:023:09:552
662224991914,3cyclictest629-21in:imuxsock22:19:592
662224991913,2cyclictest668808-21kworker/u8:2+events_unbound19:45:192
662219991917,2cyclictest719829-21kworker/u8:1+events_unbound21:45:171
662219991917,2cyclictest678775-21kworker/u8:1+flush-259:020:45:151
662219991917,2cyclictest678775-21kworker/u8:1+flush-259:020:25:171
662219991915,3cyclictest691985-21apt-get20:40:121
662226991815,2cyclictest668808-21kworker/u8:2+events_unbound20:43:503
662224991816,2cyclictest719829-21kworker/u8:1+events_unbound22:00:572
662224991814,3cyclictest724352-21kworker/u8:2+flush-259:021:50:222
662219991816,2cyclictest730885-21kworker/u8:5+flush-259:022:20:001
662219991815,2cyclictest742035-21kworker/u8:2+flush-259:022:55:001
662219991812,4cyclictest679371-21ntpq20:00:191
662226991716,1cyclictest730885-21kworker/u8:5+flush-259:022:20:103
662226991715,2cyclictest719829-21kworker/u8:1+events_unbound21:54:463
662226991715,2cyclictest678775-21kworker/u8:1+events_unbound20:35:283
662224991715,2cyclictest742035-21kworker/u8:2+flush-259:022:50:202
662224991715,2cyclictest734011-21kworker/u8:1+events_unbound22:32:462
662224991715,2cyclictest668808-21kworker/u8:2+events_unbound19:50:192
662224991715,1cyclictest672455-21kworker/u8:3+events_unbound20:14:002
662224991714,2cyclictest646991-21kworker/u8:1+flush-259:019:13:152
662224991712,3cyclictest632-21systemd-logind20:00:002
662219991715,2cyclictest729102-21kworker/u8:3+events_unbound22:30:221
662219991715,2cyclictest646991-21kworker/u8:1+events_unbound19:25:211
662219991713,3cyclictest742035-21kworker/u8:2+flush-259:022:50:011
662219991713,3cyclictest678775-21kworker/u8:1+flush-259:020:18:361
662224991614,2cyclictest734011-21kworker/u8:1+events_unbound22:27:302
662224991614,2cyclictest730885-21kworker/u8:5+flush-259:022:10:412
662224991614,2cyclictest706450-21kworker/u8:4+events_unbound21:41:202
662224991614,2cyclictest678775-21kworker/u8:1+events_unbound20:07:452
662224991614,2cyclictest652082-21kworker/u8:0+flush-259:019:15:172
662224991614,1cyclictest769-21runrttasks21:01:192
662219991613,2cyclictest726147-21/usr/sbin/munin21:50:171
662216991613,2cyclictest0-21swapper/022:55:130
662216991610,4cyclictest698247-21base6421:00:000
66222699151,4cyclictest689869-21idleruntime-cro20:35:003
662224991513,2cyclictest730885-21kworker/u8:5+flush-259:022:35:252
662224991513,2cyclictest672455-21kworker/u8:3+events_unbound19:41:092
66222499150,14cyclictest0-21swapper/223:02:162
66221999159,5cyclictest741047-21/usr/sbin/munin22:35:151
66221999151,13cyclictest737478-21/usr/sbin/munin22:25:151
662219991510,4cyclictest672483-21apt-get19:40:131
662219991510,4cyclictest671429-21ntpq19:35:181
66221699157,6cyclictest619-21dbus-daemon22:00:000
662216991512,2cyclictest652082-21kworker/u8:0+flush-259:019:35:000
662216991511,3cyclictest0-21swapper/020:40:000
662224991412,2cyclictest698588-21kworker/u8:4+events_unbound21:05:412
662224991411,2cyclictest698588-21kworker/u8:4+events_unbound21:10:132
66221999148,4cyclictest681705-21awk20:10:001
662219991411,2cyclictest737056-21taskset22:22:501
662219991410,3cyclictest769-21runrttasks20:30:581
66221699149,4cyclictest694043-21ntpq20:45:170
662216991412,1cyclictest740855-21/usr/sbin/munin22:35:130
662216991411,2cyclictest687996-21taskset20:26:300
662216991410,3cyclictest727247-21taskset21:52:140
66221699140,3cyclictest769-21runrttasks22:08:340
66221699140,1cyclictest0-21swapper/020:51:170
66221699140,13cyclictest677491-21/usr/sbin/munin19:55:170
66221999139,3cyclictest742850-21/usr/sbin/munin22:40:191
662219991311,1cyclictest0-21swapper/122:14:401
662219991310,2cyclictest769-21runrttasks21:08:361
662219991310,2cyclictest5489-21gnome-software19:24:491
66221999130,1cyclictest637-21NetworkManager22:05:221
66221699139,3cyclictest5104-40gmain19:22:210
66221699131,9cyclictest5055-10wireplumber19:40:000
662216991312,0cyclictest0-21swapper/020:15:290
662216991311,1cyclictest730554-21taskset22:03:580
662216991311,1cyclictest6800382sleep020:04:110
662216991311,1cyclictest665499-21runrttasks19:18:470
662216991311,1cyclictest0-21swapper/022:40:330
662216991311,1cyclictest0-21swapper/020:20:390
662216991310,2cyclictest698595-21/usr/sbin/munin21:00:200
66221699130,1cyclictest5256-21gnome-shell20:05:590
66221699130,1cyclictest0-21swapper/019:45:370
66221699130,12cyclictest771-21snmpd22:45:500
662219991211,1cyclictest0-21swapper/120:50:081
66221999120,0cyclictest0-21swapper/119:48:491
66221699125,5cyclictest769-21runrttasks21:40:180
662216991210,1cyclictest675612-21/usr/sbin/munin19:50:160
66221699120,1cyclictest734302-21/usr/sbin/munin22:15:160
66221699120,1cyclictest0-21swapper/019:25:120
66221699110,0cyclictest0-21swapper/022:50:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional