You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-21 - 21:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot7s.osadl.org (updated Thu Nov 21, 2024 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29593682194170,20sleep10-21swapper/119:07:501
29593222164139,21sleep20-21swapper/219:07:152
29594082156116,21sleep00-21swapper/019:08:220
29595172139108,26sleep30-21swapper/319:09:443
2959779998078,2cyclictest3024664-21kworker/u8:2+events_unbound23:00:022
2959779997169,2cyclictest2975494-21kworker/u8:2+flush-259:020:00:362
2959779995853,3cyclictest631-21rs:main2
2959779995350,2cyclictest3030692-21kworker/u8:3+flush-259:022:55:002
2959779995250,2cyclictest3030692-21kworker/u8:3+flush-259:023:50:112
2959779995048,2cyclictest2961226-21kworker/u8:1+flush-259:019:30:202
2959779995048,2cyclictest2961226-21kworker/u8:1+flush-259:019:30:202
2959784994943,4cyclictest2972686-21latency_hist19:50:013
2959779994946,2cyclictest3035926-21kworker/u8:0+flush-259:023:05:242
2959784994844,3cyclictest3005629-21kworker/u8:1+events_unbound21:53:383
2959779994844,3cyclictest3024664-21kworker/u8:2+flush-259:000:05:002
2959779994841,5cyclictest3045696-21apt-get23:35:002
2959776994741,4cyclictest3045689-21latency_hist23:35:011
2959774994642,3cyclictest3024664-21kworker/u8:2+flush-259:000:15:000
2959779994543,2cyclictest3052974-21kworker/u8:0+events_unbound00:10:172
2959774994539,4cyclictest618-21cron20:30:000
2959776994339,3cyclictest3052974-21kworker/u8:0+flush-259:000:00:001
2959779994239,2cyclictest2999098-21kworker/u8:0+events_unbound22:45:212
2959776994035,3cyclictest3037534-21latency_hist23:10:001
2959774994036,3cyclictest2948457-21kworker/u8:3+flush-259:019:25:000
2959776993934,3cyclictest2969513-21latency_hist19:40:011
2959779993836,2cyclictest3002479-21kworker/u8:2+flush-259:021:30:212
2959779993836,2cyclictest2979348-21kworker/u8:3+flush-259:020:20:182
2959774993835,2cyclictest2977154-21kworker/u8:1+flush-259:020:35:000
2959779993631,4cyclictest3028461-21meminfo22:40:162
2959776993633,2cyclictest2999098-21kworker/u8:0+flush-259:021:45:001
2959776993633,2cyclictest2999098-21kworker/u8:0+flush-259:021:44:591
2959779993531,3cyclictest2984378-21kworker/u8:2+flush-259:020:30:132
2959776993529,4cyclictest2989062-21apt-get20:40:001
29665892340,2chrt0-21swapper/119:30:001
2959776993431,2cyclictest2997322-21kworker/u8:1+flush-259:021:10:001
2959776993429,4cyclictest2970359-21ntpq19:40:181
2959774993430,3cyclictest2967942-21latency_hist19:35:010
2959774993430,3cyclictest2967942-21latency_hist19:35:010
2959784993328,4cyclictest3015942-21ntpq22:00:163
2959779993331,2cyclictest2989307-21kworker/u8:3+flush-259:021:00:132
2959779993328,4cyclictest3023825-21ntpq22:25:172
30109262320,1chrt3010919-21ntpq21:45:181
2959779993231,1cyclictest3024664-21kworker/u8:2+events_unbound00:30:002
2959779993231,1cyclictest2999098-21kworker/u8:0+events_unbound22:20:212
2959779993229,2cyclictest3030692-21kworker/u8:3+events_unbound00:30:222
2959774993227,3cyclictest3034114-21systemctl23:00:010
2959784993127,3cyclictest2977636-21cat20:05:013
2959784993126,3cyclictest3059922-21latency_hist00:25:013
2959784993123,6cyclictest3024664-21kworker/u8:2+flush-259:023:00:003
2959779993129,2cyclictest3024664-21kworker/u8:2+flush-259:023:59:552
2959779993129,2cyclictest2971158-21kworker/u8:0+events_unbound20:10:202
2959779993125,4cyclictest3039109-21apt-get23:15:002
2959776993128,2cyclictest2997322-21kworker/u8:1+flush-259:021:05:001
2959779993028,2cyclictest3024664-21kworker/u8:2+flush-259:022:55:202
2959779993028,2cyclictest2948457-21kworker/u8:3+flush-259:019:50:202
2959776993027,3cyclictest3024664-21kworker/u8:2+flush-259:022:55:321
2959779992927,2cyclictest2977154-21kworker/u8:1+events_unbound20:05:182
2959779992927,2cyclictest2948457-21kworker/u8:3+events_unbound19:35:232
2959774992925,3cyclictest3002479-21kworker/u8:2+flush-259:021:30:000
2959784992825,2cyclictest3024664-21kworker/u8:2+flush-259:023:40:163
2959779992826,2cyclictest2999098-21kworker/u8:0+events_unbound22:08:362
2959779992824,3cyclictest3007238-21apt-get21:35:132
2959784992725,2cyclictest2958025-21kworker/u8:0+flush-259:019:10:253
2959784992722,4cyclictest2983548-21ntpq20:20:143
2959779992725,2cyclictest2999098-21kworker/u8:0+flush-259:022:00:192
2959779992725,2cyclictest2999098-21kworker/u8:0+flush-259:021:15:152
2959779992725,2cyclictest2989307-21kworker/u8:3+events_unbound21:20:172
2959779992725,2cyclictest2977154-21kworker/u8:1+events_unbound20:15:252
2959776992724,2cyclictest2958025-21kworker/u8:0+events_unbound19:15:211
2959784992620,4cyclictest3038306-21ntpq23:10:183
2959779992624,2cyclictest2989307-21kworker/u8:3+flush-259:020:55:212
2959779992624,2cyclictest2958025-21kworker/u8:0+flush-259:019:25:352
2959779992624,2cyclictest2948457-21kworker/u8:3+flush-259:019:45:132
2959774992620,4cyclictest2977482-21apt20:05:000
2959784992521,3cyclictest2999098-21kworker/u8:0+flush-259:021:15:013
2959784992520,4cyclictest3027017-21ntpq22:35:203
2959779992523,2cyclictest3030692-21kworker/u8:3+flush-259:023:25:152
2959779992523,2cyclictest3030692-21kworker/u8:3+events_unbound23:22:372
2959774992523,2cyclictest3024664-21kworker/u8:2+flush-259:023:20:000
2959774992522,2cyclictest3030692-21kworker/u8:3+flush-259:000:10:000
2959774992521,3cyclictest3047823-21apt-get23:40:120
2959774992521,3cyclictest2999098-21kworker/u8:0+flush-259:021:10:160
2959774992518,4cyclictest3064680-21sendmail00:40:000
2959784992418,4cyclictest2996708-21ntpq21:00:163
2959779992422,2cyclictest2984378-21kworker/u8:2+flush-259:020:50:212
2959776992422,2cyclictest2984378-21kworker/u8:2+events_unbound20:50:191
2959776992422,2cyclictest2977154-21kworker/u8:1+flush-259:020:25:211
2959776992418,4cyclictest2990916-21apt-get20:45:011
2959776992418,4cyclictest2964063-21ntpq19:20:181
2959774992418,3cyclictest2958025-21kworker/u8:0+flush-259:019:30:000
2959784992317,4cyclictest631-21rs:main3
2959779992319,3cyclictest3056979-21kworker/u8:1+events_unbound00:15:152
2959779992318,4cyclictest2989055-21latency_hist20:40:012
2959776992318,4cyclictest3005629-21kworker/u8:1+flush-259:022:30:001
2959774992319,3cyclictest3021728-21/usr/sbin/munin22:20:190
2959784992217,3cyclictest433058-21systemd-journal23:29:593
2959784992211,8cyclictest619-21dbus-daemon20:20:003
2959779992220,2cyclictest2961226-21kworker/u8:1+flush-259:019:40:312
2959779992219,2cyclictest3002479-21kworker/u8:2+flush-259:021:42:082
2959779992219,2cyclictest3002479-21kworker/u8:2+flush-259:021:42:082
2959779992216,3cyclictest0-21swapper/200:09:592
2959776992219,2cyclictest3045648-21kworker/u8:1+events_unbound23:35:201
2959776992217,3cyclictest3013563-21latency_hist21:55:011
2959774992218,3cyclictest433058-21systemd-journal20:20:010
2959784992120,1cyclictest2989307-21kworker/u8:3+flush-259:020:44:553
2959779992119,2cyclictest3005629-21kworker/u8:1+flush-259:021:55:072
2959779992119,2cyclictest3005629-21kworker/u8:1+events_unbound22:15:392
2959779992119,2cyclictest2977154-21kworker/u8:1+events_unbound20:45:182
295977999210,20cyclictest3041000-21unin-run23:20:002
2959776992118,2cyclictest2977154-21kworker/u8:1+flush-259:020:20:011
2959774992119,2cyclictest2989307-21kworker/u8:3+flush-259:020:50:000
2959774992116,3cyclictest3044115-21sh23:30:000
2959774992114,5cyclictest2989075-21systemctl20:40:010
2959784992017,2cyclictest3045648-21kworker/u8:1+flush-259:023:35:013
2959784992016,3cyclictest3011712-21apt-get21:50:003
2959779992018,2cyclictest2989578-21kworker/u8:4+events_unbound20:43:222
2959779992018,2cyclictest2948457-21kworker/u8:3+events_unbound19:14:052
2959776992015,4cyclictest3001459-21ntpq21:15:171
2959774992017,2cyclictest3005629-21kworker/u8:1+flush-259:022:35:220
2959784991917,2cyclictest3035926-21kworker/u8:0+flush-259:023:08:093
295978499190,17cyclictest1-21systemd21:45:003
295978499190,17cyclictest1-21systemd21:44:593
2959779991917,2cyclictest3045648-21kworker/u8:1+events_unbound23:35:022
2959779991917,2cyclictest2977154-21kworker/u8:1+events_unbound20:25:342
2959779991916,2cyclictest2948457-21kworker/u8:3+events_unbound19:57:372
2959779991914,4cyclictest2999865-21ntpq21:10:182
2959776991917,2cyclictest3024664-21kworker/u8:2+flush-259:023:15:181
2959776991915,3cyclictest3035926-21kworker/u8:0+flush-259:023:05:011
2959776991915,3cyclictest3005629-21kworker/u8:1+flush-259:022:20:011
2959776991913,4cyclictest2980894-21latency_hist20:15:001
2959774991915,3cyclictest3024664-21kworker/u8:2+flush-259:000:00:000
2959774991914,4cyclictest3059110-21ntpq00:20:160
2959774991914,3cyclictest3049756-21latency_hist23:50:010
2959784991815,2cyclictest5104-40gmain23:53:123
2959784991815,2cyclictest2999099-21systemd-run21:09:593
2959784991812,3cyclictest0-21swapper/300:10:003
2959784991811,5cyclictest433058-21systemd-journal19:28:353
2959784991811,5cyclictest3053949-21python300:00:203
2959779991816,2cyclictest3030692-21kworker/u8:3+events_unbound23:47:052
2959779991816,2cyclictest3030692-21kworker/u8:3+events_unbound23:47:052
2959779991816,2cyclictest3024664-21kworker/u8:2+events_unbound23:41:312
2959779991816,2cyclictest3005629-21kworker/u8:1+events_unbound22:12:392
2959779991816,2cyclictest3002479-21kworker/u8:2+flush-259:021:29:182
2959779991816,2cyclictest2989307-21kworker/u8:3+events_unbound21:05:192
2959779991816,2cyclictest2958025-21kworker/u8:0+events_unbound19:20:172
2959779991815,2cyclictest2999098-21kworker/u8:0+events_unbound22:32:072
2959776991816,2cyclictest2989307-21kworker/u8:3+flush-259:020:55:251
2959776991816,2cyclictest2958025-21kworker/u8:0+flush-259:019:10:201
2959774991816,2cyclictest3024664-21kworker/u8:2+flush-259:022:40:180
2959774991814,3cyclictest3046154-21cp23:35:000
2959784991715,2cyclictest2961226-21kworker/u8:1+flush-259:019:45:013
2959784991712,4cyclictest2986700-21ntpq20:30:173
2959784991712,3cyclictest3023056-21latency_hist22:25:013
2959784991712,1cyclictest2979565-21/usr/sbin/munin20:10:173
295978499171,14cyclictest3052976-21munin-run00:00:013
2959784991711,4cyclictest0-21swapper/321:00:003
2959779991716,1cyclictest2999098-21kworker/u8:0+events_unbound22:37:002
2959779991715,2cyclictest3002479-21kworker/u8:2+flush-259:021:45:092
2959776991713,3cyclictest433058-21systemd-journal22:30:381
295977699171,15cyclictest769-21runrttasks20:21:001
2959776991711,4cyclictest3061504-21awk00:29:591
2959776991711,4cyclictest3043355-21ntpq23:25:171
2959776991711,4cyclictest3041756-21ntpq23:20:161
2959774991713,3cyclictest3005629-21kworker/u8:1+flush-259:022:15:010
2959774991713,2cyclictest2991177-21apt-key20:45:020
2959774991712,3cyclictest3008545-21latency_hist21:40:000
2959774991711,4cyclictest2997508-21awk21:05:000
295977499170,1cyclictest906-21rtkit-daemon22:45:580
2959784991612,3cyclictest3006969-21idleruntime-cro21:35:003
2959784991610,4cyclictest0-21swapper/322:00:003
295978499160,15cyclictest5256-21gnome-shell20:28:593
2959779991614,2cyclictest3024664-21kworker/u8:2+events_unbound00:24:382
2959779991614,2cyclictest2958025-21kworker/u8:0+events_unbound19:18:372
2959776991614,2cyclictest3052974-21kworker/u8:0+events_unbound00:00:131
2959776991614,2cyclictest3005629-21kworker/u8:1+flush-259:022:10:161
2959776991614,1cyclictest896-21sendmail-mta22:20:481
2959776991612,3cyclictest3024664-21kworker/u8:2+events_unbound00:20:161
2959776991612,2cyclictest629-21in:imuxsock00:40:011
2959776991610,2cyclictest3056986-21/usr/sbin/munin00:15:201
295977699160,15cyclictest0-21swapper/121:35:161
295977699160,13cyclictest3050224-21df_inode23:50:111
295977499169,1cyclictest2979983-21/usr/sbin/munin20:10:180
2959774991611,3cyclictest0-21swapper/021:50:000
295978499159,4cyclictest3018306-21sh22:10:003
2959784991513,2cyclictest2990916-21apt-get20:45:023
2959784991512,2cyclictest3044359-21kworker/3:1+cgroup_destroy23:34:593
2959784991512,2cyclictest3031219-21apt-get22:50:113
295978499150,14cyclictest771-21snmpd23:45:033
295978499150,14cyclictest771-21snmpd23:45:023
295978499150,14cyclictest3064539-21chrt00:38:413
295978499150,12cyclictest3041434-21sh23:20:123
2959776991513,1cyclictest433058-21systemd-journal22:45:361
2959776991511,2cyclictest619-21dbus-daemon20:35:001
2959776991510,3cyclictest2967958-21basename19:35:011
2959776991510,3cyclictest2967958-21basename19:35:001
295977699150,3cyclictest3027767-21taskset22:39:561
2959774991511,3cyclictest3024664-21kworker/u8:2+flush-259:000:30:010
2959774991510,4cyclictest3003830-21awk21:25:000
295977499150,2cyclictest618-21cron19:15:000
2959784991412,2cyclictest2999098-21kworker/u8:0+flush-259:022:40:043
295978499141,12cyclictest0-21swapper/321:35:143
2959784991410,3cyclictest3030692-21kworker/u8:3+flush-259:000:25:173
295978499140,2cyclictest2994100-21taskset20:52:193
295978499140,1cyclictest0-21swapper/322:19:593
295978499140,13cyclictest769-21runrttasks19:50:023
295978499140,13cyclictest3058092-21runrttasks00:17:073
295978499140,13cyclictest2988811-21runrttasks20:36:583
295978499140,13cyclictest2962878-21seq19:16:093
295977999145,2cyclictest191rcu_preempt00:40:002
295977699149,3cyclictest3032525-21idleruntime-cro22:55:001
295977699148,4cyclictest3014221-21meminfo21:55:151
2959776991412,1cyclictest739-21mta-sts-daemon21:25:131
2959776991411,2cyclictest3049639-21sshd23:48:371
2959776991411,2cyclictest3049639-21sshd23:48:361
2959776991411,2cyclictest0-21swapper/120:00:111
295977699140,2cyclictest5258-21gmain19:56:021
295977699140,13cyclictest2999314-21/usr/sbin/munin21:10:211
295977699140,12cyclictest3037805-21apt-get23:10:141
295977699140,12cyclictest3028598-21ntpq22:40:171
295977499149,3cyclictest0-21swapper/021:59:590
2959774991411,2cyclictest769-21runrttasks23:01:400
2959774991411,2cyclictest3025108-21df_inode22:30:170
2959774991410,3cyclictest3030692-21kworker/u8:3+flush-259:023:35:000
2959774991410,3cyclictest3005629-21kworker/u8:1+flush-259:022:10:010
295977499140,2cyclictest2995565-21taskset20:55:520
295977499140,13cyclictest5104-40gmain21:52:520
295977499140,13cyclictest2982485-21perf20:24:550
295978499139,3cyclictest433058-21systemd-journal21:22:573
295978499139,2cyclictest3040959-21munin-run23:20:003
295978499131,1cyclictest491ktimers/320:05:223
2959784991311,1cyclictest3034944-21/usr/sbin/munin23:00:123
295977699138,4cyclictest2978513-21ntpq20:05:171
295977699137,4cyclictest3016739-21latency_hist22:05:001
295977699131,1cyclictest0-21swapper/120:45:031
295977699130,3cyclictest3056147-21/usr/sbin/munin00:10:161
295977699130,2cyclictest3005874-21/usr/sbin/munin21:30:151
295977699130,12cyclictest737-21mta-sts-daemon00:07:011
295977499139,3cyclictest769-21runrttasks20:07:400
295977499139,2cyclictest3042540-21idleruntime-cro23:25:000
295977499139,2cyclictest0-21swapper/023:55:000
2959774991312,0cyclictest0-21swapper/019:16:090
2959774991311,2cyclictest3002479-21kworker/u8:2+events_unbound21:30:220
2959774991311,1cyclictest2992990-21/usr/sbin/munin20:50:110
2959774991310,2cyclictest3037403-21taskset23:08:290
2959774991310,2cyclictest3001261-21latency21:15:150
2959774991310,2cyclictest0-21swapper/019:40:000
295977499130,1cyclictest0-21swapper/019:47:050
295977499130,12cyclictest769-21runrttasks22:52:590
295977499130,11cyclictest3023324-21/usr/sbin/munin22:25:120
295977499130,11cyclictest2972918-21runrttasks19:50:030
295977499130,11cyclictest0-21swapper/023:45:030
295977499130,11cyclictest0-21swapper/021:40:190
295977499130,11cyclictest0-21swapper/021:40:190
295978499127,4cyclictest3062314-21ntpq00:30:213
295978499120,1cyclictest2963703-21/usr/sbin/munin19:20:153
295978499120,12cyclictest491ktimers/321:15:113
295978499120,0cyclictest0-21swapper/300:10:093
295977699128,3cyclictest3003465-21sshd21:20:361
295977699120,11cyclictest3049070-21runrttasks23:44:261
295977699120,0cyclictest0-21swapper/119:51:001
295977499128,2cyclictest0-21swapper/000:04:590
2959774991211,1cyclictest3030692-21kworker/u8:3+flush-259:000:15:550
2959774991210,1cyclictest0-21swapper/022:20:000
295977499120,1cyclictest3016016-21/usr/sbin/munin22:00:170
295977499120,11cyclictest2999035-21taskset21:09:410
295977499120,0cyclictest0-21swapper/019:56:310
295978499119,1cyclictest3025923-21taskset22:31:303
295978499119,1cyclictest3018971-21meminfo22:10:183
295978499110,2cyclictest2974917-21meminfo19:55:163
295978499110,1cyclictest0-21swapper/322:25:123
295978499110,0cyclictest0-21swapper/321:25:523
295977699116,3cyclictest3017210-21diskstats22:05:141
295977699110,11cyclictest0-21swapper/100:30:231
295977499110,9cyclictest0-21swapper/000:30:130
295977499110,1cyclictest0-21swapper/023:11:210
295977499110,10cyclictest2969532-21perf19:44:550
295978499108,1cyclictest2967986-21ls19:35:003
295978499108,1cyclictest2967986-21ls19:35:003
29597849990,0cyclictest0-21swapper/322:47:413
29597769995,3cyclictest2972353-21chrt19:45:511
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional