You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-25 - 14:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Mon Nov 25, 2024 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12602254118,502sleep112619-21cron23:02:261
14907234612,319sleep014911-21ntpq23:07:580
182642298117,87sleep218294-21latency_hist21:37:252
2560022976,243sleep2370-21snmpd23:46:102
237702276100,56sleep20-21swapper/220:12:282
1330022309,206sleep213347-21ntpq19:37:532
25675222511,193sleep225730-21fschecks_count20:17:452
18296222238,163sleep218445-21grep23:17:572
4472221420,162sleep20-21swapper/219:12:302
20383221415,129sleep220400-21modprobe23:27:252
517722069,160sleep25195-21fschecks_count19:12:462
22147219988,62sleep222185-21processes21:47:592
30720219648,98sleep230769-21modprobe22:17:262
2753421969,17sleep20-21swapper/223:52:402
30556218310,81sleep20-21swapper/220:32:472
29481217816,59sleep20-21swapper/223:57:492
6751217426,136sleep21-21systemd20:58:062
14202217412,60sleep20-21swapper/223:07:272
6727216740,114sleep26752-21fschecks_count22:43:132
787121657,147sleep2370-21snmpd19:21:382
268621655,11sleep20-21swapper/222:29:082
17257216423,120sleep317287-21sh23:17:253
933021615,29sleep20-21swapper/222:51:212
11972215915,127sleep311841-21latency_hist21:17:283
7782215621,117sleep37827-21/usr/sbin/munin21:02:513
25065215658,53sleep225246-1kworker/2:1H21:57:482
10905214918,116sleep010940-21ntp_kernel_err21:12:490
22361214716,114sleep122514-21awk21:48:021
7498214678,17sleep20-21swapper/221:02:462
15570214513,23sleep20-21swapper/221:27:492
14653214241,43sleep20-21swapper/219:42:372
31037214117,95sleep20-21swapper/200:02:492
28517214019,99sleep20-21swapper/222:08:012
12488213810,109sleep20-21swapper/221:17:522
42621369,27sleep20-21swapper/220:37:592
26716213428,47sleep20-21swapper/223:47:562
15991213333,88sleep216017-21cpu23:12:402
14155213118,98sleep3227-21systemd-journal19:42:263
14680213059,49sleep20-21swapper/223:07:522
29196212763,15sleep20-21swapper/220:27:582
8623212620,21sleep20-21swapper/219:22:512
3177821269,100sleep231927-21grep22:17:582
1903921259,97sleep21-21systemd19:57:272
21975212412,99sleep321947-21latency_hist23:32:273
15439212416,91sleep115447-21acpi21:27:391
27257212318,55sleep20-21swapper/222:03:012
27140212256,16sleep20-21swapper/220:22:412
4933212030,41sleep24945-21open_files00:22:522
3273212010,27sleep23350-21switchtime20:47:562
3692211640,57sleep20-21swapper/222:32:522
32303211621,16sleep20-21swapper/200:07:462
31931211621,78sleep331940-21/usr/sbin/munin22:17:593
19101211534,51sleep319120-21apt-get23:22:383
5592211423,35sleep20-21swapper/220:57:292
23982211411,87sleep024001-21apt-get20:12:380
10302211419,82sleep010264-21latency_hist21:12:290
8014211317,80sleep08163-21latency_hist19:22:290
26839211316,78sleep326840-21sshd20:22:183
18052211320,79sleep318114-21sendmail_mailqu23:17:543
1396211322,16sleep20-21swapper/200:12:422
123621138,55sleep21274-21grep20:42:482
9681211213,81sleep19683-21acpi22:52:371
1331211123,73sleep21511-21modprobe19:02:292
905921099,19sleep20-21swapper/221:07:462
3426210830,39sleep23520-21grep00:17:552
23525210812,81sleep21-21systemd23:37:272
1055521088,19sleep20-21swapper/219:28:002
9534210718,72sleep39648-21latency_hist19:27:283
3212421064,90sleep00-21swapper/022:20:140
19894210617,74sleep020039-21latency_hist21:42:280
2066221059,84sleep21-21systemd20:02:272
2066221059,84sleep21-21systemd20:02:272
16888210538,43sleep20-21swapper/219:48:022
1399221037,74sleep20-21swapper/221:22:442
7786210144,17sleep20-21swapper/200:32:492
20698210017,66sleep020852-21latency_hist20:02:290
995529914,69sleep09980-21irqstats22:52:480
591229921,54sleep15972-21fschecks_time20:57:441
3186529919,26sleep00-21swapper/020:37:290
306852995,82sleep00-21swapper/022:16:590
198232996,35sleep20-21swapper/221:42:142
129002999,16sleep10-21swapper/119:37:441
3273129822,56sleep1424-21tail20:37:581
1298429820,62sleep013025-21uniq19:37:470
1935629715,21sleep10-21swapper/119:57:421
1935629715,21sleep10-21swapper/119:57:421
1233529424,50sleep311833-21apt-get21:17:363
105352949,68sleep20-21swapper/221:12:392
1007529410,22sleep20-21swapper/222:52:502
42452938,16sleep30-21swapper/320:52:383
348629316,58sleep13509-21irqstats22:32:471
295929315,60sleep12994-21df00:17:421
39672920,7sleep10-21swapper/119:07:591
2496329213,62sleep024772-21/usr/sbin/munin21:57:460
2373229215,62sleep023824-21mailstats21:52:580
2291829217,59sleep322942-21proc_pri20:07:533
2021229210,69sleep020247-21fschecks_count21:42:440
247952919,22sleep10-21swapper/121:57:391
2607129011,64sleep126074-21entropy23:47:431
2079929023,49sleep20-21swapper/223:27:502
168529019,40sleep11682-21apt-get22:27:421
1236829016,62sleep112393-21cpu21:17:421
672728814,51sleep35801-21/usr/sbin/munin20:57:593
67022889,65sleep26829-21cat00:28:002
159902889,60sleep10-21swapper/123:12:391
1179328822,37sleep10-21swapper/119:32:551
105722880,41sleep310576-21unixbench_singl22:52:583
851128613,22sleep00-21swapper/019:22:490
766428616,50sleep07722-21fschecks_count00:32:470
649428615,57sleep36546-21processes00:27:573
316528612,53sleep02669-21/usr/sbin/munin20:47:540
1142028624,44sleep011526-21tune2fs19:32:460
527828525,38sleep10-21swapper/122:37:551
125282856,29sleep20-21swapper/223:01:302
211572846,29sleep20-21swapper/221:44:282
1620828418,45sleep016358-21grep21:28:030
143828424,21sleep30-21swapper/322:27:283
12722848,28sleep20-21swapper/222:25:532
58722836,27sleep20-21swapper/222:39:502
249452837,28sleep20-21swapper/223:40:582
2348128316,51sleep123523-21awk23:37:251
124722836,29sleep20-21swapper/219:36:032
231962827,28sleep20-21swapper/221:52:352
218582827,29sleep20-21swapper/220:03:282
1916728126,26sleep00-21swapper/019:57:290
51602800,6sleep00-21swapper/022:37:490
507728022,27sleep10-21swapper/100:22:551
3212328013,27sleep30-21swapper/320:37:443
250512805,28sleep20-21swapper/220:14:342
172252806,12sleep00-21swapper/019:49:010
1298627922,39sleep013079-21fschecks_time23:02:450
725527813,22sleep30-21swapper/322:43:263
382127826,29sleep00-21swapper/019:07:590
1818427821,39sleep118294-21ls23:17:561
1461927824,27sleep30-21swapper/321:22:573
309927714,19sleep10-21swapper/120:47:511
2298727728,29sleep30-21swapper/321:52:283
1349727722,37sleep313608-21timerandwakeup23:02:563
950927621,29sleep10-21swapper/122:52:291
2127127625,27sleep00-21swapper/020:02:490
1754427619,36sleep117668-21cat19:52:291
147127512,43sleep01612-21latency_hist22:27:290
3165127420,27sleep10-21swapper/122:17:571
2152127416,43sleep321638-21cat20:02:573
1180927425,26sleep111882-21systemd-run21:17:261
3269427320,29sleep10-21swapper/100:07:571
306852730,8sleep30-21swapper/320:32:513
2761027314,44sleep027658-21processes20:22:560
2937227122,24sleep00-21swapper/020:28:000
1932927115,33sleep00-21swapper/019:57:370
57212708,50sleep00-21swapper/000:27:180
449927021,31sleep04517-21cron22:37:260
2722627013,41sleep327281-21fschecks_count20:22:463
2139127020,24sleep10-21swapper/120:02:551
182962708,41sleep3286-21dbus-daemon21:37:263
2724226912,45sleep3370-21snmpd23:52:253
2617126913,41sleep126266-21threads20:17:561
169992697,53sleep00-21swapper/023:14:110
1362426916,34sleep0227-21systemd-journal21:22:260
967126816,31sleep39828-21grep21:08:003
64802670,8sleep10-21swapper/119:17:281
319026717,35sleep33244-21sed00:17:483
2966326712,38sleep029731-21/usr/sbin/munin23:57:550
159826718,23sleep00-21swapper/000:12:480
1296326724,21sleep10-21swapper/123:02:421
869026628,19sleep10-21swapper/119:22:541
764826613,22sleep00-21swapper/021:02:480
470126619,31sleep14703-21ntp_states20:52:521
335326617,26sleep30-21swapper/320:47:573
3094726620,27sleep131036-21/usr/sbin/munin00:02:481
1825726615,20sleep30-21swapper/319:52:553
1159726620,27sleep00-21swapper/022:57:490
830826517,33sleep38372-21idleruntime22:47:503
791926514,37sleep17894-21latency_hist22:47:281
530126513,37sleep05353-21interrupts19:12:480
418226523,19sleep00-21swapper/000:22:250
300326518,20sleep30-21swapper/322:32:283
2863426519,21sleep30-21swapper/322:08:023
281126414,22sleep02875-21systemctl19:07:270
267226421,23sleep30-21swapper/300:17:283
1839326422,23sleep10-21swapper/119:52:581
1017026416,34sleep310206-21ntpq19:27:533
212522638,12sleep00-21swapper/023:28:000
204826324,17sleep30-21swapper/319:02:543
1931126315,27sleep00-21swapper/023:22:470
1834426322,21sleep00-21swapper/021:37:270
178226315,26sleep00-21swapper/022:27:490
166112637,12sleep30-21swapper/321:31:053
1490226320,19sleep30-21swapper/319:42:543
3030626218,25sleep10-21swapper/120:32:291
2511126211,35sleep325253-21cut23:42:283
2471726215,29sleep124862-21/usr/sbin/munin20:13:001
2419626221,26sleep124234-21grep23:37:501
1811626215,20sleep00-21swapper/023:17:550
602526117,27sleep06020-21apt-get00:27:420
2279326116,28sleep022866-21/usr/sbin/munin23:32:560
815326015,22sleep30-21swapper/300:32:583
2546726017,21sleep30-21swapper/321:57:573
2331926023,19sleep10-21swapper/121:52:461
1465226018,25sleep114170-21apt-get19:42:361
494625919,18sleep30-21swapper/300:22:533
45825920,19sleep00-21swapper/000:08:000
2168125915,27sleep021675-21apt-get21:47:450
16932597,12sleep10-21swapper/100:12:491
1444525917,23sleep30-21swapper/323:07:453
676325816,19sleep10-21swapper/119:17:451
609525821,20sleep00-21swapper/020:57:470
2753225816,26sleep027528-21apt-get23:52:380
2609725819,20sleep00-21swapper/020:17:550
2416625816,23sleep00-21swapper/023:37:490
2248325817,24sleep022530-21df20:07:420
2057225817,25sleep3370-21snmpd21:42:513
1976225813,31sleep319807-21ntp_states19:57:523
1976225813,31sleep319807-21ntp_states19:57:523
1884425816,24sleep30-21swapper/321:37:473
161125816,27sleep31652-21ps20:42:573
2972425716,24sleep10-21swapper/122:12:491
2925425714,23sleep10-21swapper/122:12:281
2527225718,23sleep30-21swapper/320:17:263
1390925714,26sleep013905-21apt-get21:22:380
123025715,23sleep11235-21diskstats20:42:471
1177625715,25sleep311782-21ntp_states19:32:533
1044325714,25sleep10-21swapper/119:27:591
738625617,22sleep10-21swapper/121:02:371
467725617,22sleep34798-21ntp_states19:12:313
44612566,36sleep00-21swapper/020:52:470
3268125613,18sleep10-21swapper/122:22:461
2766025612,21sleep10-21swapper/120:22:571
163325621,22sleep11638-21entropy19:02:441
154925616,22sleep00-21swapper/020:42:540
941225519,22sleep11-21systemd21:07:541
675025515,24sleep36756-21df_inode19:17:433
315502558,19sleep30-21swapper/300:02:593
2911925514,24sleep129113-21apt-get23:57:401
2330125517,23sleep323307-21diskstats21:52:443
946725413,18sleep00-21swapper/022:52:280
508525417,18sleep30-21swapper/322:37:493
3067425413,23sleep030666-21apt-get00:02:390
2138525413,25sleep121396-21cron21:47:251
2065225412,25sleep120660-21/usr/sbin/munin23:27:411
977325312,24sleep09767-21apt-get19:27:410
757425314,22sleep17571-21apt-get00:32:391
358825315,22sleep33593-21ntp_states19:07:543
2270925315,21sleep322791-21mailstats23:32:553
1709225311,28sleep117120-21cut21:32:441
762825210,31sleep00-21swapper/022:44:200
321325210,17sleep30-21swapper/322:32:363
2868925212,23sleep328682-21apt-get20:27:433
2635325215,21sleep126351-21apt-get22:02:391
200932525,35sleep10-21swapper/123:23:411
1444425217,16sleep10-21swapper/121:22:551
3105325115,17sleep00-21swapper/020:33:010
3077025112,22sleep330798-21munin-run22:17:273
2580925113,15sleep00-21swapper/023:47:300
2269825115,21sleep122727-21irqstats20:07:471
1857225112,22sleep118574-21apt-get21:37:371
1624025114,18sleep30-21swapper/319:47:473
1529625117,17sleep00-21swapper/019:43:040
1462925118,17sleep10-21swapper/123:07:511
286222505,14sleep10-21swapper/123:55:141
2663025011,15sleep00-21swapper/022:02:510
108425015,19sleep30-21swapper/300:12:273
2635624914,20sleep326351-21apt-get22:02:413
291172485,32sleep00-21swapper/022:11:510
2825624815,14sleep30-21swapper/323:52:583
2911624714,17sleep30-21swapper/323:57:383
2544824213,18sleep10-21swapper/123:44:121
71392417,15sleep10-21swapper/100:30:381
2575123910,16sleep30-21swapper/323:47:293
2568623910,12sleep00-21swapper/023:47:170
295082387,12sleep10-21swapper/120:28:011
114052387,15sleep30-21swapper/322:57:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional