You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 09:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot0.osadl.org (updated Fri Nov 22, 2024 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2325121440,142sleep126950irq/26-ahci[00021:56:031
1475421310,4sleep22963399cyclictest21:52:162
1950621280,2sleep319501-21sshd22:10:523
1408121230,2sleep20-21swapper/223:00:412
2436721160,2sleep31848-21nfsd23:14:393
29254211033,43sleep30-21swapper/319:08:323
1218921000,3sleep01959-21sshd22:57:220
261002930,1sleep00-21swapper/022:33:090
245842900,2sleep10-21swapper/121:28:451
205382890,3sleep22963399cyclictest22:48:302
205382890,3sleep22963399cyclictest22:48:292
60462880,4sleep10-21swapper/122:56:431
240462850,3sleep20-21swapper/221:28:432
233452850,2sleep10-21swapper/122:20:241
28382840,4sleep30-21swapper/300:30:433
150162830,3sleep10-21swapper/122:16:341
9182810,3sleep30-21swapper/323:57:083
2910928133,44sleep10-21swapper/119:06:511
207942810,3sleep22963399cyclictest21:43:462
102562810,2sleep110255-21sshd00:24:241
52732790,1sleep20-21swapper/223:57:352
47772790,2sleep34772-21sshd21:54:253
143802790,2sleep10-21swapper/121:43:081
141932790,2sleep10-21swapper/122:44:371
244052780,3sleep32963499cyclictest21:40:583
152812780,2sleep20-21swapper/200:11:222
152102750,2sleep10-21swapper/123:07:171
84362740,1sleep010-21rcuc/023:41:380
3122740,1sleep3431ktimersoftd/321:20:193
149592740,3sleep20-21swapper/223:48:542
307902720,2sleep130788-21sshd00:26:281
132592720,2sleep113257-21sshd00:35:261
182002690,3sleep318197-21bash22:32:203
320032680,4sleep32963499cyclictest00:13:073
157002650,2sleep02963199cyclictest23:52:080
2931026327,23sleep20-21swapper/219:09:122
307852610,3sleep22963399cyclictest00:30:182
2906925633,11sleep00-21swapper/019:06:210
5422550,1sleep10-21swapper/122:46:251
5422550,1sleep10-21swapper/122:46:241
323542370,1sleep00-21swapper/021:26:210
320792371,33sleep2135650irq/27-eth000:19:572
57102350,2sleep3431ktimersoftd/322:40:213
13032320,1sleep00-21swapper/022:27:320
2963499301,27cyclictest20535-21sshd00:29:093
29632993025,3cyclictest20436-21bash21:16:031
2963499281,22cyclictest1847-21nfsd23:15:253
295232280,26sleep2135650irq/27-eth023:37:252
147462280,2sleep31847-21nfsd21:58:223
71102270,1sleep07090-21sshd23:00:070
60222270,25sleep326950irq/26-ahci[00023:03:113
326472270,4sleep12963299cyclictest21:47:541
2963499272,20cyclictest24693-21sshd22:08:253
29631992720,5cyclictest8164-21sshd23:06:340
263812270,2sleep20-21swapper/222:42:292
2963499262,3cyclictest30332-21sshd21:29:203
282682260,0sleep00-21swapper/021:22:560
93732250,2sleep20-21swapper/221:45:382
2963499251,21cyclictest31233-21sshd22:21:093
2963499250,8cyclictest30380-21sshd22:55:533
158742250,3sleep02963199cyclictest00:35:450
2963399243,13cyclictest20152-21sshd23:33:192
191042240,1sleep10-21swapper/123:20:221
49202230,4sleep22963399cyclictest00:07:132
2963499230,8cyclictest27882-21sshd23:08:373
249662230,2sleep10-21swapper/121:22:371
79272220,3sleep32963499cyclictest20:44:593
60782220,2sleep20-21swapper/200:04:092
50712220,2sleep20-21swapper/222:06:302
2963399223,13cyclictest4151-21sshd21:17:372
215022220,2sleep20-21swapper/220:00:242
162872220,1sleep30-21swapper/321:46:183
315452212,17sleep2135650irq/27-eth023:21:382
2963499212,5cyclictest22624-21sshd22:38:593
2963499210,18cyclictest5656-21sshd21:36:093
20262210,2sleep32405-21runrttasks00:16:403
89742200,17sleep2135650irq/27-eth022:19:102
2963499203,7cyclictest0-21swapper/322:45:373
2963499203,7cyclictest0-21swapper/322:45:373
2963499201,7cyclictest22533-21bash22:26:273
2963499201,14cyclictest8743-21sshd00:07:373
2963499200,18cyclictest1959-21sshd21:11:443
2963499200,17cyclictest17053-21sshd22:01:333
2963399202,11cyclictest16709-21sshd22:22:532
47382194,13sleep3135650irq/27-eth000:20:223
2963499199,8cyclictest23654-21sshd22:52:023
2963499192,15cyclictest2372-21grep22:15:233
29634991917,1cyclictest0-21swapper/323:30:203
2963499191,6cyclictest32286-21bash23:25:013
2963499191,6cyclictest135650irq/27-eth023:41:183
29634991913,4cyclictest5576-21munin-run20:39:593
2963499191,16cyclictest5316-21sshd23:25:243
2963499191,16cyclictest25150-21sshd21:31:513
29634991910,7cyclictest25193-21sshd23:36:573
29634991910,4cyclictest30773-21sshd23:50:213
2963499190,17cyclictest1847-21nfsd19:45:173
2963499190,16cyclictest0-21swapper/319:27:523
2963499190,13cyclictest10163-21sshd21:15:113
2963499190,10cyclictest135650irq/27-eth020:11:093
29632991917,1cyclictest0-21swapper/100:17:081
29631991912,5cyclictest4361-21bash22:50:090
36302180,1sleep03628-21seq21:39:070
2963499181,8cyclictest24052-21sshd23:46:313
2963499181,12cyclictest7582-21bash00:01:013
29634991810,6cyclictest26711-21sshd00:36:563
2963499180,12cyclictest135650irq/27-eth020:54:233
29633991813,3cyclictest18265-21sshd22:58:002
2963399181,11cyclictest19247-21sshd22:04:532
2963399180,4cyclictest831-21cp23:44:132
2963399180,11cyclictest11257-21sshd22:37:532
2963299181,10cyclictest1910-21sshd23:28:201
2963299180,6cyclictest0-21swapper/123:00:181
2963199184,11cyclictest8887-21sshd00:31:250
2963199180,5cyclictest20067-21sshd00:29:050
2963199180,17cyclictest15177-21sshd22:38:160
2963199180,14cyclictest32717-21id21:41:480
122082181,15sleep2135650irq/27-eth022:25:272
2963499171,14cyclictest20639-21munin-node20:00:153
2963499170,6cyclictest135650irq/27-eth021:00:003
2963499170,15cyclictest17155-21munin-node21:05:143
2963499170,14cyclictest0-21swapper/319:38:033
2963499170,14cyclictest0-21swapper/319:31:533
2963399173,11cyclictest5351-21sshd23:28:402
2963399172,9cyclictest22675-21sshd00:36:292
2963399172,5cyclictest11500-21sshd00:21:022
2963399172,12cyclictest0-21swapper/221:00:472
29633991710,3cyclictest25485-21sshd23:53:092
2963399170,8cyclictest27669-21sshd21:56:292
2963299175,3cyclictest7158-21sshd23:44:501
2963299172,3cyclictest19769-21sshd21:31:201
29632991712,3cyclictest20760-21bash21:37:381
2963299171,14cyclictest135650irq/27-eth022:08:351
2963299170,15cyclictest2260-21bash22:12:211
2963199176,7cyclictest17436-21sshd23:58:510
2963199171,4cyclictest669-21sshd22:49:490
2963199171,4cyclictest669-21sshd22:49:490
2963199171,12cyclictest0-21swapper/022:20:150
2963199170,3cyclictest0-21swapper/000:12:520
2963199170,1cyclictest25391-21sshd23:49:580
2963199170,12cyclictest0-21swapper/022:17:470
307102161,3sleep10-21swapper/123:37:321
2963499167,7cyclictest23094-21latency20:05:163
29634991613,2cyclictest0-21swapper/320:20:153
2963499160,9cyclictest16343-21missed_timers19:50:183
2963399169,6cyclictest0-21swapper/223:07:212
2963399168,3cyclictest7456-21sshd00:27:342
2963399167,5cyclictest1383-21sshd21:11:172
2963399163,1cyclictest342-21sshd21:38:472
2963399162,11cyclictest135650irq/27-eth021:33:012
2963399161,13cyclictest594-21switchtime23:15:232
2963399161,12cyclictest0-21swapper/220:39:182
29633991610,1cyclictest19414-21sshd21:22:042
2963399160,12cyclictest135650irq/27-eth023:10:522
2963399160,10cyclictest0-21swapper/221:09:482
29632991612,3cyclictest3203-21sshd23:47:421
29632991612,3cyclictest30089-21bash00:03:161
29632991611,4cyclictest20841-21sshd23:14:191
29632991610,4cyclictest24399-21bash21:50:121
2963299160,14cyclictest0-21swapper/122:51:241
2963299160,12cyclictest36-21rcuop/220:35:551
2963299160,11cyclictest26308-21munin-run22:30:001
2963199168,4cyclictest0-21swapper/021:57:050
2963199168,4cyclictest0-21swapper/021:12:470
2963199167,4cyclictest1398-21bash00:06:530
2963199160,6cyclictest0-21swapper/000:22:280
2963199160,5cyclictest0-21swapper/023:24:160
2963199160,14cyclictest0-21swapper/020:40:190
2963199160,14cyclictest0-21swapper/000:17:320
2963199160,11cyclictest31862-21sshd00:03:260
2963499158,6cyclictest135650irq/27-eth020:15:573
2963499157,5cyclictest0-21swapper/319:18:083
2963499151,12cyclictest0-21swapper/320:31:533
2963499150,6cyclictest11878-21netstat19:40:203
2963499150,13cyclictest31680-21diskstats20:25:113
2963399157,6cyclictest12045-21sshd22:50:502
2963399155,9cyclictest0-21swapper/222:30:002
2963399152,1cyclictest11171-21sed20:50:172
2963399150,13cyclictest15856-21sshd22:10:302
2963299152,12cyclictest0-21swapper/123:17:521
2963299152,11cyclictest29890-21bash00:30:151
29632991511,3cyclictest7319-21sshd22:34:311
29632991511,3cyclictest20559-21id00:11:561
29632991511,3cyclictest15918-21sshd00:05:091
29632991510,3cyclictest30345-21sshd23:53:391
2963299150,4cyclictest0-21swapper/120:06:281
2963299150,14cyclictest1848-21nfsd23:30:461
2963299150,11cyclictest9809-21sshd20:46:331
2963199159,2cyclictest0-21swapper/022:11:000
2963199158,3cyclictest1847-21nfsd22:05:200
2963199155,5cyclictest7939-21bash22:40:330
2963199153,1cyclictest0-21swapper/022:02:190
2963199152,2cyclictest24861-21sshd23:27:240
2963199150,9cyclictest24299-21sshd21:34:580
2963199150,9cyclictest10077-21sshd23:10:040
2963199150,12cyclictest0-21swapper/020:45:310
2963499141,12cyclictest1848-21nfsd21:01:223
2963499140,8cyclictest30322-21hddtemp_smartct19:10:143
2963499140,3cyclictest135650irq/27-eth019:21:063
2963399140,13cyclictest0-21swapper/220:20:032
2963399140,12cyclictest32560-21proc_pri20:25:182
2963399140,11cyclictest0-21swapper/220:43:052
2963399140,11cyclictest0-21swapper/219:30:052
2963299147,6cyclictest13746-21sh20:55:161
2963299142,10cyclictest13923-21cp22:01:151
2963299141,12cyclictest36-21rcuop/222:35:131
29632991411,2cyclictest26015-21id21:10:301
2963299141,12cyclictest135650irq/27-eth019:45:021
2963299140,5cyclictest31480-21sshd19:12:321
2963299140,13cyclictest0-21swapper/119:21:271
2963299140,10cyclictest135650irq/27-eth020:21:131
2963199149,1cyclictest28884-21sshd23:34:120
2963199148,5cyclictest0-21swapper/023:35:100
2963199148,4cyclictest0-21swapper/020:05:160
2963199141,9cyclictest2281950kworker/u8:121:15:030
2963199140,6cyclictest27978-21bash23:18:070
2963199140,1cyclictest0-21swapper/021:47:360
2963199140,13cyclictest0-21swapper/021:50:550
207272140,12sleep1135650irq/27-eth023:55:501
203282140,1sleep10-21swapper/120:00:161
2963499139,3cyclictest18844-21ntp_states19:55:183
2963499130,5cyclictest135650irq/27-eth020:45:183
2963399137,5cyclictest0-21swapper/219:22:132
2963399136,5cyclictest0-21swapper/219:55:152
2963399131,3cyclictest0-21swapper/220:10:162
2963399131,11cyclictest0-21swapper/220:06:362
2963399130,4cyclictest1927-21snmpd19:45:022
2963399130,4cyclictest1179-21munin-node20:30:172
2963399130,3cyclictest135650irq/27-eth020:57:012
2963399130,12cyclictest2405-21runrttasks19:14:352
2963399130,12cyclictest135650irq/27-eth019:50:142
2963399130,12cyclictest0-21swapper/220:16:202
2963399130,12cyclictest0-21swapper/219:43:332
2963399130,12cyclictest0-21swapper/219:37:452
2963399130,12cyclictest0-21swapper/219:25:392
2963399130,12cyclictest0-21swapper/219:17:012
29632991311,1cyclictest0-21swapper/119:53:521
2963299130,12cyclictest10898-21munin-node19:40:231
2963299130,12cyclictest0-21swapper/120:30:031
2963299130,12cyclictest0-21swapper/120:16:231
2963299130,12cyclictest0-21swapper/120:10:551
2963299130,12cyclictest0-21swapper/119:55:071
2963299130,12cyclictest0-21swapper/119:32:041
2963299130,12cyclictest0-21swapper/119:25:061
2963299130,12cyclictest0-21swapper/119:18:231
2963299130,10cyclictest135650irq/27-eth020:40:211
29631991311,1cyclictest0-21swapper/019:40:460
2963199130,9cyclictest0-21swapper/021:00:210
2963199130,1cyclictest0-21swapper/019:35:100
2963199130,12cyclictest3855-21forks20:35:110
2963199130,12cyclictest0-21swapper/020:50:230
2963199130,12cyclictest0-21swapper/019:50:510
2963199130,12cyclictest0-21swapper/019:10:150
2963199130,10cyclictest0-21swapper/019:25:330
2963399128,3cyclictest8326-21df20:45:102
29632991210,1cyclictest0-21swapper/121:08:131
29632991210,1cyclictest0-21swapper/121:00:101
2963299120,11cyclictest0-21swapper/120:50:551
2963299120,11cyclictest0-21swapper/120:25:061
2963299120,11cyclictest0-21swapper/119:35:031
2963199122,5cyclictest0-21swapper/020:18:330
2963199121,1cyclictest0-21swapper/019:20:570
2963199120,8cyclictest0-21swapper/019:30:110
2963199120,2cyclictest0-21swapper/020:00:450
2963199120,12cyclictest0-21swapper/020:20:450
2963199120,12cyclictest0-21swapper/019:48:310
2963199120,11cyclictest0-21swapper/021:06:090
2963199120,11cyclictest0-21swapper/020:55:020
2963199120,11cyclictest0-21swapper/020:30:530
2963199120,11cyclictest0-21swapper/020:28:220
2963199110,1cyclictest0-21swapper/019:55:470
2963199110,10cyclictest0-21swapper/020:10:150
2963199110,10cyclictest0-21swapper/019:15:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional