You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackaslot0.osadl.org (updated Thu Apr 03, 2025 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1543021000,2sleep21937699cyclictest00:35:132
129022960,0sleep20-21swapper/223:40:252
25542930,1sleep31938099cyclictest00:20:413
111852910,0sleep311129-21sshd22:31:473
226292660,1sleep222628-21sshd21:41:542
180062610,2sleep01936999cyclictest22:29:400
61552600,0sleep20-21swapper/222:42:362
154572600,0sleep00-21swapper/000:28:550
4272590,0sleep00-21swapper/022:47:550
108332590,0sleep20-21swapper/222:12:002
314622580,1sleep00-21swapper/023:19:230
209302570,1sleep020924-21interrupts22:55:160
197502570,1sleep319745-21bash23:03:483
141262570,3sleep11937299cyclictest21:30:041
155292550,0sleep115528-21bash23:55:061
287442530,1sleep128741-21sshd21:48:111
276562530,0sleep211878-21diskmemload21:19:512
111852530,0sleep011183-21bash00:06:190
309952520,0sleep10-21swapper/121:17:131
298562520,0sleep20-21swapper/223:04:442
290472520,1sleep11744-21runrttasks21:28:321
268282520,0sleep026823-21sshd21:22:340
231802520,0sleep31744-21runrttasks22:58:193
192702520,1sleep219267-21cp22:32:312
42982510,0sleep04293-21sshd00:17:410
256612510,1sleep125619-21sshd22:50:091
221662510,0sleep30-21swapper/300:32:423
159922510,1sleep30-21swapper/322:03:543
148482510,0sleep10-21swapper/122:03:491
128872510,1sleep312882-21sshd22:06:313
40992490,0sleep20-21swapper/200:27:442
36842490,0sleep33683-21sshd21:57:133
167952490,0sleep10-21swapper/123:12:161
163932490,1sleep116391-21sshd21:41:221
163672490,1sleep11744-21runrttasks22:43:321
241342480,0sleep311878-21diskmemload23:55:453
1887024825,19sleep30-21swapper/319:06:393
91732470,0sleep30-21swapper/321:32:223
240262470,1sleep024019-21sshd22:38:340
1904224725,18sleep20-21swapper/219:08:512
1893524724,19sleep10-21swapper/119:07:301
1701024722,7sleep00-21swapper/019:05:130
153082470,0sleep30-21swapper/321:41:153
177822450,1sleep31744-21runrttasks00:19:083
260032440,0sleep00-21swapper/022:30:170
191162400,0sleep10-21swapper/100:22:181
19372992321,1cyclictest23132-21sshd23:38:361
44922220,0sleep10-21swapper/123:22:421
19372992221,1cyclictest3323-21sshd22:56:331
1937299220,21cyclictest1770-21sshd23:50:531
19369992220,1cyclictest14401-21sshd23:03:190
41772210,0sleep20-21swapper/223:48:192
19380992119,1cyclictest0-21swapper/323:19:533
19372992117,1cyclictest20069-21sshd00:12:571
302792190,1sleep130277-21sshd21:11:281
269332190,0sleep20-21swapper/220:30:252
19372991918,1cyclictest868-21sshd23:08:021
19372991917,1cyclictest2972-21bash22:16:571
40292180,1sleep30-21swapper/300:14:363
19380991816,1cyclictest0-21swapper/323:23:433
19376991817,1cyclictest0-21swapper/221:20:472
19376991816,1cyclictest0-21swapper/222:01:202
37172170,1sleep20-21swapper/222:56:352
25012170,0sleep00-21swapper/000:11:200
19380991716,1cyclictest0-21swapper/323:13:433
19380991715,1cyclictest0-21swapper/322:54:203
19372991715,1cyclictest0-21swapper/122:29:211
19369991715,1cyclictest30125-21cp23:50:290
1936999170,12cyclictest0-21swapper/022:03:030
91422160,0sleep10-21swapper/122:11:511
52222160,1sleep337-21rcuc/322:22:453
316012160,0sleep2291ktimersoftd/223:07:522
282472160,1sleep00-21swapper/022:53:080
280782160,0sleep00-21swapper/023:47:310
19380991614,1cyclictest31496-21sshd22:44:513
1936999169,3cyclictest0-21swapper/023:27:250
1936999160,15cyclictest0-21swapper/023:09:020
1936999160,10cyclictest8608-21sshd22:23:040
14102160,1sleep31382-21sshd23:28:113
131362160,12sleep3115750irq/25-eth000:06:303
76062158,1sleep00-21swapper/022:08:580
73772150,1sleep37344-21sshd23:51:243
286672150,0sleep20-21swapper/221:59:262
238772150,1sleep223867-21sshd00:22:482
19376991513,1cyclictest115750irq/25-eth023:50:232
1936999159,6cyclictest0-21swapper/020:01:500
19369991513,1cyclictest26912-21sshd00:23:070
19369991513,1cyclictest21454-21sshd22:12:560
1936999150,9cyclictest0-21swapper/020:35:170
1936999150,14cyclictest0-21swapper/022:18:130
1936999150,14cyclictest0-21swapper/021:18:470
1936999150,11cyclictest15173-21sshd21:49:520
32172140,0sleep03212-21bash21:40:170
275942140,0sleep327535-21sshd22:27:383
1938099148,2cyclictest7275-21munin-node21:15:123
19380991413,1cyclictest0-21swapper/321:29:513
1938099140,13cyclictest1590-21sshd23:38:443
1937699140,13cyclictest636-21sshd22:50:402
1937699140,13cyclictest10571-21sshd23:31:482
1937699140,13cyclictest0-21swapper/221:38:442
1937699140,13cyclictest0-21swapper/200:13:362
1937299140,13cyclictest0-21swapper/123:15:051
1936999149,4cyclictest0-21swapper/023:35:040
1936999149,2cyclictest0-21swapper/000:00:020
1936999148,4cyclictest0-21swapper/023:20:520
1936999147,5cyclictest0-21swapper/023:42:210
1936999143,10cyclictest0-21swapper/021:50:510
1936999143,10cyclictest0-21swapper/000:31:030
1936999142,10cyclictest17609-21sshd22:43:380
1936999140,3cyclictest0-21swapper/021:10:110
1936999140,2cyclictest1503-21nfsd21:33:330
1936999140,13cyclictest25110-21bash21:36:320
131002140,1sleep013085-21sshd21:55:150
1938099139,3cyclictest0-21swapper/322:15:513
1938099139,2cyclictest0-21swapper/321:36:363
1938099139,1cyclictest115750irq/25-eth023:32:213
1938099139,1cyclictest0-21swapper/321:10:423
1938099138,2cyclictest0-21swapper/321:45:113
1938099133,6cyclictest32483-21kworker/3:023:46:063
1938099131,1cyclictest0-21swapper/322:36:583
1938099130,1cyclictest6843-21sshd23:40:013
1938099130,13cyclictest0-21swapper/300:38:013
1938099130,12cyclictest8114-21bash21:51:583
1938099130,12cyclictest14210-21bash23:09:153
1938099130,12cyclictest0-21swapper/321:21:353
1938099130,10cyclictest0-21swapper/300:28:473
1937699138,4cyclictest0-21swapper/223:25:002
1937699130,4cyclictest15438-21rm23:17:582
1937699130,4cyclictest0-21swapper/221:11:522
1937699130,4cyclictest0-21swapper/200:34:482
1937699130,1cyclictest0-21swapper/222:49:292
1937699130,12cyclictest31432-21sshd21:25:512
1937699130,12cyclictest0-21swapper/223:35:412
1937699130,12cyclictest0-21swapper/223:13:412
1937699130,12cyclictest0-21swapper/222:17:002
1937699130,12cyclictest0-21swapper/222:06:582
1937699130,12cyclictest0-21swapper/200:06:292
1937699130,10cyclictest0-21swapper/222:36:542
1937299139,1cyclictest28953-21bash00:02:011
1937299139,1cyclictest0-21swapper/100:26:391
1937299138,2cyclictest21868-21bash21:55:591
1937299133,9cyclictest21-21ksoftirqd/100:31:371
1937299133,1cyclictest5479-21sshd22:05:521
1937299133,1cyclictest21-21ksoftirqd/100:07:191
1937299131,1cyclictest0-21swapper/122:46:111
1937299130,4cyclictest0-21swapper/100:15:131
1937299130,2cyclictest0-21swapper/122:34:111
1937299130,12cyclictest2950-21bash23:31:061
1937299130,12cyclictest0-21swapper/123:46:081
1937299130,12cyclictest0-21swapper/123:42:041
1937299130,12cyclictest0-21swapper/122:35:551
1937299130,12cyclictest0-21swapper/122:22:171
1937299130,12cyclictest0-21swapper/121:50:161
1937299130,12cyclictest0-21swapper/121:21:161
1937299130,10cyclictest27672-21sshd21:36:461
1936999139,4cyclictest0-21swapper/020:07:420
1936999139,3cyclictest0-21swapper/021:28:320
1936999138,2cyclictest0-21swapper/000:37:080
1936999133,9cyclictest0-21swapper/023:10:140
1936999130,9cyclictest0-21swapper/023:55:520
1936999130,10cyclictest23251-21sshd23:30:070
1938099122,9cyclictest115750irq/25-eth022:10:573
1938099120,11cyclictest20823-21bash00:01:143
1938099120,11cyclictest0-21swapper/322:46:273
1937699128,3cyclictest0-21swapper/200:00:052
1937699128,1cyclictest0-21swapper/222:29:012
1937699122,1cyclictest0-21swapper/221:30:282
1937699121,10cyclictest0-21swapper/223:55:132
19376991210,1cyclictest23649-21bash21:50:282
1937699120,11cyclictest31966-21sshd21:45:362
1937299129,2cyclictest0-21swapper/123:25:211
1937299129,2cyclictest0-21swapper/100:35:171
1937299120,11cyclictest25257-21sshd23:01:191
1936999120,10cyclictest0-21swapper/020:21:430
1938099111,1cyclictest8206-21kworker/3:119:10:183
19380991110,1cyclictest8206-21kworker/3:119:50:203
1938099110,11cyclictest0-21swapper/319:56:403
1937699119,1cyclictest0-21swapper/222:20:112
19376991110,1cyclictest0-21swapper/220:28:272
1937699110,10cyclictest32383-21sshd23:25:112
1937299119,1cyclictest0-21swapper/119:15:021
19372991111,0cyclictest729-40kipmi020:05:191
1937299110,1cyclictest0-21swapper/120:52:481
1937299110,1cyclictest0-21swapper/120:16:061
1937299110,1cyclictest0-21swapper/119:56:571
1937299110,11cyclictest0-21swapper/121:06:511
1937299110,11cyclictest0-21swapper/120:46:181
1936999110,6cyclictest0-21swapper/020:10:190
1936999110,6cyclictest0-21swapper/020:10:190
110562110,0sleep20-21swapper/200:15:152
1938099109,1cyclictest0-21swapper/320:08:383
1938099109,1cyclictest0-21swapper/320:01:403
1938099109,1cyclictest0-21swapper/319:39:283
1938099100,1cyclictest987-21sh20:45:173
1938099100,10cyclictest0-21swapper/321:03:283
1938099100,10cyclictest0-21swapper/320:59:103
1938099100,10cyclictest0-21swapper/320:20:363
1938099100,10cyclictest0-21swapper/319:29:563
1937699109,1cyclictest0-21swapper/221:09:132
1937699109,1cyclictest0-21swapper/220:00:342
1937699109,1cyclictest0-21swapper/219:31:252
1937699109,1cyclictest0-21swapper/219:21:392
1937699100,9cyclictest0-21swapper/219:35:172
1937699100,1cyclictest0-21swapper/219:47:342
1937699100,1cyclictest0-21swapper/219:40:082
1937699100,1cyclictest0-21swapper/219:11:532
1937299109,1cyclictest0-21swapper/120:25:371
1937299109,1cyclictest0-21swapper/120:02:301
1937299109,1cyclictest0-21swapper/119:29:461
1937299109,1cyclictest0-21swapper/119:14:121
1937299108,1cyclictest0-21swapper/120:20:141
1937299100,1cyclictest0-21swapper/121:03:011
1937299100,1cyclictest0-21swapper/120:36:321
1937299100,1cyclictest0-21swapper/120:33:241
1937299100,1cyclictest0-21swapper/120:10:311
1937299100,1cyclictest0-21swapper/120:10:311
1937299100,10cyclictest0-21swapper/120:42:071
1937299100,10cyclictest0-21swapper/119:33:221
1936999109,1cyclictest0-21swapper/019:54:320
1936999100,9cyclictest944-21grep20:45:170
1936999100,9cyclictest0-21swapper/020:55:160
1936999100,9cyclictest0-21swapper/020:30:170
1936999100,9cyclictest0-21swapper/020:25:410
1936999100,1cyclictest9682-21perl19:55:130
1936999100,1cyclictest5027-21fschecks_time19:45:150
1936999100,1cyclictest0-21swapper/019:30:170
1936999100,10cyclictest0-21swapper/019:10:190
193809998,1cyclictest0-21swapper/320:31:593
193809998,1cyclictest0-21swapper/320:17:073
193809998,1cyclictest0-21swapper/319:43:453
193809998,1cyclictest0-21swapper/319:34:423
193809997,1cyclictest16855-21fschecks_time20:10:183
193809997,1cyclictest16855-21fschecks_time20:10:183
193769998,1cyclictest0-21swapper/220:24:162
193769998,1cyclictest0-21swapper/220:12:112
193769998,1cyclictest0-21swapper/220:12:112
193769998,1cyclictest0-21swapper/220:07:332
193769998,1cyclictest0-21swapper/219:55:142
193769998,1cyclictest0-21swapper/219:53:432
193769998,1cyclictest0-21swapper/219:25:022
193729998,1cyclictest0-21swapper/119:48:561
193729998,1cyclictest0-21swapper/119:38:421
193729997,1cyclictest0-21swapper/119:20:161
193699997,1cyclictest3297-21fschecks_time20:50:190
193699997,1cyclictest31071-21perl20:40:160
193699991,5cyclictest0-21swapper/020:15:310
193699990,7cyclictest0-21swapper/019:25:070
193699990,7cyclictest0-21swapper/019:15:160
193699990,6cyclictest0-21swapper/019:40:230
193809987,1cyclictest0-21swapper/319:46:553
193809987,1cyclictest0-21swapper/319:17:443
193809986,1cyclictest3323-21tune2fs20:50:193
193809981,3cyclictest0-21swapper/321:07:143
193809980,8cyclictest0-21swapper/320:39:123
193809980,3cyclictest0-21swapper/320:26:043
193769987,1cyclictest0-21swapper/220:40:452
193769986,1cyclictest21787-21munin-node19:15:002
193769986,1cyclictest0-21swapper/220:45:162
193729985,1cyclictest1579-21kworker/1:119:52:491
193699987,1cyclictest0-21swapper/019:37:200
193699981,6cyclictest0-21swapper/019:20:230
193699980,8cyclictest0-21swapper/021:02:510
193809970,6cyclictest0-21swapper/320:40:163
193809970,6cyclictest0-21swapper/319:20:143
193769973,4cyclictest0-21swapper/220:55:172
193769971,5cyclictest0-21swapper/220:15:212
193769970,6cyclictest0-21swapper/221:00:142
193769970,6cyclictest0-21swapper/220:50:152
193729971,5cyclictest0-21swapper/120:55:121
193699970,6cyclictest0-21swapper/021:05:160
193769960,5cyclictest0-21swapper/220:35:162
193729960,5cyclictest0-21swapper/119:40:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional