You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Thu Apr 03, 2025 00:44:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
177221570,0sleep01766-21sshd21:58:490
2072721300,2sleep31848-21nfsd21:57:323
703721270,3sleep01195899cyclictest22:48:230
2338221270,4sleep21196799cyclictest22:46:492
1350921220,2sleep30-21swapper/321:41:333
2380921190,0sleep323794-21sshd22:16:073
1661721180,5sleep21196799cyclictest21:23:362
145132960,0sleep010-21rcuc/021:11:080
56362940,1sleep00-21swapper/021:31:440
104482920,0sleep00-21swapper/023:54:000
297902910,2sleep31846-21nfsd21:21:453
192142900,2sleep21196799cyclictest21:17:432
295882880,2sleep229590-21id22:38:072
93632870,5sleep30-21swapper/322:11:463
62332870,2sleep30-21swapper/300:19:233
1145328734,49sleep10-21swapper/119:06:421
191762860,1sleep019169-21sshd00:01:010
1149028634,48sleep30-21swapper/319:07:123
219912840,5sleep11196299cyclictest21:24:071
155672840,2sleep215566-21bash23:07:522
143702840,2sleep30-21swapper/321:35:313
316532830,2sleep30-21swapper/321:31:073
278932830,1sleep00-21swapper/022:07:270
63812820,2sleep26379-21sshd22:29:532
148502820,1sleep00-21swapper/021:17:180
49152810,2sleep14914-21bash21:16:181
21202810,0sleep12119-21bash22:17:101
12572810,2sleep30-21swapper/323:43:403
125542810,2sleep30-21swapper/323:23:093
4042800,2sleep11847-21nfsd23:09:361
134942800,0sleep00-21swapper/022:21:130
37142780,2sleep10-21swapper/123:59:371
102182780,0sleep00-21swapper/000:23:060
71452770,2sleep10-21swapper/123:13:161
258922770,2sleep30-21swapper/323:52:183
81482750,2sleep30-21swapper/321:10:313
218882750,1sleep01848-21nfsd19:30:190
112942730,3sleep21196799cyclictest22:18:032
177452720,2sleep317742-21sshd23:57:483
92432710,4sleep20-21swapper/200:22:592
102252710,2sleep10-21swapper/123:22:541
317312670,1sleep21196799cyclictest22:07:502
23982640,4sleep11196299cyclictest23:43:471
16162640,3sleep31197199cyclictest22:56:593
194242620,4sleep21196799cyclictest21:42:082
300462610,3sleep11196299cyclictest00:28:551
1166025733,12sleep00-21swapper/019:09:220
1159725136,9sleep20-21swapper/219:08:312
118772500,1sleep011874-21sshd23:01:010
96922390,37sleep2135650irq/27-eth000:16:142
43612360,3sleep10-21swapper/121:37:431
116862360,1sleep20-21swapper/222:57:582
78772340,4sleep3431ktimersoftd/322:23:493
191762330,0sleep219174-21bash00:27:442
205862310,29sleep3135650irq/27-eth000:07:423
14812300,2sleep20-21swapper/223:22:032
203262297,20sleep3135650irq/27-eth023:45:303
180202290,3sleep10-21swapper/123:54:441
141012290,2sleep214100-21sshd23:57:262
133142290,2sleep041ktimersoftd/023:47:580
46062280,1sleep041ktimersoftd/023:06:470
21242280,2sleep32095-21sshd23:12:463
183482280,1sleep10-21swapper/122:52:281
180732260,2sleep318067-21sshd23:29:593
67072250,2sleep26654-21sshd23:10:092
37432250,1sleep00-21swapper/023:31:350
22412250,2sleep12242-21sshd23:19:051
8552240,2sleep1824-21sshd22:14:051
311592240,0sleep00-21swapper/000:05:290
3012240,1sleep31197199cyclictest00:25:413
11971992414,4cyclictest25570-21sshd22:50:083
1197199240,3cyclictest0-21swapper/322:02:153
34562230,3sleep33454-21sshd23:15:593
228972230,21sleep2135650irq/27-eth022:43:372
320222220,1sleep02405-21runrttasks00:32:270
1197199225,16cyclictest44-21ksoftirqd/322:26:543
11752220,2sleep21171-21bash21:28:212
1197199217,9cyclictest44-21ksoftirqd/321:54:313
11971992117,2cyclictest20725-21sshd23:01:553
97822200,4sleep31197199cyclictest00:03:253
261562200,1sleep20-21swapper/200:35:142
136512200,18sleep1135650irq/27-eth022:09:131
1197199205,3cyclictest382-21sshd00:35:483
1197199204,15cyclictest44-21ksoftirqd/322:47:043
1197199200,3cyclictest3508-21sshd21:46:493
1196799201,17cyclictest0-21swapper/221:53:552
19902190,2sleep11984-21rm22:23:151
160932190,1sleep01848-21nfsd23:23:290
1197199197,11cyclictest44-21ksoftirqd/322:31:113
1197199196,9cyclictest44-21ksoftirqd/322:35:423
1197199196,0cyclictest0-21swapper/321:26:413
1197199193,15cyclictest44-21ksoftirqd/322:41:173
1197199190,4cyclictest24700-21bash21:18:153
1196799193,8cyclictest15525-21bash00:34:102
1196799193,8cyclictest13105-21sshd23:38:422
11967991913,4cyclictest20059-21sshd00:10:442
11967991912,5cyclictest4417-21sadc21:50:002
1195899197,7cyclictest25976-21sshd21:24:300
11958991910,5cyclictest5571-21sshd22:35:450
1197199188,9cyclictest44-21ksoftirqd/323:39:423
1197199187,9cyclictest44-21ksoftirqd/300:23:053
1197199187,10cyclictest44-21ksoftirqd/300:10:133
1197199182,7cyclictest17475-21sshd23:08:043
1197199182,3cyclictest23939-21sshd23:33:343
1196299186,3cyclictest22291-21sshd21:33:201
1196299180,12cyclictest12909-21sshd22:36:291
1195899181,12cyclictest0-21swapper/023:14:080
53872170,2sleep10-21swapper/122:57:221
265202170,1sleep026514-21sshd00:11:230
230812170,1sleep223083-21id21:36:232
128022173,12sleep1135650irq/27-eth023:47:571
1197199175,11cyclictest44-21ksoftirqd/322:06:183
1197199172,11cyclictest44-21ksoftirqd/300:31:013
1197199170,15cyclictest0-21swapper/319:55:213
1196799174,3cyclictest3558-21cp23:43:532
1196799172,11cyclictest20342-21bash23:01:532
11967991714,2cyclictest16406-21bash22:52:172
11967991712,3cyclictest1927-21snmpd23:51:052
1196299175,10cyclictest0-21swapper/123:30:241
11962991712,4cyclictest22999-21sshd21:45:341
1196299170,4cyclictest25916-21sshd00:01:411
1196299170,13cyclictest135650irq/27-eth000:36:121
1195899174,2cyclictest22106-21sshd22:31:150
1195899170,9cyclictest0-21swapper/021:45:090
1195899170,16cyclictest31406-21sshd00:29:050
1195899170,15cyclictest13405-21sshd21:38:340
1195899170,13cyclictest7705-21sshd22:42:070
1195899170,13cyclictest0-21swapper/023:44:110
1195899170,12cyclictest0-21swapper/023:26:050
1197199163,12cyclictest0-21swapper/320:40:203
1196799169,3cyclictest10950irq/23-ehci_hcd19:10:132
1196799164,11cyclictest10950irq/23-ehci_hcd20:54:372
1196799161,7cyclictest47-21rcuop/322:22:452
1196799160,14cyclictest5885-21sshd22:14:342
1196799160,14cyclictest1972-21sshd22:01:502
1196799160,14cyclictest1893-21nscd23:28:072
1196299168,3cyclictest823-21sshd22:47:471
1196299168,3cyclictest0-21swapper/122:29:591
1196299168,3cyclictest0-21swapper/100:07:281
1196299165,10cyclictest28646-21sshd23:02:441
1196299160,4cyclictest12996-21sshd21:59:531
1196299160,3cyclictest135650irq/27-eth023:35:171
1196299160,3cyclictest0-21swapper/122:41:451
1196299160,14cyclictest28606-21bash22:31:531
1196299160,14cyclictest1567-21sshd21:28:241
1196299160,14cyclictest0-21swapper/121:13:221
1196299160,14cyclictest0-21swapper/100:24:081
1195899168,7cyclictest0-21swapper/021:00:150
1195899166,1cyclictest0-21swapper/021:26:210
11958991610,2cyclictest0-21swapper/023:56:120
1195899160,7cyclictest28942-21bash22:25:510
1195899160,1cyclictest0-21swapper/000:37:330
1195899160,15cyclictest0-21swapper/020:20:180
1196799157,4cyclictest4001-21sshd21:55:542
1196799153,11cyclictest15744-21sshd21:14:212
1196799152,9cyclictest15841-21bash23:48:132
1196799152,8cyclictest21412-21perl21:30:132
1196799151,2cyclictest17243-21sshd00:07:222
11967991511,3cyclictest24439-21sshd22:34:372
1196799151,10cyclictest0-21swapper/223:32:402
1196799150,4cyclictest9063-21sshd23:19:442
1196299152,1cyclictest3949-21sshd00:19:101
11962991510,3cyclictest23667-21bash21:42:321
1196299150,6cyclictest26284-21sshd22:01:011
1196299150,6cyclictest22346-21sshd00:10:591
1196299150,3cyclictest8789-21bash23:25:551
1195899152,9cyclictest15686-21sshd00:16:520
1195899150,7cyclictest0-21swapper/021:53:440
1195899150,3cyclictest14067-21id23:38:460
1195899150,1cyclictest221112sleep020:43:230
1195899150,14cyclictest0-21swapper/022:58:470
1195899150,11cyclictest0-21swapper/022:10:440
1197199143,3cyclictest18772-21latency20:35:173
1197199143,10cyclictest10950irq/23-ehci_hcd19:26:463
11971991412,1cyclictest0-21swapper/319:10:193
1197199140,3cyclictest18055-21grep19:20:243
1197199140,12cyclictest0-21swapper/319:45:233
1196799149,4cyclictest29386-21sshd00:02:062
1196799143,10cyclictest10950irq/23-ehci_hcd20:07:252
1196799142,9cyclictest1096-21perf21:10:002
1196799140,5cyclictest2405-21runrttasks19:53:302
1196799140,5cyclictest2405-21runrttasks19:53:302
1196299147,3cyclictest10950irq/23-ehci_hcd20:45:131
1196299143,10cyclictest10950irq/23-ehci_hcd20:35:081
1196299143,10cyclictest10950irq/23-ehci_hcd20:16:451
1196299141,1cyclictest0-21swapper/120:25:121
11962991410,3cyclictest5596-21minicom20:06:401
1196299140,3cyclictest1848-21nfsd00:30:001
1196299140,13cyclictest31283-21processes19:50:231
1196299140,13cyclictest31283-21processes19:50:221
1196299140,13cyclictest19663-21sshd21:51:191
1196299140,12cyclictest1927-21snmpd20:50:481
1195899147,6cyclictest31320-21bash22:16:510
11958991410,3cyclictest0-21swapper/020:17:030
1195899140,8cyclictest17107-21sshd23:17:230
1195899140,3cyclictest23020-21sshd22:52:560
1195899140,13cyclictest2848-21bash21:43:390
1195899140,13cyclictest0-21swapper/020:55:420
1195899140,13cyclictest0-21swapper/020:32:040
1195899140,13cyclictest0-21swapper/019:42:340
1195899140,12cyclictest0-21swapper/022:00:020
1195899140,12cyclictest0-21swapper/020:36:150
1197199135,0cyclictest0-21swapper/319:35:153
1197199134,7cyclictest44-21ksoftirqd/320:45:223
1197199132,10cyclictest135650irq/27-eth020:27:033
11971991311,1cyclictest0-21swapper/321:05:253
1197199130,12cyclictest44-21ksoftirqd/319:40:473
1197199130,12cyclictest0-21swapper/320:16:173
1197199130,12cyclictest0-21swapper/320:07:033
1197199130,12cyclictest0-21swapper/320:01:033
1197199130,12cyclictest0-21swapper/319:51:553
1197199130,12cyclictest0-21swapper/319:51:553
1197199130,11cyclictest21332-21diskstats19:30:153
1197199130,10cyclictest36-21rcuop/220:57:203
1196799139,3cyclictest135650irq/27-eth021:00:202
1196799139,3cyclictest135650irq/27-eth020:56:072
1196799138,4cyclictest2405-21runrttasks19:35:392
1196799138,1cyclictest10950irq/23-ehci_hcd20:35:212
1196799132,10cyclictest135650irq/27-eth019:47:172
11967991311,1cyclictest0-21swapper/220:31:522
11967991311,1cyclictest0-21swapper/219:25:052
1196799130,12cyclictest1848-21nfsd20:25:462
1196799130,12cyclictest0-21swapper/220:22:382
1196799130,12cyclictest0-21swapper/220:10:282
1196799130,12cyclictest0-21swapper/220:01:202
1196799130,12cyclictest0-21swapper/219:55:062
1196799130,12cyclictest0-21swapper/219:41:082
1196799130,12cyclictest0-21swapper/219:31:542
1196299139,3cyclictest144550irq/28-eth1-rx-20:02:101
1196299134,7cyclictest0-21swapper/121:00:131
1196299131,11cyclictest1848-21nfsd19:15:121
11962991311,1cyclictest0-21swapper/120:42:311
11962991311,1cyclictest0-21swapper/119:37:421
1196299130,4cyclictest16587-21munin-node19:20:111
1196299130,4cyclictest0-21swapper/120:56:391
1196299130,12cyclictest1848-21nfsd21:07:301
1196299130,12cyclictest0-21swapper/120:31:021
1196299130,12cyclictest0-21swapper/119:57:401
1196299130,12cyclictest0-21swapper/119:47:481
1196299130,12cyclictest0-21swapper/119:40:341
1196299130,12cyclictest0-21swapper/119:30:281
1196299130,12cyclictest0-21swapper/119:25:181
1196299130,11cyclictest6959-21munin-node20:10:151
1196299130,11cyclictest11523-21df_inode20:20:131
1195899137,5cyclictest6099-21chrt20:05:320
1195899137,5cyclictest0-21swapper/020:26:390
1195899134,8cyclictest0-21swapper/019:15:290
11958991310,2cyclictest6469-21kworker/0:119:37:420
1195899130,1cyclictest0-21swapper/019:54:580
1195899130,1cyclictest0-21swapper/019:54:580
1195899130,12cyclictest13058-21grep19:10:190
1195899130,12cyclictest0-21swapper/021:05:010
1195899130,12cyclictest0-21swapper/020:45:180
1195899130,12cyclictest0-21swapper/019:21:060
1195899130,10cyclictest0-21swapper/019:28:100
1197199124,7cyclictest44-21ksoftirqd/321:00:113
1197199124,7cyclictest44-21ksoftirqd/320:50:163
1197199124,7cyclictest44-21ksoftirqd/319:15:163
1197199120,3cyclictest7450-21munin-node20:10:183
1197199120,11cyclictest0-21swapper/320:20:113
1196799128,3cyclictest15118-21memory19:15:202
1196799127,3cyclictest9210-21df20:15:132
1196799120,3cyclictest20897-21hddtemp_smartct20:40:152
1196799120,10cyclictest22801-21df_inode20:45:142
1196799120,10cyclictest0-21swapper/219:20:132
1196299120,11cyclictest12702-21idleruntime19:10:161
1195899123,1cyclictest0-21swapper/019:46:230
11958991210,1cyclictest0-21swapper/020:51:250
1195899120,11cyclictest3308-21ntp_states20:00:190
1195899120,11cyclictest0-21swapper/020:10:140
1195899120,11cyclictest0-21swapper/019:55:160
1197199113,7cyclictest44-21ksoftirqd/320:30:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional