You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 07:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot0.osadl.org (updated Wed Apr 02, 2025 00:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2122321280,2sleep0728299cyclictest21:12:330
74021250,0sleep0732-21cp23:44:070
2674021190,1sleep3729999cyclictest22:10:593
6822210533,44sleep30-21swapper/319:06:563
1864021000,3sleep325-21rcuop/119:35:123
309332970,2sleep30-21swapper/322:05:183
212172960,6sleep20-21swapper/222:13:342
696829535,56sleep10-21swapper/119:08:461
36452950,2sleep20-21swapper/221:47:292
194862940,0sleep019488-21pmu-power21:15:210
71372920,1sleep30-21swapper/323:10:463
273132890,2sleep027293-21sshd21:58:500
96152880,1sleep20-21swapper/223:57:222
286822860,2sleep30-21swapper/322:51:033
146052860,0sleep00-21swapper/000:20:370
43332850,2sleep14329-21bash00:13:091
176712850,1sleep117669-21bash00:04:351
155832840,1sleep20-21swapper/222:00:402
53112820,4sleep3729999cyclictest22:21:093
308402820,2sleep00-21swapper/023:31:350
101332820,2sleep110122-21sshd23:04:511
319162810,2sleep10-21swapper/123:10:081
46142790,0sleep1728599cyclictest23:19:541
42612790,2sleep24259-21sed23:35:142
175862790,1sleep30-21swapper/323:05:333
111352790,2sleep211133-21sshd22:49:292
85902780,2sleep225-21rcuop/100:07:012
253102780,1sleep01848-21nfsd22:04:470
16882780,2sleep10-21swapper/121:13:461
134282770,0sleep213425-21bash22:43:282
6442760,4sleep2729399cyclictest23:22:322
153852760,1sleep315372-21sshd21:12:003
213232750,2sleep30-21swapper/323:02:493
161682750,2sleep10-21swapper/122:13:051
698927434,11sleep20-21swapper/219:09:042
266812740,1sleep00-21swapper/023:52:520
134492740,1sleep1728599cyclictest21:24:001
105722740,1sleep00-21swapper/022:06:270
178572720,1sleep00-21swapper/000:11:110
268632710,2sleep126848-21sshd22:35:271
57012700,3sleep30-21swapper/300:19:523
201552700,2sleep120154-21sshd22:01:071
277612680,2sleep10-21swapper/100:09:001
75852650,3sleep3729999cyclictest00:00:163
234102630,2sleep0728299cyclictest22:50:310
690326035,13sleep00-21swapper/019:07:570
54452370,0sleep35448-21id22:30:233
120502360,1sleep31847-21nfsd23:32:553
94102300,3sleep30-21swapper/323:44:573
279132300,1sleep10-21swapper/123:49:561
96672290,3sleep30-21swapper/321:29:513
56312270,24sleep1135650irq/27-eth021:38:321
326382260,2sleep30-21swapper/323:25:373
311692264,20sleep310950irq/23-ehci_hcd22:39:013
206312260,1sleep00-21swapper/000:01:290
154192260,1sleep01847-21nfsd22:46:470
137812261,23sleep1135650irq/27-eth000:23:561
73912253,20sleep126950irq/26-ahci[00022:18:231
60402250,0sleep06036-21bash21:26:230
254302250,2sleep30-21swapper/321:22:023
152632250,1sleep015245-21sshd21:21:020
299932240,1sleep029990-21bash23:37:380
286182241,20sleep1135650irq/27-eth000:35:301
265172230,1sleep20-21swapper/222:20:122
75932220,19sleep3135650irq/27-eth022:46:033
304442221,19sleep3135650irq/27-eth000:05:543
165142224,16sleep210950irq/23-ehci_hcd22:07:022
124402220,1sleep20-21swapper/223:26:492
728299213,15cyclictest27850-21sshd23:28:190
262982210,2sleep20-21swapper/223:09:372
85312200,1sleep00-21swapper/000:33:160
260822200,18sleep3135650irq/27-eth022:41:343
62452190,17sleep1135650irq/27-eth000:26:281
298902190,17sleep326950irq/26-ahci[00022:29:473
729999186,3cyclictest2178-21sshd23:56:383
729999180,6cyclictest0-21swapper/322:02:373
729399184,0cyclictest0-21swapper/222:19:502
729399181,8cyclictest33-21ksoftirqd/222:55:562
7293991815,2cyclictest18848-21sshd21:30:342
7293991814,3cyclictest18822-21bash21:55:022
7293991813,4cyclictest33-21ksoftirqd/200:14:152
7293991810,2cyclictest33-21ksoftirqd/223:40:562
728599186,7cyclictest0-21swapper/122:23:311
728599180,6cyclictest0-21swapper/121:45:561
728299180,17cyclictest10-21rcuc/021:34:320
326332180,2sleep20-21swapper/221:19:442
250722180,2sleep31848-21nfsd21:40:173
250072180,0sleep225006-21bash23:34:102
214972180,2sleep10-21swapper/123:52:201
729999178,4cyclictest19097-21sshd00:14:393
729999177,5cyclictest16381-21sshd00:24:113
729999172,2cyclictest21480-21sshd23:15:183
729999170,15cyclictest9116-21sshd21:17:283
7293991712,4cyclictest19745-21sshd23:02:392
729399171,14cyclictest0-21swapper/200:28:282
728599176,3cyclictest21219-21sshd22:07:281
728599175,3cyclictest30501-21bash21:53:031
7285991713,3cyclictest4241-21sshd23:35:141
7285991712,4cyclictest2662-21sshd23:22:431
7285991711,3cyclictest144550irq/28-eth1-rx-20:08:271
728599170,12cyclictest2536-21sshd22:27:071
728599170,0cyclictest0-21swapper/123:40:231
728299170,16cyclictest1848-21nfsd22:28:150
728299170,15cyclictest11096-21bash22:15:350
51202170,0sleep00-21swapper/000:36:320
29052170,15sleep1135650irq/27-eth022:42:291
22022170,14sleep3135650irq/27-eth021:35:133
167842171,14sleep3135650irq/27-eth000:30:443
729999167,7cyclictest11808-21sshd00:27:023
729999163,11cyclictest21613-21bash21:30:503
729999160,7cyclictest10371-21sshd21:54:143
729999160,15cyclictest0-21swapper/321:46:513
729999160,11cyclictest8183-21munin-node23:20:163
729999160,11cyclictest30366-21sshd23:50:093
729999160,11cyclictest2385-21sshd23:35:053
729399166,5cyclictest0-21swapper/223:17:312
729399164,10cyclictest16073-21sshd21:21:072
729399161,6cyclictest0-21swapper/222:32:022
7293991612,3cyclictest36-21rcuop/200:20:112
729399161,14cyclictest0-21swapper/222:54:502
729399160,7cyclictest0-21swapper/220:21:162
729399160,11cyclictest135650irq/27-eth020:46:232
728599168,6cyclictest3168-21sshd21:56:311
728599166,2cyclictest5197-21sshd21:44:331
728599166,1cyclictest4988-21sshd23:29:151
728599166,1cyclictest4013-21sshd22:54:591
728599166,1cyclictest13649-21sshd21:30:121
728599165,2cyclictest0-21swapper/123:34:181
7285991612,3cyclictest6038-21sshd22:30:261
7285991610,5cyclictest10950irq/23-ehci_hcd20:40:591
728599160,4cyclictest11137-21sshd21:26:521
728599160,3cyclictest21531-21sshd23:55:231
728599160,11cyclictest22185-21sshd00:31:151
728299167,8cyclictest0-21swapper/023:56:420
7282991611,4cyclictest2614-21df21:35:130
728299160,1cyclictest30175-21sshd22:20:240
728299160,1cyclictest0-21swapper/023:45:040
728299160,15cyclictest4688-21cp21:41:230
728299160,15cyclictest26683-21sshd23:12:430
728299160,15cyclictest0-21swapper/023:00:130
728299160,15cyclictest0-21swapper/000:05:080
728299160,14cyclictest12727-21sshd21:51:200
297392160,0sleep00-21swapper/021:50:000
241112160,2sleep124094-21sshd21:18:551
729999159,4cyclictest6197-21sshd22:15:133
729999152,1cyclictest16522-21bash23:48:503
729999150,6cyclictest0-21swapper/320:35:213
729999150,3cyclictest10926-21sshd22:55:303
729999150,14cyclictest22390-21bash21:55:183
729999150,14cyclictest0-21swapper/300:36:063
729999150,13cyclictest2405-21runrttasks19:51:583
729399158,6cyclictest2419-21sshd21:41:102
729399157,7cyclictest0-21swapper/200:17:372
729399156,4cyclictest33-21ksoftirqd/221:51:222
729399155,9cyclictest33-21ksoftirqd/223:10:242
729399154,2cyclictest11861-21sshd00:00:372
729399154,10cyclictest30242-21sshd00:32:052
729399153,11cyclictest28681-21sshd21:25:282
729399153,11cyclictest10950irq/23-ehci_hcd19:45:302
7293991512,2cyclictest407-21sshd23:50:172
7293991510,4cyclictest24512-21sshd23:46:252
729399150,13cyclictest19731-21sshd22:25:392
729399150,10cyclictest0-21swapper/219:55:222
728599157,3cyclictest135650irq/27-eth000:15:271
728599150,9cyclictest5323-21sshd22:45:491
728599150,8cyclictest25092-21sshd22:56:541
728599150,13cyclictest0-21swapper/120:45:181
728299157,7cyclictest13214-21sshd22:37:150
728299157,7cyclictest0-21swapper/000:18:090
728299153,11cyclictest0-21swapper/021:05:380
7282991511,1cyclictest0-21swapper/022:10:100
728299150,1cyclictest0-21swapper/022:56:540
728299150,14cyclictest0-21swapper/000:28:130
135932150,0sleep00-21swapper/023:05:100
729999147,6cyclictest114622sleep319:18:103
729999143,10cyclictest10950irq/23-ehci_hcd20:25:113
729999141,4cyclictest0-21swapper/320:00:213
7299991411,2cyclictest10950irq/23-ehci_hcd20:48:183
7299991410,3cyclictest10950irq/23-ehci_hcd20:17:083
7299991410,3cyclictest10950irq/23-ehci_hcd20:08:463
729999140,5cyclictest8120-21runrttasks19:10:183
729999140,13cyclictest0-21swapper/319:40:353
729399148,5cyclictest135650irq/27-eth019:30:452
729399147,6cyclictest3946-21sendmail_mailtr20:15:212
729399142,11cyclictest1848-21nfsd00:36:522
729399141,11cyclictest0-21swapper/221:11:102
729399141,11cyclictest0-21swapper/220:40:162
729399141,11cyclictest0-21swapper/220:35:122
729399140,5cyclictest26523-21munin-node19:50:252
729399140,13cyclictest0-21swapper/220:06:342
729399140,12cyclictest3010-21bash21:35:142
729399140,12cyclictest1690-21sshd22:36:082
728599145,0cyclictest0-21swapper/123:06:081
728599144,2cyclictest10950irq/23-ehci_hcd19:47:051
7285991411,2cyclictest17880-21ls19:30:251
7285991410,3cyclictest144550irq/28-eth1-rx-21:01:031
7285991410,3cyclictest10950irq/23-ehci_hcd19:17:421
728299149,2cyclictest24295-21sshd22:32:130
728299148,5cyclictest0-21swapper/020:03:250
728299140,8cyclictest22242-21sshd22:41:100
728299140,13cyclictest14912-21sshd23:20:440
728299140,13cyclictest0-21swapper/020:44:110
728299140,13cyclictest0-21swapper/020:20:050
66972140,13sleep3135650irq/27-eth020:22:003
729999138,4cyclictest1848-21nfsd19:55:433
729999136,3cyclictest10950irq/23-ehci_hcd19:25:263
729999130,12cyclictest2405-21runrttasks21:01:103
729999130,12cyclictest1848-21nfsd20:10:213
729999130,12cyclictest0-21swapper/320:55:273
729999130,12cyclictest0-21swapper/319:30:443
729999130,11cyclictest23389-21df19:45:123
729999130,11cyclictest12177-21munin-node19:20:143
729399138,4cyclictest19341-21if_err_eth019:35:152
729399137,5cyclictest0-21swapper/220:25:142
729399137,5cyclictest0-21swapper/220:02:182
729399131,3cyclictest0-21swapper/221:05:172
729399131,3cyclictest0-21swapper/220:30:092
7293991311,1cyclictest0-21swapper/220:55:102
7293991311,1cyclictest0-21swapper/219:28:582
7293991311,1cyclictest0-21swapper/219:19:012
729399131,10cyclictest0-21swapper/221:00:152
729399131,10cyclictest0-21swapper/220:50:192
729399130,12cyclictest135650irq/27-eth019:41:012
729399130,12cyclictest135650irq/27-eth019:23:222
729399130,11cyclictest8095-21iostat19:10:162
728599139,3cyclictest10950irq/23-ehci_hcd20:00:541
728599137,5cyclictest0-21swapper/121:07:211
728599131,11cyclictest1848-21nfsd20:51:141
728599130,5cyclictest0-21swapper/120:28:541
728599130,4cyclictest27124-21taskset19:51:331
728599130,12cyclictest1848-21nfsd20:55:531
728599130,12cyclictest0-21swapper/120:38:061
728599130,12cyclictest0-21swapper/119:40:021
728599130,12cyclictest0-21swapper/119:35:201
728599130,12cyclictest0-21swapper/119:25:121
728599130,12cyclictest0-21swapper/119:21:061
728599130,12cyclictest0-21swapper/119:14:331
728599130,10cyclictest135650irq/27-eth020:20:011
728299138,4cyclictest0-21swapper/020:35:110
728299137,5cyclictest0-21swapper/019:49:530
728299137,5cyclictest0-21swapper/019:28:320
728299137,4cyclictest0-21swapper/020:46:220
728299134,9cyclictest0-21swapper/019:20:060
7282991311,1cyclictest0-21swapper/019:52:310
728299130,12cyclictest6112-21sshd23:16:540
728299130,12cyclictest0-21swapper/021:02:120
728299130,12cyclictest0-21swapper/020:31:170
728299130,12cyclictest0-21swapper/020:25:320
728299130,12cyclictest0-21swapper/020:16:590
728299130,12cyclictest0-21swapper/020:14:020
728299130,12cyclictest0-21swapper/019:55:440
728299130,12cyclictest0-21swapper/019:40:320
728299130,12cyclictest0-21swapper/019:38:130
728299130,12cyclictest0-21swapper/019:15:060
729999128,3cyclictest135650irq/27-eth021:05:193
729999120,3cyclictest18212-21xargs20:50:283
729999120,3cyclictest12471-21munin-node20:40:123
729999120,10cyclictest8125-21munin-node20:30:093
729399120,3cyclictest992-21hddtemp_smartct20:10:122
728599120,11cyclictest1927-21snmpd19:59:381
728599120,11cyclictest0-21swapper/120:31:021
728599120,11cyclictest0-21swapper/120:15:101
728599120,11cyclictest0-21swapper/120:10:581
728299129,1cyclictest0-21swapper/019:30:560
728299127,5cyclictest0-21swapper/020:51:190
728299126,5cyclictest0-21swapper/019:11:200
728299120,12cyclictest0-21swapper/020:57:480
728299120,11cyclictest0-21swapper/020:07:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional