You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot6.osadl.org (updated Thu Apr 03, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3247423240,5sleep125-21ksoftirqd/121:07:251
629821540,3sleep03276499cyclictest22:25:220
629821540,3sleep03276499cyclictest22:25:220
1518911110,4ptp4l401ktimersoftd/321:06:493
51442660,4sleep03276499cyclictest00:39:390
323302644,10sleep20-21swapper/221:05:332
64172570,2sleep10-21swapper/123:23:201
31699527,10cyclictest41-21ksoftirqd/301:10:103
31699517,9cyclictest41-21ksoftirqd/322:00:073
31699513,7cyclictest11853-21sed01:24:573
31699507,8cyclictest41-21ksoftirqd/301:34:573
31699507,8cyclictest41-21ksoftirqd/300:50:053
31699507,10cyclictest41-21ksoftirqd/302:30:113
31699506,9cyclictest41-21ksoftirqd/301:55:143
31699504,7cyclictest41-21ksoftirqd/322:50:123
31699497,10cyclictest41-21ksoftirqd/321:29:513
31699496,9cyclictest41-21ksoftirqd/322:10:183
31699496,9cyclictest41-21ksoftirqd/301:15:093
31699496,9cyclictest41-21ksoftirqd/300:54:573
31699496,8cyclictest16916-21ssh01:30:303
31699495,9cyclictest41-21ksoftirqd/322:24:533
31699495,9cyclictest41-21ksoftirqd/322:24:533
31699495,9cyclictest41-21ksoftirqd/300:20:153
31699495,8cyclictest41-21ksoftirqd/300:04:553
31699494,8cyclictest41-21ksoftirqd/301:00:123
31699494,7cyclictest41-21ksoftirqd/323:34:553
31699494,7cyclictest41-21ksoftirqd/300:40:123
31699494,10cyclictest41-21ksoftirqd/323:39:543
31699493,8cyclictest41-21ksoftirqd/300:30:103
31699493,7cyclictest41-21ksoftirqd/323:40:153
316994919,8cyclictest41-21ksoftirqd/302:17:383
30699490,17cyclictest14054-21ssh01:28:412
31699485,9cyclictest41-21ksoftirqd/323:14:543
31699484,7cyclictest41-21ksoftirqd/300:35:133
31699482,8cyclictest41-21ksoftirqd/321:50:113
31699476,9cyclictest41-21ksoftirqd/321:15:103
31699476,8cyclictest41-21ksoftirqd/323:14:563
31699476,8cyclictest41-21ksoftirqd/322:30:143
31699475,7cyclictest7772-21munin-run02:39:583
31699474,7cyclictest25840-21diskmemload23:50:193
316994740,4cyclictest41-21ksoftirqd/300:09:593
316994740,4cyclictest41-21ksoftirqd/300:09:593
31699473,9cyclictest41-21ksoftirqd/301:50:113
31699473,9cyclictest41-21ksoftirqd/301:50:113
31699473,8cyclictest41-21ksoftirqd/322:09:523
31699471,8cyclictest41-21ksoftirqd/323:49:343
30699475,9cyclictest30388-21sh01:47:112
72882460,2sleep10-21swapper/101:56:531
31699467,9cyclictest41-21ksoftirqd/322:25:223
31699467,9cyclictest41-21ksoftirqd/322:25:223
31699465,10cyclictest41-21ksoftirqd/302:00:133
31699464,9cyclictest41-21ksoftirqd/300:45:293
31699463,8cyclictest41-21ksoftirqd/323:20:193
31699463,6cyclictest41-21ksoftirqd/321:49:523
31699463,14cyclictest41-21ksoftirqd/301:40:213
316994621,4cyclictest41-21ksoftirqd/323:54:593
30699460,17cyclictest8229-21taskset01:58:392
30699460,12cyclictest9954-21chrt00:44:512
30699460,12cyclictest4168-21awk22:20:222
30699460,12cyclictest4168-21awk22:20:222
31699456,9cyclictest41-21ksoftirqd/322:55:133
31699456,9cyclictest41-21ksoftirqd/321:55:093
31699456,9cyclictest41-21ksoftirqd/321:29:523
31699456,10cyclictest41-21ksoftirqd/321:35:143
31699455,9cyclictest41-21ksoftirqd/322:45:083
31699454,9cyclictest41-21ksoftirqd/323:05:133
31699454,9cyclictest41-21ksoftirqd/300:07:013
31699454,10cyclictest41-21ksoftirqd/302:10:123
31699453,7cyclictest41-21ksoftirqd/323:00:213
31699452,6cyclictest41-21ksoftirqd/323:29:263
316994519,7cyclictest41-21ksoftirqd/302:23:173
30699450,16cyclictest30872-21ssh02:28:192
30699450,13cyclictest7710-21ssh23:25:222
30699450,12cyclictest6222-21ssh23:22:582
30699450,12cyclictest26767-21ssh23:49:272
30699450,12cyclictest25840-21diskmemload01:21:162
31699447,9cyclictest41-21ksoftirqd/300:19:023
31699446,10cyclictest41-21ksoftirqd/322:40:083
31699445,8cyclictest41-21ksoftirqd/321:40:053
31699445,10cyclictest41-21ksoftirqd/301:47:333
31699443,8cyclictest41-21ksoftirqd/302:09:583
31699443,7cyclictest41-21ksoftirqd/322:35:073
31699443,7cyclictest41-21ksoftirqd/321:20:103
31699443,7cyclictest41-21ksoftirqd/321:10:093
31699442,8cyclictest41-21ksoftirqd/301:25:003
30699440,13cyclictest11805-21ptp4l-jitter23:30:162
30699440,12cyclictest6643-21ssh02:37:122
30699440,12cyclictest3003-21ssh23:58:242
30699440,12cyclictest15198-21taskset00:13:152
30699440,12cyclictest15198-21taskset00:13:152
30699440,11cyclictest25840-21diskmemload01:38:032
30699440,11cyclictest25840-21diskmemload01:31:132
30699440,11cyclictest19343-21ssh00:17:242
31699433,7cyclictest41-21ksoftirqd/302:28:263
31699432,7cyclictest41-21ksoftirqd/300:27:513
31699431,7cyclictest41-21ksoftirqd/301:06:393
30699430,14cyclictest22213-21missed_timers00:20:192
30699430,12cyclictest7928-21ssh00:03:582
30699430,12cyclictest27699-21taskset02:21:112
30699430,12cyclictest18741-21irqrtprio21:45:122
30699430,11cyclictest12006-21ssh00:48:192
30699420,16cyclictest5622-21taskset21:16:182
30699420,13cyclictest19660-21ssh02:12:152
30699420,12cyclictest5137-21ssh00:39:392
30699420,12cyclictest28691-21ssh01:07:372
30699420,12cyclictest27030-21chrt22:02:152
31699413,8cyclictest41-21ksoftirqd/322:15:223
30699410,16cyclictest27135-21taskset00:27:412
30699410,13cyclictest12662-21awk21:30:242
30699410,12cyclictest29731-21ssh23:51:202
30699410,12cyclictest22479-21chrt21:54:482
30699410,12cyclictest1033-21snmpd01:42:412
30699401,20cyclictest4038-21ssh01:15:182
30699390,8cyclictest2264-21unixbench_singl21:10:212
30699390,20cyclictest24655-21awk01:00:292
30699390,18cyclictest25840-21diskmemload02:17:192
30699390,14cyclictest15062-21ssh00:50:212
30699390,12cyclictest0-21swapper/223:15:072
315022383,20sleep030543-21/usr/sbin/munin21:05:190
30699380,12cyclictest103302sleep221:25:462
30699380,11cyclictest25840-21diskmemload01:59:592
30699370,13cyclictest16160-21cat23:35:152
30699360,13cyclictest1644-21vmstat01:50:222
30699360,13cyclictest1644-21vmstat01:50:222
30699360,12cyclictest20450-21hddtemp_smartct23:40:142
30699360,11cyclictest10301-21sh00:08:042
30699350,17cyclictest29959-21taskset23:13:582
30699350,12cyclictest14945-21chrt02:07:022
30699350,10cyclictest0-21swapper/200:32:412
30699340,19cyclictest20584-21ssh00:56:292
30199341,10cyclictest17596-21http00:54:561
30699330,20cyclictest12692-21irqrtprio22:40:132
30699330,19cyclictest658-21irqstats02:30:132
30699330,12cyclictest0-21swapper/221:40:162
30699320,19cyclictest7675-21hddtemp_smartct22:30:132
30699320,18cyclictest25840-21diskmemload01:12:002
30699310,19cyclictest849-21irqrtprio22:15:122
30199312,3cyclictest121rcu_preempt01:49:571
258132310,2sleep00-21swapper/000:25:140
32764993021,6cyclictest13646-21fschecks_time00:50:070
30699300,8cyclictest15180-21taskset21:37:222
30699300,11cyclictest0-21swapper/223:01:222
32764992922,4cyclictest9-21ksoftirqd/001:53:270
32764992922,4cyclictest9-21ksoftirqd/001:53:270
30699290,19cyclictest31430-21awk22:10:182
30699290,10cyclictest23771-21memory21:55:102
270142290,2sleep00-21swapper/001:04:530
100972290,2sleep10-21swapper/121:25:211
30699280,12cyclictest0-21swapper/222:05:172
30699280,11cyclictest0-21swapper/222:45:082
301992821,4cyclictest17046-21snmp_o5powerctl21:40:161
32764992721,3cyclictest9-21ksoftirqd/001:42:470
32764992721,3cyclictest9-21ksoftirqd/001:12:100
32764992720,4cyclictest9-21ksoftirqd/021:43:140
30699270,17cyclictest20715-21users22:55:232
301992721,3cyclictest25-21ksoftirqd/123:50:211
301992721,3cyclictest25-21ksoftirqd/102:26:491
301992720,4cyclictest25-21ksoftirqd/102:09:051
32764992621,3cyclictest9-21ksoftirqd/001:20:090
32764992620,3cyclictest9-21ksoftirqd/002:20:370
32764992620,3cyclictest9-21ksoftirqd/002:00:150
30699260,12cyclictest17203-21if_enp1s022:50:142
30699260,12cyclictest0-21swapper/223:05:182
30699260,10cyclictest11240-21awk22:35:232
301992620,3cyclictest25-21ksoftirqd/122:40:381
301992620,3cyclictest25-21ksoftirqd/122:10:131
301992620,3cyclictest25-21ksoftirqd/121:16:421
301992620,3cyclictest25-21ksoftirqd/100:28:081
301992619,4cyclictest25-21ksoftirqd/121:50:061
301992619,4cyclictest15920-21tail22:45:231
20822261,21sleep02092-21ssh23:56:400
32764992518,4cyclictest9-21ksoftirqd/021:26:040
32764992518,4cyclictest9-21ksoftirqd/001:30:280
32764992518,4cyclictest4957-21ssh00:00:190
318482250,5sleep1241ktimersoftd/123:54:561
306992519,3cyclictest33-21ksoftirqd/221:23:032
30699250,12cyclictest6760-21chrt22:28:212
30699250,12cyclictest6760-21chrt22:28:212
301992519,3cyclictest25-21ksoftirqd/121:57:281
301992519,3cyclictest25-21ksoftirqd/121:37:321
301992519,3cyclictest25-21ksoftirqd/102:15:481
301992519,3cyclictest25-21ksoftirqd/101:43:561
301992519,3cyclictest25-21ksoftirqd/101:27:141
301992518,4cyclictest18720-21sed21:45:111
32764992418,3cyclictest9-21ksoftirqd/022:56:520
32764992418,3cyclictest9-21ksoftirqd/022:07:590
32764992417,4cyclictest9-21ksoftirqd/022:52:400
32764992417,4cyclictest9-21ksoftirqd/021:13:510
32764992417,4cyclictest9-21ksoftirqd/000:32:490
301992418,4cyclictest25-21ksoftirqd/102:24:271
301992418,3cyclictest25-21ksoftirqd/122:19:281
301992418,3cyclictest25-21ksoftirqd/101:21:321
301992418,3cyclictest25-21ksoftirqd/100:21:171
301992417,4cyclictest4740-21/usr/sbin/munin00:00:171
301992417,4cyclictest25-21ksoftirqd/123:02:121
301992416,5cyclictest25-21ksoftirqd/121:34:511
301992414,7cyclictest5598-21sh01:17:231
32764992317,3cyclictest9-21ksoftirqd/022:18:550
32764992317,3cyclictest9-21ksoftirqd/021:34:020
32764992316,4cyclictest9-21ksoftirqd/022:36:050
32764992316,4cyclictest9-21ksoftirqd/022:12:570
32764992316,4cyclictest9-21ksoftirqd/021:17:250
32764992316,4cyclictest4533-21grep01:15:250
32764992312,8cyclictest9-21ksoftirqd/023:24:300
301992318,3cyclictest25-21ksoftirqd/121:23:151
301992317,3cyclictest25-21ksoftirqd/122:36:161
301992317,3cyclictest25-21ksoftirqd/122:33:531
301992317,3cyclictest25-21ksoftirqd/121:10:491
301992317,3cyclictest25-21ksoftirqd/102:36:321
301992316,4cyclictest25-21ksoftirqd/123:09:511
301992316,4cyclictest25-21ksoftirqd/122:53:291
301992315,5cyclictest47450irq/100-ens3-Tx23:10:101
30199231,20cyclictest0-21swapper/100:46:211
32764992217,3cyclictest9-21ksoftirqd/002:34:430
32764992216,3cyclictest9-21ksoftirqd/023:07:250
32764992216,3cyclictest9-21ksoftirqd/021:52:160
32764992215,4cyclictest9-21ksoftirqd/022:24:050
32764992215,4cyclictest9-21ksoftirqd/022:24:050
32764992215,4cyclictest29494-1kworker/u9:101:09:560
301992217,3cyclictest25-21ksoftirqd/123:40:221
301992217,3cyclictest1907-21ssh01:14:331
301992216,4cyclictest25-21ksoftirqd/122:26:071
301992216,4cyclictest25-21ksoftirqd/122:26:071
30199221,3cyclictest0-21swapper/123:25:291
3276499214,8cyclictest0-21swapper/000:20:110
3276499213,11cyclictest12074-21ssh01:25:000
32764992116,3cyclictest9-21ksoftirqd/023:00:060
32764992116,3cyclictest9-21ksoftirqd/002:25:150
32764992116,3cyclictest9-21ksoftirqd/000:15:100
32764992114,4cyclictest23856-21strings21:55:110
32764992114,4cyclictest2214-21sh23:19:080
32764992114,4cyclictest21815-21grep02:15:150
32764992112,6cyclictest19243-21missed_timers00:55:150
32764992112,4cyclictest10514-21cpu23:30:060
3276499210,9cyclictest258402diskmemload00:05:160
301992117,2cyclictest0-21swapper/122:01:091
301992115,4cyclictest25-21ksoftirqd/122:58:151
301992115,4cyclictest25-21ksoftirqd/101:05:461
301992115,4cyclictest15657-21cut01:30:161
301992115,3cyclictest9509-21cpuspeed_turbos02:00:131
301992115,3cyclictest19298-21mandb02:11:341
301992114,4cyclictest7113-21mailstats00:40:201
301992113,4cyclictest15039-21acpi23:35:061
30199211,17cyclictest31231-21grep23:15:091
30199211,17cyclictest190-21jbd2/sda1-823:46:331
3276499208,3cyclictest121rcu_preempt23:35:110
32764992016,3cyclictest9-21ksoftirqd/023:09:590
32764992014,3cyclictest9-21ksoftirqd/022:32:350
32764992014,3cyclictest4617-21/usr/sbin/munin01:55:210
32764992014,3cyclictest29446-21sh01:45:260
32764992014,3cyclictest21319-21sendmail_mailst01:35:200
32764992014,3cyclictest16778-21/usr/sbin/munin02:10:220
32764992013,4cyclictest9938-21ssh00:44:490
32764992013,4cyclictest8395-21grep21:24:510
32764992013,4cyclictest26724-21sh23:49:220
32764992012,4cyclictest1505-21runrttasks23:28:200
32764992011,5cyclictest4530-21gltestperf02:35:170
3276499201,15cyclictest25840-21diskmemload23:51:330
30199206,6cyclictest913-21phc2sys02:30:151
30199202,10cyclictest19971-21ssh01:35:091
301992014,3cyclictest32228-21sh00:33:261
301992014,3cyclictest14292-21sh00:11:331
301992014,3cyclictest14292-21sh00:11:331
301992014,3cyclictest1278-21expr00:35:091
301992013,4cyclictest3768-21run-parts22:20:151
301992013,4cyclictest3768-21run-parts22:20:151
301992013,4cyclictest26094-21sh01:03:081
301992013,4cyclictest18856-21sh00:55:131
301992013,4cyclictest16642-21expr00:15:101
301992013,4cyclictest10745-21sh00:08:561
30199201,17cyclictest0-21swapper/101:51:581
30199201,17cyclictest0-21swapper/101:51:581
32764991914,3cyclictest19859-21/usr/sbin/munin23:40:180
32764991913,3cyclictest9-21ksoftirqd/022:49:470
32764991913,3cyclictest25434-21sh22:00:060
32764991913,3cyclictest18322-21df_inode21:45:070
32764991913,3cyclictest11863-21/usr/sbin/munin22:40:120
3276499191,2cyclictest0-21swapper/000:49:010
32764991912,4cyclictest25840-21diskmemload02:07:140
32764991912,4cyclictest14151-21cut21:35:110
32764991912,4cyclictest12619-21cut00:10:090
32764991912,4cyclictest12619-21cut00:10:090
301991913,3cyclictest28314-21cut22:05:121
30199191,16cyclictest0-21swapper/123:30:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional