You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-02 - 06:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Wed Apr 02, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1491821390,4sleep10-21swapper/100:25:551
1491821390,4sleep10-21swapper/100:25:551
2729321300,3sleep10-21swapper/100:00:271
2729321300,3sleep10-21swapper/100:00:271
508521240,2chrt0-21swapper/322:55:453
247902790,3chrt0-21swapper/122:00:571
247902790,3chrt0-21swapper/122:00:571
78032720,4sleep10-21swapper/121:41:031
78032720,4sleep10-21swapper/121:41:031
87662660,6sleep227-21ksoftirqd/223:37:372
167892650,2sleep20-21swapper/200:29:132
167892650,2sleep20-21swapper/200:29:132
527099641,51cyclictest17969-21ssh00:30:490
527099641,51cyclictest17969-21ssh00:30:490
173172610,2chrt16161-21/usr/sbin/munin22:30:562
173172610,2chrt16161-21/usr/sbin/munin22:30:562
27592590,3chrt2270-21/usr/sbin/munin21:35:542
27592590,3chrt2270-21/usr/sbin/munin21:35:542
151452570,4sleep114677-21/usr/sbin/munin21:50:481
151452570,4sleep114677-21/usr/sbin/munin21:50:481
220662550,1sleep20-21swapper/221:20:302
32152540,4sleep227-21ksoftirqd/223:31:012
32152540,4sleep227-21ksoftirqd/223:31:012
26412530,2chrt0-21swapper/123:30:561
26412530,2chrt0-21swapper/123:30:561
206652530,2sleep00-21swapper/023:14:150
206652530,2sleep00-21swapper/023:14:150
162522520,2sleep216262-21ssh23:09:162
162522520,2sleep216262-21ssh23:09:162
53362510,4chrt0-21swapper/121:38:461
53362510,4chrt0-21swapper/121:38:461
38342510,2sleep03835-21diskmemload23:31:570
38342510,2sleep03835-21diskmemload23:31:570
17572510,3sleep10-21swapper/121:35:171
17572510,3sleep10-21swapper/121:35:171
81172500,3sleep00-21swapper/023:36:380
11912490,2sleep20-21swapper/222:50:502
11912490,2sleep20-21swapper/222:50:502
136082480,10sleep333-21ksoftirqd/323:05:223
136082480,10sleep333-21ksoftirqd/323:05:223
89392470,4sleep10-21swapper/123:37:521
149902470,2sleep00-21swapper/023:45:220
527099466,25cyclictest38650irq/55-eth019:50:550
527099466,25cyclictest38650irq/55-eth019:50:550
317602460,2sleep30-21swapper/300:05:433
317602460,2sleep30-21swapper/300:05:433
254182460,5sleep121-21ksoftirqd/123:19:351
192832460,3sleep10-21swapper/121:16:071
158352450,2sleep00-21swapper/021:12:490
158352450,2sleep00-21swapper/021:12:490
5272994414,3cyclictest131rcu_preempt21:20:502
5272994414,3cyclictest131rcu_preempt21:20:502
197342440,3sleep10-21swapper/122:34:291
197342440,3sleep10-21swapper/122:34:291
527099421,11cyclictest0-21swapper/020:20:480
527099421,11cyclictest0-21swapper/020:20:480
246722420,5sleep20-21swapper/223:18:272
75792410,4sleep00-21swapper/020:55:370
75792410,4sleep00-21swapper/020:55:370
527199419,6cyclictest131rcu_preempt19:20:481
527199419,6cyclictest131rcu_preempt19:20:481
78302400,3sleep17839-21sh22:20:151
218992400,2sleep00-21swapper/023:53:540
218992400,2sleep00-21swapper/023:53:540
527299391,3cyclictest131rcu_preempt00:00:482
527199394,31cyclictest22487-21ssh21:20:491
527199394,31cyclictest22487-21ssh21:20:491
527099396,23cyclictest38650irq/55-eth022:35:470
527099392,25cyclictest38650irq/55-eth022:15:470
527099390,30cyclictest6989-21runrttasks21:40:530
527099390,30cyclictest6989-21runrttasks21:40:530
527299375,23cyclictest8678-20Xorg19:15:482
527299375,23cyclictest8678-20Xorg19:15:482
527299375,23cyclictest8678-20Xorg19:15:482
527299373,28cyclictest8678-20Xorg21:10:512
527299373,28cyclictest8678-20Xorg21:10:512
527199374,29cyclictest30918-21apt-get23:25:461
527199371,31cyclictest14748-21taskset21:50:441
527299362,23cyclictest8678-20Xorg19:55:482
527299362,23cyclictest8678-20Xorg19:55:482
527299360,2cyclictest0-21swapper/220:10:472
326712360,2sleep232682-21ssh23:28:332
326712360,2sleep232682-21ssh23:28:332
527099353,24cyclictest17311-21cat23:10:430
527099353,24cyclictest17311-21cat23:10:430
527099352,6cyclictest12-21ksoftirqd/021:15:480
251012350,4sleep30-21swapper/322:00:593
251012350,4sleep30-21swapper/322:00:593
56572340,5sleep121-21ksoftirqd/100:14:331
56572340,5sleep121-21ksoftirqd/100:14:331
281532340,3sleep20-21swapper/222:05:362
281532340,3sleep20-21swapper/222:05:362
257032340,2sleep10-21swapper/120:10:581
527299331,25cyclictest8678-20Xorg22:20:482
527299331,25cyclictest8678-20Xorg22:20:482
527199339,12cyclictest21-21ksoftirqd/120:20:481
527199339,12cyclictest21-21ksoftirqd/120:20:481
292792330,2sleep10-21swapper/121:29:341
292792330,2sleep10-21swapper/121:29:341
54042320,2sleep30-21swapper/321:38:523
54042320,2sleep30-21swapper/321:38:523
527299321,22cyclictest8678-20Xorg19:50:542
527299321,22cyclictest8678-20Xorg19:50:542
527099322,23cyclictest5154-21ssh22:55:490
527099322,23cyclictest5154-21ssh22:55:490
527099321,3cyclictest131rcu_preempt20:45:480
527099321,3cyclictest131rcu_preempt20:45:480
46392325,19sleep24680-21sshd19:10:582
46392325,19sleep24680-21sshd19:10:582
31832320,2sleep03190-21ssh22:15:000
31832320,2sleep03190-21ssh22:15:000
252682320,2sleep00-21swapper/023:57:080
252682320,2sleep00-21swapper/023:57:080
527299312,22cyclictest8678-20Xorg20:50:542
527299312,22cyclictest8678-20Xorg20:50:542
527199313,22cyclictest21983-21runrttasks19:40:491
527199313,22cyclictest21983-21runrttasks19:40:491
5270993114,12cyclictest38650irq/55-eth000:00:500
5270993114,12cyclictest38650irq/55-eth000:00:500
295742310,2sleep30-21swapper/322:07:343
295742310,2sleep30-21swapper/322:07:343
24042310,2sleep30-21swapper/300:10:463
231042310,1sleep223115-21ssh22:37:412
215162310,2sleep221526-21ssh23:53:172
215162310,2sleep221526-21ssh23:53:172
142432310,2sleep28678-20Xorg00:25:042
142432310,2sleep28678-20Xorg00:25:042
142432310,2sleep28678-20Xorg00:25:042
100732310,10sleep012-21ksoftirqd/023:00:530
100732310,10sleep012-21ksoftirqd/023:00:530
5270993013,12cyclictest38650irq/55-eth022:20:490
5270993013,12cyclictest38650irq/55-eth022:20:490
206132300,3sleep10-21swapper/119:55:501
206132300,3sleep10-21swapper/119:55:501
146042300,2sleep00-21swapper/021:50:390
138882300,4sleep30-21swapper/323:43:363
527299292,19cyclictest27741-21ssh00:00:492
527299292,19cyclictest27741-21ssh00:00:492
527299291,20cyclictest8678-20Xorg20:35:472
527299291,20cyclictest8678-20Xorg20:35:472
527199295,22cyclictest31070-21grep20:30:431
527199295,22cyclictest31070-21grep20:30:431
527199295,20cyclictest31832-21ssh22:10:451
527199295,20cyclictest31832-21ssh22:10:451
320952290,2sleep232106-21ssh22:49:302
320952290,2sleep232106-21ssh22:49:302
217072290,2sleep121717-21ssh23:53:361
217072290,2sleep121717-21ssh23:53:361
169312290,2sleep216941-21ssh21:52:192
169312290,2sleep216941-21ssh21:52:192
5272992811,4cyclictest131rcu_preempt21:00:482
5272992811,4cyclictest131rcu_preempt21:00:482
527199288,4cyclictest131rcu_preempt19:25:491
527199283,8cyclictest21-21ksoftirqd/121:00:511
527199283,8cyclictest21-21ksoftirqd/121:00:511
5271992819,6cyclictest16214-21cut22:30:441
5271992819,6cyclictest16214-21cut22:30:441
5271992810,3cyclictest131rcu_preempt19:45:491
5271992810,3cyclictest131rcu_preempt19:45:491
527299277,5cyclictest27-21ksoftirqd/219:20:482
527299277,5cyclictest27-21ksoftirqd/219:20:482
5272992719,6cyclictest0-21swapper/221:40:482
5272992719,6cyclictest0-21swapper/221:40:482
527199278,7cyclictest21-21ksoftirqd/123:08:571
527199278,7cyclictest21-21ksoftirqd/123:08:571
527199271,23cyclictest588-21ssh22:50:441
527199271,23cyclictest588-21ssh22:50:441
527199270,25cyclictest6582-21ssh00:15:501
527099278,6cyclictest12-21ksoftirqd/019:26:020
5270992716,9cyclictest0-21swapper/020:20:410
5270992716,9cyclictest0-21swapper/020:20:410
527399263,18cyclictest0-21swapper/322:50:443
527399263,18cyclictest0-21swapper/322:50:443
527299261,18cyclictest30920-21ssh23:25:462
527199269,9cyclictest21-21ksoftirqd/121:13:441
527199269,9cyclictest21-21ksoftirqd/121:13:441
527199268,7cyclictest131rcu_preempt22:06:271
527199268,7cyclictest131rcu_preempt22:06:271
527199263,6cyclictest21-21ksoftirqd/123:50:441
527199263,6cyclictest21-21ksoftirqd/123:50:441
5271992619,4cyclictest0-21swapper/123:00:481
5271992619,4cyclictest0-21swapper/123:00:481
5270992614,10cyclictest11549-21df21:05:500
5270992614,10cyclictest11549-21df21:05:500
225992260,3sleep30-21swapper/321:58:583
225992260,3sleep30-21swapper/321:58:583
225992260,3sleep30-21swapper/321:58:583
100792260,4sleep3321rcuc/323:39:433
527399253,13cyclictest0-21swapper/323:10:443
527399253,13cyclictest0-21swapper/323:10:443
527399251,19cyclictest0-21swapper/321:55:453
527199256,11cyclictest21-21ksoftirqd/120:47:531
527199256,11cyclictest21-21ksoftirqd/120:47:531
527199254,5cyclictest21-21ksoftirqd/100:09:171
527199253,18cyclictest13040-21diskmemload00:30:491
527199253,18cyclictest13040-21diskmemload00:30:491
527199252,19cyclictest1852-21apt-get23:30:451
5271992512,6cyclictest21-21ksoftirqd/122:25:451
527199251,19cyclictest21983-21runrttasks00:20:491
527199251,19cyclictest21983-21runrttasks00:20:491
527199251,19cyclictest21983-21runrttasks00:20:491
527099252,14cyclictest19374-21cat21:55:450
40202256,11sleep10-21swapper/119:10:451
40202256,11sleep10-21swapper/119:10:451
316462250,3sleep10-21swapper/120:30:501
316462250,3sleep10-21swapper/120:30:501
527399243,13cyclictest0-21swapper/321:40:533
527399243,13cyclictest0-21swapper/321:40:533
5272992416,3cyclictest647-21rm00:08:022
527199242,5cyclictest21-21ksoftirqd/122:35:531
5271992417,5cyclictest8678-20Xorg20:00:491
5271992417,5cyclictest8678-20Xorg20:00:491
5271992417,5cyclictest21-21ksoftirqd/100:01:021
5271992417,5cyclictest21-21ksoftirqd/100:01:021
5271992411,6cyclictest21-21ksoftirqd/123:27:001
527099247,13cyclictest25287-21ssh22:40:490
527099247,13cyclictest25287-21ssh22:40:490
527099245,6cyclictest23157-21vmstat23:16:030
527099241,8cyclictest0-21swapper/020:10:470
527099241,19cyclictest2321-21ssh00:10:450
527099240,18cyclictest17710-21seq21:53:320
130842240,6sleep1211ksoftirqd/121:10:371
130842240,6sleep1211ksoftirqd/121:10:371
81912230,2sleep30-21swapper/322:58:513
81912230,2sleep30-21swapper/322:58:513
78422230,3chrt0-21swapper/222:20:152
527199239,6cyclictest131rcu_preempt21:55:571
527199239,6cyclictest131rcu_preempt21:55:571
527199239,6cyclictest131rcu_preempt21:55:571
527199237,12cyclictest131rcu_preempt20:55:471
527199237,12cyclictest131rcu_preempt20:55:471
527199235,3cyclictest131rcu_preempt19:35:511
527199235,3cyclictest131rcu_preempt19:35:511
527199234,8cyclictest21-21ksoftirqd/120:52:291
527199234,8cyclictest21-21ksoftirqd/120:52:291
527199234,6cyclictest21-21ksoftirqd/100:35:581
527199234,6cyclictest21-21ksoftirqd/100:35:581
527199231,18cyclictest591-21ls20:35:491
527199231,18cyclictest591-21ls20:35:491
5271992311,6cyclictest21-21ksoftirqd/120:07:321
5271992311,6cyclictest21-21ksoftirqd/120:07:321
5271992310,8cyclictest21-21ksoftirqd/120:15:561
5271992310,8cyclictest21-21ksoftirqd/120:15:561
527099235,13cyclictest28658-21kworker/0:223:25:460
527099234,16cyclictest30050irq/49-vc409:15:180
527099234,16cyclictest30050irq/49-vc409:15:180
527099231,4cyclictest131rcu_preempt21:25:440
527099231,4cyclictest131rcu_preempt21:25:440
5270992311,9cyclictest16924-21ntp_states22:30:530
5270992311,9cyclictest16924-21ntp_states22:30:530
527099231,18cyclictest10718-21taskset00:20:490
527099231,18cyclictest10718-21taskset00:20:490
527099231,18cyclictest10718-21taskset00:20:490
527099231,13cyclictest38650irq/55-eth021:55:500
527099231,13cyclictest38650irq/55-eth021:55:500
527099231,13cyclictest38650irq/55-eth021:55:500
5270992311,10cyclictest12903-21ssh22:25:470
5270992311,10cyclictest12903-21ssh22:25:470
200912230,2sleep10-21swapper/123:13:201
200912230,2sleep10-21swapper/123:13:201
527299228,5cyclictest131rcu_preempt22:29:412
527299228,5cyclictest131rcu_preempt22:29:412
527299226,8cyclictest131rcu_preempt00:36:032
527299226,8cyclictest131rcu_preempt00:36:032
5272992216,4cyclictest0-21swapper/220:20:432
5272992216,4cyclictest0-21swapper/220:20:432
5272992216,3cyclictest0-21swapper/219:30:512
5272992216,3cyclictest0-21swapper/219:30:512
527199229,3cyclictest131rcu_preempt19:15:591
527199229,3cyclictest131rcu_preempt19:15:591
527199229,3cyclictest131rcu_preempt19:15:591
527199228,3cyclictest131rcu_preempt22:55:481
527199228,3cyclictest131rcu_preempt22:55:481
527199227,9cyclictest21-21ksoftirqd/123:41:021
527199226,3cyclictest131rcu_preempt22:40:531
527199226,3cyclictest131rcu_preempt22:40:531
527199222,6cyclictest21-21ksoftirqd/122:20:511
527099228,8cyclictest12-21ksoftirqd/023:49:370
527099228,8cyclictest12-21ksoftirqd/023:49:370
527099221,4cyclictest131rcu_preempt21:36:010
527099221,4cyclictest131rcu_preempt21:36:010
5270992213,2cyclictest12-21ksoftirqd/020:55:470
5270992213,2cyclictest12-21ksoftirqd/020:55:470
5270992212,5cyclictest12-21ksoftirqd/019:50:420
5270992212,5cyclictest12-21ksoftirqd/019:50:420
527099221,15cyclictest13836-21ntp_states19:35:560
527099221,15cyclictest13836-21ntp_states19:35:560
5270992211,3cyclictest13040-21diskmemload00:37:260
5270992211,3cyclictest13040-21diskmemload00:37:260
40482222,15sleep30-21swapper/319:10:513
40482222,15sleep30-21swapper/319:10:513
527299219,5cyclictest131rcu_preempt21:55:502
527299219,5cyclictest131rcu_preempt21:55:502
527299219,5cyclictest131rcu_preempt21:55:502
527299219,4cyclictest131rcu_preempt23:59:282
527299219,3cyclictest27-21ksoftirqd/223:40:472
527299218,5cyclictest27-21ksoftirqd/222:59:232
527299218,5cyclictest27-21ksoftirqd/222:59:232
527299215,4cyclictest0-21swapper/219:45:492
527299215,4cyclictest0-21swapper/219:45:492
527299211,9cyclictest8931-21unixbench_multi20:56:012
527299211,9cyclictest8931-21unixbench_multi20:56:012
527299211,3cyclictest0-21swapper/221:06:342
527299211,3cyclictest0-21swapper/221:06:342
527299211,16cyclictest15059-21seq19:40:492
527299211,16cyclictest15059-21seq19:40:492
527299210,15cyclictest29902-21latency_hist21:30:412
527299210,15cyclictest29902-21latency_hist21:30:412
527199218,3cyclictest131rcu_preempt22:51:451
527199218,3cyclictest131rcu_preempt22:51:451
5271992111,5cyclictest21-21ksoftirqd/119:30:581
5271992111,5cyclictest21-21ksoftirqd/119:30:581
5271992111,4cyclictest21-21ksoftirqd/119:50:451
5271992111,4cyclictest21-21ksoftirqd/119:50:451
527099219,4cyclictest0-21swapper/022:01:420
527099219,4cyclictest0-21swapper/022:01:420
527099219,3cyclictest131rcu_preempt20:40:500
527099219,3cyclictest131rcu_preempt20:40:500
527099219,3cyclictest131rcu_preempt20:40:500
527099219,10cyclictest31836-21ssh21:31:290
527099219,10cyclictest31836-21ssh21:31:290
527099218,10cyclictest5790-21ssh00:14:470
527099218,10cyclictest5790-21ssh00:14:470
527099217,9cyclictest12-21ksoftirqd/022:54:580
527099217,9cyclictest12-21ksoftirqd/022:54:580
527099211,15cyclictest31412-21ls20:30:480
527099211,15cyclictest31412-21ls20:30:480
527099211,14cyclictest5987-21df_inode19:15:500
527099211,14cyclictest5987-21df_inode19:15:500
527099211,14cyclictest5987-21df_inode19:15:500
527099211,13cyclictest131rcu_preempt19:30:580
527099211,13cyclictest131rcu_preempt19:30:580
527099211,13cyclictest0-21swapper/023:26:100
527099211,13cyclictest0-21swapper/023:26:100
527099211,10cyclictest15037-21/usr/sbin/munin19:40:580
527099211,10cyclictest15037-21/usr/sbin/munin19:40:580
527099211,10cyclictest0-21swapper/021:26:140
527099211,10cyclictest0-21swapper/021:26:140
5270992110,6cyclictest12-21ksoftirqd/021:04:030
5270992110,6cyclictest12-21ksoftirqd/021:04:030
5270992110,6cyclictest12-21ksoftirqd/000:28:040
5270992110,6cyclictest12-21ksoftirqd/000:28:040
527399202,12cyclictest0-21swapper/320:35:493
527399202,12cyclictest0-21swapper/320:35:493
5273992011,7cyclictest0-21swapper/321:11:043
5273992011,7cyclictest0-21swapper/321:11:043
527399201,11cyclictest0-21swapper/322:20:483
527399201,11cyclictest0-21swapper/322:20:483
527299209,4cyclictest131rcu_preempt23:49:052
527299209,4cyclictest131rcu_preempt23:49:052
527299209,4cyclictest131rcu_preempt22:45:322
527299209,4cyclictest131rcu_preempt22:45:322
527299209,4cyclictest131rcu_preempt21:48:342
527299209,4cyclictest131rcu_preempt20:00:592
527299209,4cyclictest131rcu_preempt20:00:592
527299209,4cyclictest131rcu_preempt19:25:582
527299207,7cyclictest27-21ksoftirqd/222:07:342
527299207,7cyclictest27-21ksoftirqd/222:07:342
5272992015,3cyclictest27-21ksoftirqd/220:40:542
5272992015,3cyclictest27-21ksoftirqd/220:40:542
5272992015,3cyclictest27-21ksoftirqd/220:40:542
5272992015,3cyclictest27-21ksoftirqd/200:18:372
5272992014,3cyclictest27-21ksoftirqd/220:21:002
5272992014,3cyclictest27-21ksoftirqd/220:21:002
5272992013,5cyclictest5189-21ssh00:13:452
5272992013,5cyclictest5189-21ssh00:13:452
527299200,16cyclictest4612-21ls20:45:542
527299200,16cyclictest4612-21ls20:45:542
527199204,6cyclictest21-21ksoftirqd/120:45:401
527199204,6cyclictest21-21ksoftirqd/120:45:401
527199204,6cyclictest21-21ksoftirqd/120:45:401
527099208,5cyclictest12-21ksoftirqd/022:10:180
527099208,5cyclictest12-21ksoftirqd/022:10:180
527099206,9cyclictest131rcu_preempt00:20:210
5270992011,8cyclictest0-21swapper/022:47:470
5270992011,8cyclictest0-21swapper/022:47:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional