You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-19 - 11:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sat Apr 19, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
63178399255252,1cyclictest0-21swapper/321:27:093
631779991761,170cyclictest667635-21kworker/2:0+i915-unordered21:49:212
63177999175167,6cyclictest800979-21kworker/u16:3+i915-dp23:55:042
63177999174168,4cyclictest633360-21kworker/u16:4+i915-dp19:53:482
63177999174168,4cyclictest633360-21kworker/u16:4+i915-dp19:53:472
63177999173167,4cyclictest683227-21kworker/u16:3+i915-dp21:14:132
63177999172166,4cyclictest779720-21kworker/u16:4+i915-dp23:02:112
63177999171165,4cyclictest800979-21kworker/u16:3+i915-dp00:03:382
63177999171165,4cyclictest633360-21kworker/u16:4+i915-dp21:56:522
63177999171163,6cyclictest851175-21kworker/u16:1+i915-dp00:25:582
63177999171162,6cyclictest752348-21kworker/u16:3+i915-dp22:32:582
631779991710,167cyclictest829920-21kworker/2:2+i915-unordered00:35:042
631779991710,167cyclictest829920-21kworker/2:2+i915-unordered00:35:032
63177999170164,4cyclictest718069-21kworker/u16:2+i915-dp22:57:092
63177999170164,4cyclictest676397-21kworker/u16:0+i915-dp23:09:042
63178399169166,1cyclictest0-21swapper/322:30:473
63178399169166,1cyclictest0-21swapper/300:32:413
63177999169163,4cyclictest752348-21kworker/u16:3+i915-dp22:29:572
63177999169163,4cyclictest676397-21kworker/u16:0+i915-dp22:24:442
63177999169163,4cyclictest657112-21kworker/u16:2+i915-dp20:14:172
63177999169161,6cyclictest676397-21kworker/u16:0+i915-dp21:30:502
63178399168165,1cyclictest0-21swapper/323:26:113
63177999168163,3cyclictest657112-21kworker/u16:2+i915-dp20:33:152
63177999168163,3cyclictest633360-21kworker/u16:4+i915-dp20:41:082
63177999168162,4cyclictest657112-21kworker/u16:2+i915-dp20:36:472
63177999168159,7cyclictest814502-21kworker/u16:2+i915-dp00:09:002
631779991680,164cyclictest616312-21kworker/2:3+i915-unordered19:38:022
63177999167161,4cyclictest676397-21kworker/u16:0+i915-dp22:03:452
63177999167161,4cyclictest633360-21kworker/u16:4+i915-dp20:06:532
63177999167160,4cyclictest758734-21kworker/u16:4+i915-dp22:40:112
631779991671,163cyclictest646468-21kworker/2:1+i915-unordered20:25:242
631779991670,163cyclictest616312-21kworker/2:3+i915-unordered19:14:332
63178399166163,1cyclictest0-21swapper/323:10:043
63177999166160,4cyclictest814502-21kworker/u16:2+i915-dp23:53:032
63177999166160,4cyclictest633360-21kworker/u16:4+i915-dp22:11:492
63177999166160,4cyclictest633360-21kworker/u16:4+i915-dp21:51:192
63177999166157,7cyclictest718069-21kworker/u16:2+i915-dp23:15:472
631779991661,161cyclictest646468-21kworker/2:1+i915-unordered20:00:422
631779991660,163cyclictest829920-21kworker/2:2+i915-unordered00:31:432
631779991660,163cyclictest829920-21kworker/2:2+i915-unordered00:24:502
63178399165162,1cyclictest0-21swapper/323:33:553
63178399165162,1cyclictest0-21swapper/323:33:553
631779991650,162cyclictest829920-21kworker/2:2+i915-unordered00:13:552
631779991650,162cyclictest667635-21kworker/2:0+i915-unordered21:09:442
631779991650,159cyclictest667635-21kworker/2:0+i915-unordered23:44:122
63178399164161,1cyclictest0-21swapper/320:32:543
63178399164161,1cyclictest0-21swapper/319:49:053
63178399164161,1cyclictest0-21swapper/319:34:193
63177999164158,4cyclictest718069-21kworker/u16:2+i915-dp22:49:052
631779991641,160cyclictest667635-21kworker/2:0+i915-unordered23:37:192
631779991640,161cyclictest667635-21kworker/2:0+i915-unordered23:13:492
631779991640,161cyclictest667635-21kworker/2:0+i915-unordered20:53:572
631779991640,160cyclictest667635-21kworker/2:0+i915-unordered21:27:212
63177999163157,4cyclictest789789-21kworker/u16:1+i915-dp23:46:302
63177999163157,4cyclictest633360-21kworker/u16:4+i915-dp21:16:442
631779991630,160cyclictest667635-21kworker/2:0+i915-unordered22:37:532
631779991630,160cyclictest646468-21kworker/2:1+i915-unordered20:21:422
631779991630,159cyclictest667635-21kworker/2:0+i915-unordered22:06:392
631779991630,159cyclictest667635-21kworker/2:0+i915-unordered21:37:462
631779991621,158cyclictest616312-21kworker/2:3+i915-unordered19:21:562
631779991620,159cyclictest616312-21kworker/2:3+i915-unordered19:34:212
631779991620,159cyclictest616312-21kworker/2:3+i915-unordered19:29:482
631779991620,159cyclictest616312-21kworker/2:3+i915-unordered19:15:322
631779991620,158cyclictest667635-21kworker/2:0+i915-unordered22:18:142
631779991620,157cyclictest667635-21kworker/2:0+i915-unordered23:31:462
631779991620,157cyclictest667635-21kworker/2:0+i915-unordered23:31:462
631779991610,159cyclictest646468-21kworker/2:1+i915-unordered19:45:362
631779991610,158cyclictest667635-21kworker/2:0+i915-unordered22:54:402
631779991610,158cyclictest667635-21kworker/2:0+i915-unordered21:44:192
63178399160157,1cyclictest0-21swapper/322:08:473
631779991600,159cyclictest646468-21kworker/2:1+i915-unordered19:55:512
631779991590,156cyclictest829920-21kworker/2:2+i915-unordered00:19:482
631779991590,156cyclictest667635-21kworker/2:0+i915-unordered21:00:092
631779991590,155cyclictest667635-21kworker/2:0+i915-unordered20:56:082
631779991570,154cyclictest667635-21kworker/2:0+i915-unordered23:23:032
631779991560,152cyclictest646468-21kworker/2:1+i915-unordered20:18:212
631779991551,151cyclictest616312-21kworker/2:3+i915-unordered19:40:342
631779991550,152cyclictest667635-21kworker/2:0+i915-unordered23:25:542
63178399153150,1cyclictest0-21swapper/323:00:403
63178399153150,1cyclictest0-21swapper/319:39:113
63178399151148,1cyclictest0-21swapper/320:28:433
631779991510,147cyclictest667635-21kworker/2:0+i915-unordered21:20:592
63178399149146,1cyclictest0-21swapper/320:49:123
63178399149146,1cyclictest0-21swapper/320:49:123
63178399146143,1cyclictest0-21swapper/300:03:483
63178399144141,1cyclictest0-21swapper/323:49:423
63178399144141,1cyclictest0-21swapper/323:42:593
63178399143140,1cyclictest0-21swapper/319:14:303
63178399143140,1cyclictest0-21swapper/300:29:293
63178399143140,1cyclictest0-21swapper/300:17:343
631779991420,139cyclictest616312-21kworker/2:3+i915-unordered19:10:012
85061321410,137sleep0759807-21kworker/0:0+i915-unordered00:20:480
63178399141138,1cyclictest0-21swapper/300:06:493
63178399138135,1cyclictest0-21swapper/322:35:193
63178399138135,1cyclictest0-21swapper/319:42:533
63178399138133,2cyclictest0-21swapper/323:50:423
63178399134131,1cyclictest0-21swapper/323:00:003
63178399133130,1cyclictest0-21swapper/319:24:553
63178399132129,1cyclictest0-21swapper/321:40:053
63178399132129,1cyclictest0-21swapper/320:12:463
63178399130127,1cyclictest0-21swapper/323:22:003
63178399129126,1cyclictest0-21swapper/320:41:383
86233621280,3sleep1862340-21latency_hist00:40:001
86233621280,3sleep1862340-21latency_hist00:40:001
631492212853,71sleep30-21swapper/319:08:313
63178399126123,1cyclictest0-21swapper/323:57:453
63178399126123,1cyclictest0-21swapper/322:00:243
63178399123120,1cyclictest0-21swapper/320:01:113
63178399121118,1cyclictest0-21swapper/320:24:413
63178399120117,1cyclictest0-21swapper/320:36:263
63178399119115,2cyclictest0-21swapper/321:14:233
63178399115112,1cyclictest0-21swapper/322:22:023
63178399114111,1cyclictest0-21swapper/323:07:033
63178399110107,1cyclictest0-21swapper/322:42:123
63178399107104,1cyclictest0-21swapper/300:35:523
63178399107104,1cyclictest0-21swapper/300:35:523
6317839910198,1cyclictest0-21swapper/323:38:373
6317839910198,1cyclictest0-21swapper/320:06:333
631783999389,2cyclictest0-21swapper/321:36:233
63141728657,6sleep00-21swapper/019:07:440
63120228251,9sleep10-21swapper/119:05:321
631783998177,2cyclictest0-21swapper/321:05:093
631783998076,2cyclictest0-21swapper/321:55:213
631783997875,1cyclictest0-21swapper/319:17:213
63177999780,75cyclictest667635-21kworker/2:0+i915-unordered20:45:332
63177999780,75cyclictest667635-21kworker/2:0+i915-unordered20:45:332
631783997673,1cyclictest0-21swapper/323:18:083
631783997067,1cyclictest0-21swapper/319:58:403
631783996663,1cyclictest0-21swapper/319:51:163
631783996663,1cyclictest0-21swapper/319:51:163
631783995855,1cyclictest0-21swapper/321:19:153
631783994844,2cyclictest0-21swapper/320:18:283
631783994340,1cyclictest0-21swapper/321:24:083
631783992724,1cyclictest0-21swapper/322:49:553
6754772241,12sleep0675515-21ntpq20:50:210
631783992420,2cyclictest0-21swapper/321:05:013
631783992413,7cyclictest640460-21sleep19:30:013
631783992314,4cyclictest7737042sleep322:52:433
7718392201,15sleep0771962-21sort22:50:200
7009992201,8sleep1683062-21diskmemload21:28:541
631783992017,1cyclictest0-21swapper/321:30:033
631783992016,2cyclictest0-21swapper/320:51:153
631783992016,2cyclictest0-21swapper/300:14:563
631783991916,1cyclictest0-21swapper/322:27:383
631783991916,1cyclictest0-21swapper/322:15:233
631783991916,1cyclictest0-21swapper/322:10:203
631783991916,1cyclictest0-21swapper/321:49:013
631783991916,1cyclictest0-21swapper/320:56:183
63178399191,16cyclictest0-21swapper/321:50:163
8048222181,3sleep00-21swapper/023:29:330
7915942181,7sleep1683062-21diskmemload23:13:461
7019902181,3sleep00-21swapper/021:30:060
6876512181,11sleep00-21swapper/021:14:020
8287052171,9sleep1828756-21ntpq23:55:191
7870822171,2sleep00-21swapper/023:08:560
7386182171,2sleep00-21swapper/022:11:550
7188152170,11sleep00-21swapper/021:49:540
7071742171,4sleep0707320-21ssh21:35:190
63178399170,13cyclictest0-21swapper/300:21:533
8184342161,9sleep1683062-21diskmemload23:43:251
7685352161,2sleep00-21swapper/022:46:580
7612872161,2sleep00-21swapper/022:38:370
7296062161,3sleep0683062-21diskmemload22:00:290
8375002150,2sleep00-21swapper/000:05:250
8333412151,7sleep0833456-21ntpq00:00:200
8266542150,3sleep0826638-21fwupd23:53:380
8091492151,3sleep1683062-21diskmemload23:33:591
8091492151,3sleep1683062-21diskmemload23:33:581
7641512151,2sleep00-21swapper/022:40:420
7034792150,1sleep10-21swapper/121:30:221
64889621510,2sleep1633820-21kworker/1:3+i915-unordered19:49:261
8621772141,8sleep0862335-21wc00:40:000
8621772141,8sleep0862335-21wc00:40:000
8484872141,9sleep1848495-21rm00:19:491
8314422141,4sleep0831452-21ssh23:58:390
7792462141,6sleep00-21swapper/022:59:060
7484892140,2sleep00-21swapper/022:23:060
7319432141,9sleep1731953-21ssh22:03:461
7261822142,5sleep0633360-21kworker/u16:4+i915-dp21:56:520
7093202141,8sleep1709326-21ssh21:37:201
8546082131,3sleep0854618-21ssh00:29:140
8218112131,6sleep00-21swapper/023:48:250
8145582131,8sleep1814569-21ssh23:39:441
7942152131,5sleep00-21swapper/023:15:290
7434882131,6sleep00-21swapper/022:17:240
7192072130,2sleep10-21swapper/121:50:071
7016482131,8sleep0701658-21ssh21:29:580
6817212131,8sleep1681261-21/usr/sbin/munin21:05:171
8402772121,5sleep1683062-21diskmemload00:10:031
8346892125,4sleep1789789-21kworker/u16:1+i915-dp00:01:171
8004752121,6sleep0683062-21diskmemload23:22:510
7814002121,7sleep1781666-21unixbench_singl23:00:271
7802532121,6sleep0780277-21apt-get23:00:140
6608182125,4sleep1633360-21kworker/u16:4+i915-dp20:15:271
8569602111,5sleep1683062-21diskmemload00:30:551
8473482110,1sleep00-21swapper/000:17:470
8368122110,6sleep1836760-21apt-get00:05:021
8051202111,3sleep0805377-21ssh23:30:010
8051202111,3sleep0805377-21ssh23:30:010
7928932112,4sleep10-21swapper/123:15:141
7699062111,4sleep1683062-21diskmemload22:49:131
7640762111,4sleep1683062-21diskmemload22:40:351
7551912111,3sleep00-21swapper/022:31:160
6919162111,5sleep00-21swapper/021:18:030
6896292114,4sleep1683227-21kworker/u16:3+i915-dp21:15:231
6835722111,7sleep10-21swapper/121:10:021
6678232114,4sleep1657112-21kworker/u16:2+i915-dp20:32:341
8547172101,4sleep1683062-21diskmemload00:29:271
8518592101,3sleep1683062-21diskmemload00:23:181
8203002101,4sleep1683062-21diskmemload23:46:011
8026872101,2sleep10-21swapper/123:26:011
8003392100,5sleep1683062-21diskmemload23:22:401
7881022101,4sleep0788251-21df23:10:080
7872262101,2sleep10-21swapper/123:09:091
7782382101,2sleep10-21swapper/122:57:261
7471852101,3sleep1683062-21diskmemload22:20:581
7334522101,3sleep00-21swapper/022:05:540
7226662101,3sleep0683062-21diskmemload21:53:390
7167182101,3sleep1683062-21diskmemload21:46:261
7140362101,3sleep00-21swapper/021:42:290
7132672101,4sleep1683062-21diskmemload21:41:131
6966242100,2sleep00-21swapper/021:22:360
6936262101,3sleep1683062-21diskmemload21:20:081
6755232101,4sleep1675572-21ntpq20:50:221
6615232104,3sleep0648688-21kworker/0:2+i915-unordered20:19:190
6591532105,3sleep0633360-21kworker/u16:4+i915-dp20:11:050
6498912101,4sleep0649944-21ntpq19:50:210
6498912101,4sleep0649944-21ntpq19:50:210
6393462101,5sleep0639394-21ntpq19:25:200
857178290,2sleep00-21swapper/000:31:220
843517290,3sleep00-21swapper/000:13:270
825084291,3sleep1683062-21diskmemload23:51:101
813833291,2sleep00-21swapper/023:38:350
774840291,3sleep1683062-21diskmemload22:54:341
757940291,4sleep1758093-21df22:35:131
751823291,3sleep1683062-21diskmemload22:28:161
750360291,2sleep00-21swapper/022:25:460
744928291,2sleep10-21swapper/122:19:501
734879291,3sleep1683062-21diskmemload22:08:151
724619291,4sleep1724671-21ntpq21:55:161
678926291,4sleep0678943-21apt21:00:100
672910291,4sleep1673025-21memory20:45:171
672910291,4sleep1673025-21memory20:45:171
665397290,1sleep00-21swapper/020:26:360
662633291,4sleep0662688-21/usr/sbin/munin20:20:190
654023291,5sleep1654074-21ntpq20:00:201
644857291,4sleep0645007-21df_inode19:40:150
638637291,4sleep1638654-21apt19:25:131
817337281,4sleep0817351-21ssh23:41:390
740387280,2sleep10-21swapper/122:14:531
657744281,2sleep10-21swapper/120:10:141
649209283,3sleep1648852-21kworker/u16:2+i915-dp19:50:061
649209283,3sleep1648852-21kworker/u16:2+i915-dp19:50:061
643413281,3sleep1643461-21ntpq19:35:191
755383270,2sleep1683062-21diskmemload22:31:331
681260270,1sleep00-21swapper/021:05:120
678467270,1sleep00-21swapper/020:58:150
678437271,4sleep1633360-21kworker/u16:4+i915-dp20:57:561
668990271,3sleep1669047-21ntpq20:35:191
667257270,1sleep00-21swapper/020:30:250
661892270,3sleep10-21swapper/120:20:121
655416270,2sleep00-21swapper/020:05:160
653893270,4sleep00-21swapper/020:00:190
644287270,1sleep00-21swapper/019:37:220
640865271,3sleep0640888-21/usr/sbin/munin19:30:170
678944261,2sleep10-21swapper/121:00:121
670103260,4sleep00-21swapper/020:39:510
663958261,2sleep10-21swapper/120:25:131
656006261,2sleep10-21swapper/120:05:211
651260260,4sleep00-21swapper/019:55:020
648535260,1sleep00-21swapper/019:45:400
638235262,1sleep10-21swapper/119:23:361
638203260,4sleep00-21swapper/019:23:150
635972261,3sleep1635973-21spawn19:16:431
635195261,2sleep00-21swapper/019:15:220
633713260,1sleep10-21swapper/119:14:501
673198251,1sleep00-21swapper/020:45:200
673198251,1sleep00-21swapper/020:45:200
671806251,1sleep00-21swapper/020:40:590
651960250,1sleep10-21swapper/119:55:191
645522250,1sleep10-21swapper/119:40:201
631941250,1sleep00-21swapper/019:10:120
670590240,1sleep10-21swapper/120:40:201
640432240,1sleep10-21swapper/119:30:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional