You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-03 - 11:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Thu Apr 03, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1623991410,16rtkit-daemon0-21swapper/221:05:522
3545621190,6sleep00-21swapper/023:25:040
257182980,7sleep30-21swapper/322:55:013
162399880,71rtkit-daemon0-21swapper/219:05:362
414420328061,13sleep30-21swapper/319:05:523
162399770,6rtkit-daemon2197014-21PK-Backend21:37:102
414439027640,9sleep00-21swapper/019:08:180
162399760,7rtkit-daemon2197014-21PK-Backend23:18:482
162399750,7rtkit-daemon2197014-21PK-Backend21:40:472
41618422740,4chrt4161202-21/usr/sbin/munin19:55:202
162399740,9rtkit-daemon1622-21rtkit-daemon19:50:523
162399730,6rtkit-daemon2197014-21PK-Backend20:02:472
162399690,5rtkit-daemon0-21swapper/222:52:532
414431426840,22sleep10-21swapper/119:07:201
162399680,9rtkit-daemon3242254-21gmain21:48:000
162399680,5rtkit-daemon0-21swapper/222:22:252
499692640,7chrt2197014-21PK-Backend00:15:190
162399630,18rtkit-daemon0-21swapper/300:17:083
41819492620,32sleep342-21ksoftirqd/320:50:273
41654402570,9chrt1753-21gnome-shell20:05:272
162399560,8rtkit-daemon1622-21rtkit-daemon21:48:211
41933922550,7chrt2197014-21PK-Backend21:21:183
162399550,8rtkit-daemon1622-21rtkit-daemon19:29:422
414465299541,17cyclictest908-21gdbus20:49:340
162399540,8rtkit-daemon1622-21rtkit-daemon19:11:053
162399540,8rtkit-daemon1622-21rtkit-daemon19:11:053
41906352530,9sleep30-21swapper/321:15:193
41868432530,4sleep00-21swapper/021:05:140
414465799532,16cyclictest1753-21gnome-shell20:15:071
414465799532,16cyclictest1753-21gnome-shell20:15:061
414466299521,16cyclictest4185651-40lsb_release21:00:243
4144662995211,36cyclictest1753-21gnome-shell23:55:123
414465799522,26cyclictest4180589-21seq20:45:581
162399520,7rtkit-daemon1622-21rtkit-daemon19:48:043
414465799511,23cyclictest7320-21memory21:50:181
4144652995136,11cyclictest13-21ksoftirqd/021:00:010
162399510,38rtkit-daemon42-21ksoftirqd/321:56:303
92502500,8chrt1753-21gnome-shell21:55:261
414466299502,44cyclictest1753-21gnome-shell21:09:193
414466299502,43cyclictest1753-21gnome-shell21:27:483
414466299502,43cyclictest1753-21gnome-shell20:36:173
414466299502,42cyclictest35675-40lsb_release23:27:253
414466299502,15cyclictest1753-21gnome-shell20:45:133
4144662995015,30cyclictest860997-21snmpd22:56:273
414466299501,43cyclictest54324-40lsb_release00:30:203
4144662995012,34cyclictest4178912-21pool-org.gnome.20:45:013
414466299501,14cyclictest4167391-21grep20:10:333
414465799507,20cyclictest1069-21runrttasks21:41:361
414465799506,21cyclictest4154597-21missed_timers19:35:231
414465799503,21cyclictest15034-21ntpq22:15:271
414465799502,22cyclictest4172015-21hddtemp_smartct20:25:191
4144652995029,14cyclictest13-21ksoftirqd/020:05:540
414466299499,35cyclictest1753-21gnome-shell19:31:463
414466299498,37cyclictest1753-21gnome-shell22:10:343
414466299492,42cyclictest1753-21gnome-shell23:51:353
414466299492,42cyclictest1753-21gnome-shell21:40:143
414466299492,42cyclictest1753-21gnome-shell21:36:303
414466299492,42cyclictest1753-21gnome-shell20:23:253
414466299492,42cyclictest1753-21gnome-shell00:35:253
4144662994918,26cyclictest4154453-21if_enp2s019:35:223
4144662994915,30cyclictest859-21NetworkManager00:00:033
4144662994910,34cyclictest1753-21gnome-shell23:15:073
414465799493,22cyclictest869-21irqbalance20:10:551
4144652994939,6cyclictest13-21ksoftirqd/022:41:540
41510152480,8sleep126-21ksoftirqd/119:25:221
414466299489,34cyclictest438-20systemd-journal23:34:343
4144662994828,16cyclictest1827-21JS14:31:443
414466299482,42cyclictest1753-21gnome-shell21:46:073
414466299482,41cyclictest1753-21gnome-shell23:44:233
414466299482,41cyclictest1753-21gnome-shell23:44:223
414466299482,41cyclictest1753-21gnome-shell22:01:473
414466299482,41cyclictest1753-21gnome-shell22:01:463
414466299482,41cyclictest1753-21gnome-shell20:57:253
414466299482,41cyclictest1753-21gnome-shell00:12:203
4144662994823,21cyclictest1753-21gnome-shell20:02:343
414466299481,43cyclictest1753-21gnome-shell21:53:153
4144652994823,20cyclictest4190464-21expr21:15:140
162399480,9rtkit-daemon35660-40lsb_release23:27:232
414466299473,8cyclictest1753-21gnome-shell22:15:183
414466299472,41cyclictest1753-21gnome-shell20:26:483
414466299472,41cyclictest1753-21gnome-shell00:24:113
4144662994720,23cyclictest1753-21gnome-shell20:34:453
4144662994720,22cyclictest27783-21cpuspeed_turbos23:00:163
4144662994717,26cyclictest1753-21gnome-shell23:37:043
4144659994726,16cyclictest34-21ksoftirqd/200:30:152
414465799476,25cyclictest1753-21gnome-shell19:20:441
414465799474,27cyclictest1069-21runrttasks21:06:081
414466299469,33cyclictest1753-21gnome-shell23:10:013
414466299467,34cyclictest0-21swapper/323:10:183
414466299462,8cyclictest1753-21gnome-shell22:35:313
4144662994622,19cyclictest22721-21python322:40:303
414466299462,10cyclictest1753-21gnome-shell20:06:173
414466299461,41cyclictest1753-21gnome-shell21:15:013
414466299461,40cyclictest1753-21gnome-shell19:55:253
414465799468,16cyclictest4176823-21taskset20:36:221
414465799467,11cyclictest43270-21taskset23:52:481
414465799462,19cyclictest1753-21gnome-shell21:20:151
4144652994629,13cyclictest13-21ksoftirqd/022:56:150
414465299461,5cyclictest141rcu_preempt22:50:010
4144662994524,18cyclictest0-21swapper/322:20:563
4144662994520,20cyclictest1753-21gnome-shell19:15:243
414466299451,40cyclictest1753-21gnome-shell19:29:373
414466299451,39cyclictest34376-21python323:20:283
414465799459,18cyclictest854-21avahi-daemon00:12:521
414465799458,17cyclictest1753-21gnome-shell00:25:241
414465799458,16cyclictest1753-21gnome-shell19:43:581
414465799458,14cyclictest4184232-21taskset20:57:481
414465799457,15cyclictest39634-21apt23:40:121
414465799457,15cyclictest39634-21apt23:40:121
414465799455,21cyclictest16614-21memory22:20:271
414465799455,17cyclictest2227-21taskset21:33:171
414465799452,26cyclictest1753-21gnome-shell22:04:571
414465799452,26cyclictest1753-21gnome-shell22:04:571
414465799451,16cyclictest4160654-40lsb_release19:51:261
4144652994528,13cyclictest13-21ksoftirqd/022:27:430
414465299451,28cyclictest13-21ksoftirqd/022:53:280
414466299449,31cyclictest1753-21gnome-shell22:33:253
414466299448,31cyclictest1753-21gnome-shell20:17:083
414466299448,31cyclictest1753-21gnome-shell20:17:073
414466299447,32cyclictest1753-21gnome-shell22:26:273
4144662994419,21cyclictest391irq_work/323:45:163
414466299441,39cyclictest1753-21gnome-shell00:28:513
4144662994413,27cyclictest4155830-21cpuspeed_turbos19:40:143
414465799449,21cyclictest1809350journal-offline22:29:551
414465799447,15cyclictest1753-21gnome-shell23:27:161
414465799447,11cyclictest37441-21taskset23:32:401
414465799447,10cyclictest32480-21awk23:15:221
414465799442,14cyclictest19281-21sort22:30:251
4144652994418,9cyclictest13-21ksoftirqd/000:10:270
162399440,6rtkit-daemon2197014-21PK-Backend22:32:082
4144662994317,22cyclictest42-21ksoftirqd/322:06:533
414465799439,11cyclictest4157865-21hddtemp_smartct19:45:191
414465799439,10cyclictest53404-21apt-get00:30:081
414465799438,14cyclictest4162424-21taskset19:56:471
414465799432,18cyclictest1753-21gnome-shell20:33:141
414465799432,18cyclictest1753-21gnome-shell00:00:251
414465799431,26cyclictest1753-21gnome-shell00:07:011
414465799431,20cyclictest49548-21/usr/sbin/munin00:15:331
414465799430,18cyclictest45224-40updatedb.mlocat00:00:031
414465799430,17cyclictest21514-21runrttasks22:39:271
4144652994327,11cyclictest4148490-21sshd19:19:290
4144652994326,13cyclictest13-21ksoftirqd/022:09:480
414465299431,22cyclictest16749-21chrt22:20:280
414466299422,15cyclictest1753-21gnome-shell00:08:303
414466299421,16cyclictest1753-21gnome-shell19:20:243
414465799428,11cyclictest1753-21gnome-shell22:08:111
414465799427,15cyclictest1753-21gnome-shell21:10:551
414465799426,15cyclictest1753-21gnome-shell22:59:311
414465799424,23cyclictest1753-21gnome-shell21:03:411
414465799422,17cyclictest1753-21gnome-shell19:15:261
414465799422,16cyclictest1753-21gnome-shell23:20:131
414465799421,25cyclictest1753-21gnome-shell23:03:341
414465799421,18cyclictest13049-21cut22:10:161
414465799421,15cyclictest31052-21sed23:10:251
414465299422,9cyclictest1753-21gnome-shell23:04:310
4144652994218,20cyclictest4174054-40lsb_release20:30:240
4144652994215,6cyclictest141rcu_preempt19:43:580
162399420,11rtkit-daemon2197014-21PK-Backend23:22:062
4144662994117,20cyclictest391irq_work/322:45:333
4144659994123,13cyclictest4185001-40apt-check21:00:182
414465799417,10cyclictest56995-21runrttasks00:38:041
414465799416,15cyclictest1753-21gnome-shell21:18:361
414465799412,15cyclictest22718-21python322:40:301
414465799410,17cyclictest4163702-21ntp_states20:00:251
4144652994123,7cyclictest13-21ksoftirqd/022:11:230
414465299412,34cyclictest1753-21gnome-shell19:21:370
4144652994111,11cyclictest13-21ksoftirqd/020:50:520
4144659994022,14cyclictest4176801-40apt-check20:36:162
414465999401,8cyclictest0-21swapper/219:37:462
4144659994017,18cyclictest4174297-40lsb_release20:30:282
414465799409,9cyclictest4165441-21ntp_states20:05:271
4144657994024,12cyclictest4125-21/usr/sbin/munin21:40:011
414465799400,20cyclictest51688-21entropy00:20:201
4144652994027,9cyclictest1753-21gnome-shell20:13:530
4144652994026,9cyclictest1753-21gnome-shell21:31:080
4144652994026,10cyclictest4193682-21sshd21:24:350
414465299402,33cyclictest1753-21gnome-shell23:06:490
4144652994021,16cyclictest54-21kcompactd021:45:010
4144652994016,7cyclictest324862tr23:15:230
414465299401,4cyclictest141rcu_preempt20:36:120
162399400,7rtkit-daemon854-21avahi-daemon00:36:222
414465999393,32cyclictest1069-21runrttasks22:15:172
4144659993923,12cyclictest4158929-21apt-get19:50:012
4144659993921,14cyclictest4167294-21sensors_temp20:10:312
4144659993919,15cyclictest3242253-21nautilus19:35:012
414465799398,9cyclictest54-21kcompactd020:41:071
414465799396,12cyclictest26-21ksoftirqd/121:29:001
414465799395,11cyclictest1753-21gnome-shell23:06:381
414465799392,12cyclictest1753-21gnome-shell19:33:201
4144652993927,8cyclictest13-21ksoftirqd/000:30:020
4144652993926,10cyclictest13-21ksoftirqd/019:15:020
4144652993926,10cyclictest13-21ksoftirqd/019:15:020
4144652993919,12cyclictest13-21ksoftirqd/019:56:260
162399390,25rtkit-daemon2197014-21PK-Backend21:34:182
414465799386,9cyclictest0-21swapper/123:49:501
414465799380,14cyclictest1753-21gnome-shell22:46:261
414465299382,8cyclictest1753-21gnome-shell21:25:110
4144652993827,7cyclictest1753-21gnome-shell19:29:100
4144652993825,9cyclictest13-21ksoftirqd/023:25:010
4144652993825,9cyclictest13-21ksoftirqd/022:00:290
4144652993825,9cyclictest13-21ksoftirqd/022:00:290
4144652993824,10cyclictest4169326-21sshd20:16:300
4144652993824,10cyclictest4169326-21sshd20:16:300
4144652993820,14cyclictest13-21ksoftirqd/000:09:170
4144652993820,13cyclictest438-20systemd-journal23:58:350
4144652993820,13cyclictest3014-21df21:35:160
4144652993818,7cyclictest13-21ksoftirqd/021:02:140
4144652993815,6cyclictest141rcu_preempt19:30:270
162399380,8rtkit-daemon1753-21gnome-shell20:44:552
162399380,27rtkit-daemon0-21swapper/222:40:062
414465299376,7cyclictest141rcu_preempt00:31:410
414465299373,16cyclictest0-21swapper/020:42:460
4144652993726,7cyclictest4158924-21sh19:50:000
4144652993726,7cyclictest1753-21gnome-shell20:24:140
4144652993721,12cyclictest43489-21systemd23:55:010
4144652993721,11cyclictest438-20systemd-journal22:39:350
4144652993717,6cyclictest13-21ksoftirqd/019:50:490
162399370,26rtkit-daemon2197014-21PK-Backend20:29:482
162399370,25rtkit-daemon0-21swapper/223:33:492
4144659993621,11cyclictest47562-21cron00:10:002
4144659993618,6cyclictest34-21ksoftirqd/200:01:402
414465999361,7cyclictest1753-21gnome-shell22:29:562
4144659993612,6cyclictest141rcu_preempt21:00:032
414465799366,25cyclictest4181234-21apt20:50:151
414465299367,11cyclictest13-21ksoftirqd/020:28:020
4144652993621,11cyclictest45724-21expr00:00:170
4144652993621,11cyclictest13-21ksoftirqd/023:48:400
4144652993620,11cyclictest438-20systemd-journal23:39:350
4144652993618,6cyclictest13-21ksoftirqd/020:02:090
414465299361,6cyclictest2197014-21PK-Backend22:15:130
4144652993613,6cyclictest141rcu_preempt21:57:500
414465299361,19cyclictest31229-21runrttasks23:11:360
162399360,25rtkit-daemon2197014-21PK-Backend20:21:002
162399360,22rtkit-daemon141rcu_preempt21:16:202
4144659993524,7cyclictest4147259-21tr19:15:172
414465999351,9cyclictest141rcu_preempt00:20:142
4144659993518,13cyclictest9055-21memory21:55:212
4144659993518,12cyclictest4180242-21proc_pri20:45:292
4144659993516,14cyclictest3242253-21nautilus21:50:012
4144659993513,6cyclictest141rcu_preempt19:40:072
414465999351,12cyclictest4181190-21chrt20:50:032
414465299354,27cyclictest0-21swapper/000:38:010
4144652993521,10cyclictest19490-21sshd22:30:350
4144652993518,13cyclictest13-21ksoftirqd/021:10:060
414465299351,19cyclictest39720-21/usr/sbin/munin23:40:150
414465299351,19cyclictest39720-21/usr/sbin/munin23:40:140
4144659993427,4cyclictest2197014-21PK-Backend23:36:142
4144659993420,10cyclictest41199-21sed23:45:022
4144659993420,10cyclictest41199-21sed23:45:022
4144659993419,11cyclictest49885-21expr00:15:162
4144659993417,12cyclictest438-20systemd-journal21:55:002
414465999341,5cyclictest2197014-21PK-Backend21:15:012
4144657993418,12cyclictest4145228-21/usr/sbin/munin19:10:151
4144657993418,12cyclictest4145228-21/usr/sbin/munin19:10:151
414465299347,11cyclictest141rcu_preempt21:50:220
4144652993418,12cyclictest13-21ksoftirqd/023:33:150
414465999337,9cyclictest141rcu_preempt19:15:012
414465999337,9cyclictest141rcu_preempt19:15:012
4144659993322,7cyclictest21581-21sh22:40:012
414465999331,9cyclictest2197014-21PK-Backend23:52:572
414465999331,9cyclictest141rcu_preempt20:16:272
414465999331,9cyclictest141rcu_preempt20:16:272
4144659993319,10cyclictest28261-21cat23:00:252
414465999331,29cyclictest2197014-21PK-Backend23:49:232
4144659993311,6cyclictest141rcu_preempt19:53:172
414465999331,12cyclictest4149919-21chrt19:20:362
4144657993317,13cyclictest1753-21gnome-shell20:20:131
4144657993316,12cyclictest24235-21grep22:50:081
414465799331,12cyclictest26-21ksoftirqd/123:39:171
414465299335,6cyclictest0-21swapper/000:20:190
4144652993318,5cyclictest13-21ksoftirqd/019:35:150
162399330,5rtkit-daemon2197014-21PK-Backend23:07:262
41930792320,7sleep22197014-21PK-Backend21:20:302
414465999321,28cyclictest2197014-21PK-Backend22:11:112
414465999321,28cyclictest2197014-21PK-Backend00:25:392
4144659993210,18cyclictest1753-21gnome-shell22:06:012
414465999311,9cyclictest2197014-21PK-Backend00:10:312
4144659993118,10cyclictest34-21ksoftirqd/221:25:402
4144659993114,13cyclictest855-21dbus-daemon22:05:012
4144659993114,13cyclictest855-21dbus-daemon22:05:012
414465999311,27cyclictest2197014-21PK-Backend22:55:362
414465999311,27cyclictest2197014-21PK-Backend22:49:572
414465999311,11cyclictest0-21swapper/223:10:472
4144659993110,17cyclictest1753-21gnome-shell23:55:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional