You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-23 - 18:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Nov 23, 2024 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39143992970,12sleep33914401-21rt-features07:05:383
391333128357,16sleep20-21swapper/207:05:022
391510399782,35cyclictest1753-21gnome-shell12:05:162
391472527740,10sleep00-21swapper/007:06:490
162399750,5rtkit-daemon2197014-21PK-Backend08:24:310
162399730,7rtkit-daemon2197014-21PK-Backend08:58:590
162399730,6rtkit-daemon2197014-21PK-Backend09:19:500
162399730,6rtkit-daemon2197014-21PK-Backend09:19:500
162399730,6rtkit-daemon2197014-21PK-Backend07:47:330
162399720,30rtkit-daemon3946996-21hddtemp_smartct08:35:181
391510399712,27cyclictest4016433-21df11:55:172
3915103997112,34cyclictest3996291-21sed10:55:212
391469427146,17sleep10-21swapper/107:06:251
162399710,5rtkit-daemon0-21swapper/007:10:110
391510399691,34cyclictest3947314-21ntpq08:35:232
391510399682,49cyclictest1753-21gnome-shell10:02:512
391510399682,45cyclictest1753-21gnome-shell11:38:472
391510399672,48cyclictest1753-21gnome-shell12:19:332
391510399671,44cyclictest3966387-21cat09:30:212
391510399661,46cyclictest4003195-21cnf-update-db11:15:042
391510399661,37cyclictest3918158-21expr07:15:152
391510399653,35cyclictest3998487-40lsb_release11:00:252
391510399653,35cyclictest3998487-40lsb_release11:00:252
391510399652,36cyclictest1753-21gnome-shell08:13:012
39827102640,7sleep21753-21gnome-shell10:15:302
3915103996423,36cyclictest438-20systemd-journal09:07:442
391510399641,39cyclictest3981614-21systemd-run10:15:012
391510399641,39cyclictest3926678-21idleruntime-cro07:40:002
391510399641,37cyclictest4018492-21basename12:00:212
391510399632,42cyclictest3915313-21apt07:10:112
391510399632,37cyclictest1753-21gnome-shell07:30:272
391510399632,37cyclictest1753-21gnome-shell07:30:272
391510399632,35cyclictest3942255-40lsb_release08:21:272
391510399632,35cyclictest1753-21gnome-shell11:46:332
391510399623,32cyclictest3946430-21date08:35:022
391510399622,43cyclictest1753-21gnome-shell08:42:522
391510399622,37cyclictest1753-21gnome-shell07:27:522
391510399622,35cyclictest1753-21gnome-shell12:35:242
391510399622,34cyclictest1753-21gnome-shell10:50:182
391510399621,35cyclictest3975677-21ntp_states09:55:262
3915103996211,30cyclictest3969895-21cpuspeed_turbos09:40:132
3915103996211,27cyclictest0-21swapper/208:00:302
391510399612,53cyclictest1753-21gnome-shell09:02:522
391510399612,43cyclictest1753-21gnome-shell11:13:272
391510399612,42cyclictest1753-21gnome-shell09:35:562
391510399612,33cyclictest3991638-21users10:40:312
391510399612,33cyclictest3991638-21users10:40:312
391510399612,32cyclictest1753-21gnome-shell12:10:032
391510399611,33cyclictest3953672-21dpkg08:55:032
391510399602,42cyclictest1753-21gnome-shell07:48:132
391510399602,31cyclictest4026160-21/usr/sbin/munin12:20:292
391510399601,36cyclictest3960522-40apt-check09:12:222
39507892590,8sleep013-21ksoftirqd/008:45:250
391510399592,31cyclictest3961445-21diskstats09:15:162
391510399592,31cyclictest3961445-21diskstats09:15:152
391510399591,54cyclictest4005893-21sensors11:20:282
391510399582,34cyclictest1753-21gnome-shell08:49:032
391510399582,33cyclictest4027669-21memory12:25:222
391510399582,33cyclictest4027669-21memory12:25:222
391510399582,33cyclictest1753-21gnome-shell09:26:382
391510399582,33cyclictest1753-21gnome-shell07:42:582
3915103995819,34cyclictest4010934-21cat11:40:112
3915103995818,35cyclictest3944158-21sed08:25:292
3915103995815,32cyclictest1-21systemd11:10:002
391510399581,36cyclictest4014896-21latency11:50:212
391510399581,28cyclictest854-21avahi-daemon10:28:202
391510399581,26cyclictest0-21swapper/211:30:102
162399580,8rtkit-daemon1622-21rtkit-daemon08:00:183
162399580,5rtkit-daemon2197014-21PK-Backend09:41:021
391510399571,32cyclictest3953354-21apt-key08:55:012
3915103995617,34cyclictest3935833-21cat08:05:132
391510399561,31cyclictest3920040-21/usr/sbin/munin07:20:212
162399560,8rtkit-daemon1622-21rtkit-daemon09:57:011
162399560,8rtkit-daemon1622-21rtkit-daemon09:53:440
391510399552,30cyclictest1753-21gnome-shell10:05:102
162399550,9rtkit-daemon1622-21rtkit-daemon10:58:011
162399550,7rtkit-daemon1622-21rtkit-daemon10:27:401
3915103995425,25cyclictest3963667-21sensors_temp09:20:282
3915103995423,27cyclictest3988800-21/usr/sbin/munin10:35:292
3915103995423,27cyclictest1069-21runrttasks07:56:042
3915103995416,33cyclictest3930931-21cat07:50:192
3915103995416,33cyclictest3930931-21cat07:50:192
162399540,9rtkit-daemon1622-21rtkit-daemon10:49:501
162399540,8rtkit-daemon1622-21rtkit-daemon10:10:391
162399540,8rtkit-daemon1622-21rtkit-daemon09:37:311
3915103995317,31cyclictest3242253-21nautilus11:30:002
391510399531,47cyclictest3971964-21kernelversion09:45:212
391510399531,28cyclictest0-21swapper/210:31:252
162399530,8rtkit-daemon1622-21rtkit-daemon10:50:461
391510399521,28cyclictest13581kworker/u9:108:20:022
39177702510,5sleep1251rcuc/107:15:101
3915105995122,25cyclictest4018597-40lsb_release12:00:243
3915103995120,27cyclictest3984485-21switchtime10:20:322
3915103995118,29cyclictest1069-21runrttasks12:34:262
391510399511,46cyclictest1753-21gnome-shell09:52:102
39938512500,5sleep00-21swapper/010:50:020
39878232500,13sleep3411rcuc/310:30:283
391510399508,38cyclictest1753-21gnome-shell10:48:262
40036642490,5sleep32197014-21PK-Backend11:15:203
40094122480,8sleep11753-21gnome-shell11:35:141
3915101994711,18cyclictest26-21ksoftirqd/109:30:211
162399470,39rtkit-daemon3930158-21munin-run07:50:023
162399470,34rtkit-daemon2197014-21PK-Backend11:04:433
162399470,34rtkit-daemon2197014-21PK-Backend11:04:423
162399460,38rtkit-daemon2197014-21PK-Backend10:40:490
162399460,38rtkit-daemon2197014-21PK-Backend10:40:480
3915105994527,15cyclictest1753-21gnome-shell07:31:483
3915105994527,15cyclictest1753-21gnome-shell07:31:483
3915105994510,17cyclictest1753-21gnome-shell11:39:053
3915101994521,20cyclictest4030353-21unin-run12:35:001
3915101994516,13cyclictest26-21ksoftirqd/112:10:301
3915101994418,10cyclictest3950732-21grep08:45:231
162399440,36rtkit-daemon1753-21gnome-shell07:57:183
162399440,31rtkit-daemon1753-21gnome-shell11:05:573
3915105994320,19cyclictest3980997-21python310:10:333
3915101994326,12cyclictest438-20systemd-journal07:15:001
162399430,29rtkit-daemon3915091-21cyclictest08:57:411
3915105994229,9cyclictest42-21ksoftirqd/310:25:353
3915105994217,7cyclictest3922091-21proc_pri07:25:263
162399420,4rtkit-daemon2197014-21PK-Backend10:17:520
40201652410,7sleep013-21ksoftirqd/012:05:160
39660092410,9sleep03966006-21perf09:29:590
39308912410,8sleep02197014-21PK-Backend07:50:190
39308912410,8sleep02197014-21PK-Backend07:50:190
3915105994128,9cyclictest42-21ksoftirqd/308:25:013
3915105994116,13cyclictest42-21ksoftirqd/307:15:203
391510199414,16cyclictest0-21swapper/111:10:341
3915101994130,7cyclictest3978603-21wc10:05:151
391510199412,35cyclictest1753-21gnome-shell09:25:211
391509899411,10cyclictest4012346-21munin-plugin-st11:45:010
162399410,29rtkit-daemon0-21swapper/311:53:363
391510599409,13cyclictest1753-21gnome-shell12:11:433
391510599408,28cyclictest1753-21gnome-shell09:03:563
3915105994023,12cyclictest3958752-21apt-get09:10:093
3915105994022,14cyclictest3926013-21python307:35:283
3915105994022,13cyclictest438-20systemd-journal08:38:073
391510199402,9cyclictest1753-21gnome-shell09:15:581
391510199402,9cyclictest1753-21gnome-shell09:15:581
3915101994029,7cyclictest1753-21gnome-shell12:00:021
3915101994025,11cyclictest3974645-21cron09:55:001
3915101994017,6cyclictest141rcu_preempt07:55:011
3915101994017,6cyclictest141rcu_preempt07:55:001
3915098994027,9cyclictest4030583-21awk12:35:120
391509899401,9cyclictest2197014-21PK-Backend09:20:590
3915098994019,16cyclictest4016783-21awk11:55:250
162399400,32rtkit-daemon3915091-21cyclictest10:38:061
162399400,28rtkit-daemon2197014-21PK-Backend09:07:260
162399400,10rtkit-daemon1753-21gnome-shell11:11:143
39895352390,7chrt2197014-21PK-Backend10:35:250
39232802390,5sleep02197014-21PK-Backend07:30:060
39232802390,5sleep02197014-21PK-Backend07:30:060
3915105993924,10cyclictest1753-21gnome-shell08:57:163
391510599392,33cyclictest1753-21gnome-shell11:27:473
391510599392,32cyclictest4029197-21diskstats12:30:213
3915105993922,13cyclictest3930395-21apt07:50:093
3915105993922,13cyclictest3930395-21apt07:50:093
391510599391,10cyclictest3949499-21seq08:41:253
3915101993927,8cyclictest1753-21gnome-shell11:29:301
3915101993919,15cyclictest438-20systemd-journal12:06:431
3915101993917,18cyclictest4030998-21smartctl12:35:221
3915101993916,12cyclictest3936471-21python308:05:301
391510199391,19cyclictest3949793-21sh08:45:001
391510199391,14cyclictest0-21swapper/108:54:141
3915098993921,14cyclictest4003255-40apt-check11:15:190
3915098993921,13cyclictest3940528-21idleruntime-cro08:20:000
162399390,31rtkit-daemon13-21ksoftirqd/010:55:410
391510599382,7cyclictest1753-21gnome-shell08:32:223
3915105993824,10cyclictest3983278-21sshd10:19:453
3915105993821,12cyclictest3921209-21cat07:25:023
3915105993820,13cyclictest438-20systemd-journal08:13:033
391510599381,32cyclictest42-21ksoftirqd/310:00:103
391510199388,27cyclictest3923396-40lsb_release07:30:231
391510199388,27cyclictest3923396-40lsb_release07:30:231
391510199382,9cyclictest0-21swapper/107:37:271
391510199382,7cyclictest1753-21gnome-shell11:41:421
3915101993827,7cyclictest1753-21gnome-shell08:30:181
3915101993827,7cyclictest1753-21gnome-shell07:26:211
391510199382,32cyclictest1753-21gnome-shell12:00:141
3915101993821,12cyclictest3958766-21/usr/sbin/munin09:10:021
391510199381,14cyclictest0-21swapper/107:49:061
162399380,26rtkit-daemon2197014-21PK-Backend08:52:210
3915105993724,9cyclictest42-21ksoftirqd/308:10:013
3915105993724,9cyclictest1753-21gnome-shell10:20:153
3915105993723,11cyclictest3978381-21awk10:05:123
3915105993723,10cyclictest42-21ksoftirqd/312:37:453
391510599372,30cyclictest3993370-21users10:45:313
391510599372,11cyclictest0-21swapper/310:59:293
3915105993721,12cyclictest4028580-21systemd12:30:023
3915105993721,12cyclictest4028580-21systemd12:30:023
3915105993721,12cyclictest1-21systemd07:45:013
3915105993720,13cyclictest4007660-21expr11:30:123
3915105993718,15cyclictest1632-21dbus-daemon07:15:013
3915101993727,7cyclictest1753-21gnome-shell12:21:281
3915101993727,7cyclictest1753-21gnome-shell09:21:371
3915101993726,7cyclictest1753-21gnome-shell12:29:271
3915101993726,7cyclictest1753-21gnome-shell12:29:271
3915101993726,7cyclictest1753-21gnome-shell11:51:451
3915101993726,7cyclictest1753-21gnome-shell11:48:011
3915101993726,7cyclictest1753-21gnome-shell11:21:421
3915101993726,7cyclictest1753-21gnome-shell11:00:251
3915101993726,7cyclictest1753-21gnome-shell11:00:251
3915101993726,7cyclictest1753-21gnome-shell09:45:491
3915101993719,5cyclictest26-21ksoftirqd/110:42:121
3915101993719,5cyclictest26-21ksoftirqd/110:42:121
3915101993715,19cyclictest251rcuc/111:15:201
391510199371,16cyclictest26-21ksoftirqd/107:22:171
391510199371,14cyclictest0-21swapper/110:33:431
3915101993711,22cyclictest4024096-21kernelversion12:15:231
391509899371,4cyclictest2197014-21PK-Backend07:26:320
162399370,25rtkit-daemon0-21swapper/107:40:471
3915105993622,10cyclictest42-21ksoftirqd/308:28:213
3915105993622,10cyclictest4012961-21expr11:45:153
3915105993621,11cyclictest42-21ksoftirqd/312:19:193
3915105993620,11cyclictest438-20systemd-journal08:17:433
3915105993619,13cyclictest4020423-21memory12:05:243
3915105993614,10cyclictest42-21ksoftirqd/309:55:303
391510599361,21cyclictest175350gnome-shell09:30:333
391510199362,6cyclictest141rcu_preempt09:13:501
3915101993625,7cyclictest1753-21gnome-shell08:01:321
3915101993625,7cyclictest1753-21gnome-shell07:55:171
3915101993621,11cyclictest3940525-21perf08:20:011
391510199361,13cyclictest2197014-21PK-Backend08:25:101
3915098993621,11cyclictest3960800-21/usr/sbin/munin09:15:020
3915098993620,12cyclictest3999214-21sshd11:04:450
3915098993620,12cyclictest3999214-21sshd11:04:440
3915098993612,7cyclictest141rcu_preempt07:15:320
391509899361,18cyclictest4018847-40lsb_release12:00:280
39723912350,7sleep00-21swapper/009:45:300
3915105993525,6cyclictest3972219-21ntpq09:45:263
3915105993524,7cyclictest4026831-21basename12:25:003
391510599352,28cyclictest1753-21gnome-shell11:41:043
3915105993520,11cyclictest42-21ksoftirqd/309:16:223
3915105993520,11cyclictest42-21ksoftirqd/309:16:223
3915105993518,13cyclictest42-21ksoftirqd/309:44:203
3915105993516,15cyclictest2179-21gvfsd-metadata09:10:013
391510599351,30cyclictest3952667-21python308:50:303
391510599351,30cyclictest1753-21gnome-shell10:53:343
391510599351,20cyclictest175350gnome-shell11:20:143
3915101993524,7cyclictest1753-21gnome-shell08:10:511
3915101993519,11cyclictest438-20systemd-journal11:35:001
3915098993519,12cyclictest4015076-21grep11:50:230
3915098993519,12cyclictest3975920-21cat09:55:300
391509899351,30cyclictest2197014-21PK-Backend11:21:020
3915105993422,8cyclictest3965992-21dump-pmu-power09:30:003
3915105993420,10cyclictest42-21ksoftirqd/311:56:353
3915105993419,11cyclictest3962942-21grep09:20:133
3915105993418,12cyclictest3990274-21cron10:40:003
3915105993412,19cyclictest1753-21gnome-shell09:37:523
391510599341,13cyclictest3974071-21python309:50:333
3915101993423,7cyclictest3982566-21grep10:15:261
3915101993418,12cyclictest3941451-21hddtemp_smartct08:20:191
3915101993413,6cyclictest141rcu_preempt09:00:231
3915101993412,10cyclictest3976768-21cat10:00:131
391509899348,7cyclictest17531gnome-shell07:40:270
3915098993423,7cyclictest3944378-21fuser08:27:280
3915098993419,11cyclictest855-21dbus-daemon12:15:010
3915098993419,11cyclictest3977381-21awk10:00:230
391509899341,8cyclictest2197014-21PK-Backend11:29:580
3915098993415,16cyclictest1753-21gnome-shell07:35:130
391509899341,29cyclictest2197014-21PK-Backend08:40:240
391509899341,14cyclictest2197014-21PK-Backend11:46:130
3915098993410,6cyclictest141rcu_preempt12:15:250
39508972330,5chrt0-21swapper/308:45:273
391510599331,12cyclictest3990783-21expr10:40:143
391510599331,12cyclictest3990783-21expr10:40:133
3915101993316,13cyclictest4000253-21ntpq11:05:241
391509899332,5cyclictest0-21swapper/011:05:060
391509899331,9cyclictest3921047-21grep07:25:000
391509899331,9cyclictest2197014-21PK-Backend10:25:070
391509899331,8cyclictest2197014-21PK-Backend10:20:310
391509899331,29cyclictest2197014-21PK-Backend12:26:250
391509899331,29cyclictest2197014-21PK-Backend12:26:250
391509899331,14cyclictest2197014-21PK-Backend07:56:070
3915101993220,9cyclictest3984319-21awk10:20:261
391509899324,9cyclictest1753-21gnome-shell11:35:280
3915098993224,5cyclictest2197014-21PK-Backend10:54:450
391509899321,9cyclictest2197014-21PK-Backend08:14:510
3915098993218,10cyclictest4002757-21gpgv11:15:020
3915098993217,11cyclictest4008880-21ls11:35:000
3915098993217,10cyclictest3945116-21expr08:30:150
391509899321,28cyclictest2197014-21PK-Backend10:06:000
391509899321,27cyclictest2197014-21PK-Backend09:35:240
3915098993210,18cyclictest1753-21gnome-shell10:32:550
3915098993210,18cyclictest1753-21gnome-shell09:00:530
391509899319,18cyclictest1753-21gnome-shell09:30:220
391509899319,18cyclictest1753-21gnome-shell08:35:590
3915098993123,5cyclictest2197014-21PK-Backend12:21:260
3915098993121,7cyclictest3936074-21irqstats08:05:200
3915098993115,12cyclictest855-21dbus-daemon12:32:190
391509899311,4cyclictest2197014-21PK-Backend10:10:040
3915098993113,15cyclictest1753-21gnome-shell08:00:310
162399300,23rtkit-daemon2197014-21PK-Backend09:42:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional