You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-24 - 12:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sun Nov 24, 2024 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5019121160,5sleep10-21swapper/121:05:251
333612960,9sleep234-21ksoftirqd/220:16:022
397012940,15sleep234-21ksoftirqd/220:35:242
8411999381,8cyclictest33193-21users20:15:343
1148362930,8sleep20-21swapper/200:10:222
841199922,87cyclictest0-21swapper/320:25:153
841199922,7cyclictest0-21swapper/322:50:133
677529245,39sleep10-21swapper/119:05:111
8411999082,5cyclictest0-21swapper/320:20:103
8406998931,9cyclictest1753-21gnome-shell21:35:272
8393998822,18cyclictest13-21ksoftirqd/022:00:300
791828848,31sleep30-21swapper/319:05:463
8401998727,19cyclictest26-21ksoftirqd/121:45:291
839399879,6cyclictest141rcu_preempt20:15:510
8406998625,56cyclictest34-21ksoftirqd/221:45:142
8393998610,5cyclictest141rcu_preempt21:36:410
8406998525,10cyclictest34-21ksoftirqd/222:30:322
162399850,65rtkit-daemon0-21swapper/019:05:140
8406998424,9cyclictest1753-21gnome-shell21:05:172
8406998423,10cyclictest1753-21gnome-shell21:55:042
8406998223,8cyclictest34-21ksoftirqd/222:35:312
8393998161,15cyclictest13-21ksoftirqd/020:35:300
8393998118,20cyclictest13-21ksoftirqd/021:45:310
8406998021,9cyclictest34-21ksoftirqd/222:00:302
839399804,7cyclictest141rcu_preempt21:07:410
839399797,5cyclictest141rcu_preempt21:10:280
8411997722,52cyclictest1753-21gnome-shell21:19:543
215502760,5sleep1251rcuc/119:45:141
8411997568,5cyclictest0-21swapper/321:53:303
8411997520,51cyclictest1753-21gnome-shell21:57:353
8411997467,4cyclictest2197014-21PK-Backend21:45:203
8406997457,5cyclictest34-21ksoftirqd/221:00:022
840699732,55cyclictest1753-21gnome-shell22:56:192
8406997321,48cyclictest34-21ksoftirqd/221:50:572
841199721,16cyclictest1753-21gnome-shell21:40:263
841199721,12cyclictest1753-21gnome-shell21:10:553
8401997261,7cyclictest1753-21gnome-shell21:03:371
8401997234,7cyclictest141rcu_preempt20:35:141
8393997216,13cyclictest13-21ksoftirqd/021:03:220
841199717,44cyclictest1753-21gnome-shell20:40:013
8406997153,6cyclictest34-21ksoftirqd/222:49:502
840699713,6cyclictest141rcu_preempt21:15:152
840699711,7cyclictest141rcu_preempt21:34:572
808827144,17sleep20-21swapper/219:07:572
841199702,48cyclictest1753-21gnome-shell22:45:223
841199702,47cyclictest1753-21gnome-shell21:30:073
8393997042,4cyclictest141rcu_preempt20:55:180
305562700,4chrt30555-21/usr/sbin/munin20:10:163
305562700,4chrt30555-21/usr/sbin/munin20:10:163
841199692,10cyclictest1753-21gnome-shell22:41:033
841199692,10cyclictest1753-21gnome-shell22:37:583
8411996910,43cyclictest141rcu_preempt22:28:303
8401996952,12cyclictest26-21ksoftirqd/120:15:021
8401996952,12cyclictest26-21ksoftirqd/120:15:011
8401996948,14cyclictest26-21ksoftirqd/120:17:351
841199688,7cyclictest141rcu_preempt20:45:143
8411996845,18cyclictest1753-21gnome-shell20:53:183
841199682,12cyclictest1753-21gnome-shell21:00:163
8406996858,6cyclictest1753-21gnome-shell22:40:112
8406996853,6cyclictest34-21ksoftirqd/221:01:012
8406996851,6cyclictest34-21ksoftirqd/221:40:442
8406996837,8cyclictest141rcu_preempt22:21:062
8393996853,11cyclictest13-21ksoftirqd/022:05:310
839399682,49cyclictest141rcu_preempt21:24:580
162399680,8rtkit-daemon1622-21rtkit-daemon20:08:513
8406996749,14cyclictest34-21ksoftirqd/221:21:042
8406996733,8cyclictest141rcu_preempt21:10:182
8406996732,8cyclictest141rcu_preempt22:54:382
8406996732,8cyclictest141rcu_preempt21:25:102
840699672,23cyclictest1753-21gnome-shell20:40:252
840199675,58cyclictest84315-21latency_hist22:45:021
8393996741,5cyclictest141rcu_preempt21:25:060
841199662,22cyclictest1753-21gnome-shell20:41:093
8406996655,7cyclictest1753-21gnome-shell22:06:342
8406996633,7cyclictest141rcu_preempt20:49:222
840699662,21cyclictest1753-21gnome-shell20:33:252
8401996655,7cyclictest60555-21latency21:35:201
839399662,21cyclictest1753-21gnome-shell22:10:550
839399662,21cyclictest1753-21gnome-shell20:23:290
8411996543,17cyclictest1753-21gnome-shell22:56:293
840199653,10cyclictest1753-21gnome-shell21:30:251
8401996526,11cyclictest26-21ksoftirqd/122:50:291
840199652,19cyclictest1753-21gnome-shell20:30:261
8401996515,46cyclictest74389-21cstates22:15:141
8393996548,6cyclictest13-21ksoftirqd/021:55:110
839399652,20cyclictest1753-21gnome-shell20:30:330
839399651,58cyclictest1753-21gnome-shell21:50:110
162399650,11rtkit-daemon1622-21rtkit-daemon23:05:102
8411996453,7cyclictest1069-21runrttasks22:03:213
8406996454,6cyclictest1753-21gnome-shell20:50:192
840699642,21cyclictest1753-21gnome-shell20:20:112
840699641,36cyclictest141rcu_preempt22:30:012
840199642,58cyclictest0-21swapper/120:50:151
840199642,19cyclictest1753-21gnome-shell20:45:181
8401996417,44cyclictest52764-21perf21:15:001
8393996427,8cyclictest141rcu_preempt21:15:110
8393996426,7cyclictest141rcu_preempt21:40:100
8393996426,13cyclictest0-21swapper/022:38:040
839399642,20cyclictest1753-21gnome-shell21:30:200
839399642,19cyclictest1753-21gnome-shell22:40:190
839399641,58cyclictest1753-21gnome-shell22:54:230
841199637,52cyclictest1753-21gnome-shell22:06:253
8411996355,5cyclictest411rcuc/322:16:193
841199632,57cyclictest1753-21gnome-shell21:28:343
841199632,20cyclictest1753-21gnome-shell21:20:153
841199632,19cyclictest1753-21gnome-shell20:58:003
841199631,57cyclictest1753-21gnome-shell20:32:373
840199638,50cyclictest79670-21expr22:30:171
840199632,57cyclictest1753-21gnome-shell21:40:151
840199632,19cyclictest1753-21gnome-shell21:50:221
8401996312,47cyclictest26-21ksoftirqd/122:02:311
840199631,19cyclictest1753-21gnome-shell21:55:201
839399632,56cyclictest1753-21gnome-shell20:27:410
839399632,55cyclictest1753-21gnome-shell22:32:400
839399631,9cyclictest2197014-21PK-Backend20:40:220
839399631,33cyclictest2197014-21PK-Backend22:25:000
839399631,20cyclictest1753-21gnome-shell22:28:280
8411996217,42cyclictest411rcuc/322:10:373
8411996216,43cyclictest411rcuc/321:35:033
8411996216,43cyclictest411rcuc/321:07:073
8411996215,7cyclictest411rcuc/322:30:483
840699622,17cyclictest1753-21gnome-shell22:17:122
840699621,55cyclictest1753-21gnome-shell20:25:212
8401996252,7cyclictest0-21swapper/122:36:421
840199622,56cyclictest1753-21gnome-shell22:55:181
840199621,50cyclictest1753-21gnome-shell20:55:171
840199621,19cyclictest1753-21gnome-shell22:10:141
839399622,56cyclictest1753-21gnome-shell20:46:200
841199615,15cyclictest1753-21gnome-shell22:20:393
8406996137,8cyclictest34-21ksoftirqd/222:10:152
8401996151,7cyclictest0-21swapper/122:20:081
8401996151,7cyclictest0-21swapper/121:21:431
8401996147,9cyclictest34790-21processes20:20:271
840199611,56cyclictest1753-21gnome-shell20:27:441
8393996129,6cyclictest141rcu_preempt22:18:250
839399612,55cyclictest1753-21gnome-shell22:45:190
8393996028,6cyclictest141rcu_preempt20:53:370
839399602,53cyclictest88628-21memory22:55:230
8401995939,17cyclictest1753-21gnome-shell22:27:401
840199592,15cyclictest1753-21gnome-shell22:10:011
840199591,36cyclictest26-21ksoftirqd/121:25:121
8393995925,19cyclictest18979411snapd20:14:500
8393995925,19cyclictest18979411snapd20:14:500
840199582,52cyclictest84820-21expr22:45:151
840199582,52cyclictest1753-21gnome-shell21:19:221
162399580,9rtkit-daemon1622-21rtkit-daemon00:37:402
162399580,9rtkit-daemon1622-21rtkit-daemon00:12:203
840199572,13cyclictest41819-21sed20:40:311
8393995745,8cyclictest13-21ksoftirqd/023:45:010
8393995745,8cyclictest13-21ksoftirqd/023:45:000
162399570,8rtkit-daemon1622-21rtkit-daemon23:55:403
162399570,8rtkit-daemon1622-21rtkit-daemon23:11:543
162399570,8rtkit-daemon1622-21rtkit-daemon19:21:232
8411995530,9cyclictest42-21ksoftirqd/323:45:043
8411995530,9cyclictest42-21ksoftirqd/323:45:033
162399540,8rtkit-daemon1622-21rtkit-daemon23:24:583
162399540,8rtkit-daemon1622-21rtkit-daemon23:24:573
841199535,16cyclictest2197014-21PK-Backend00:00:183
8411995316,18cyclictest42-21ksoftirqd/300:10:013
162399530,8rtkit-daemon1622-21rtkit-daemon19:29:342
841199514,13cyclictest42-21ksoftirqd/300:31:563
109072510,7chrt2197014-21PK-Backend19:15:142
840699501,9cyclictest28750irq/122-ahci[0020:14:512
840699501,9cyclictest28750irq/122-ahci[0020:14:502
8411994913,20cyclictest42-21ksoftirqd/300:25:003
8411994913,20cyclictest42-21ksoftirqd/300:25:003
841199488,11cyclictest42-21ksoftirqd/319:50:023
841199486,27cyclictest42-21ksoftirqd/319:35:123
841199485,19cyclictest42-21ksoftirqd/323:05:133
8411994825,8cyclictest42-21ksoftirqd/319:43:543
8411994815,18cyclictest42-21ksoftirqd/323:35:003
8411994811,20cyclictest42-21ksoftirqd/319:30:303
162399480,40rtkit-daemon1753-21gnome-shell23:17:072
8406994710,5cyclictest141rcu_preempt20:00:022
841199461,14cyclictest25036-21/usr/sbin/munin19:55:143
162399460,38rtkit-daemon1753-21gnome-shell00:05:412
906402450,7sleep02197014-21PK-Backend23:00:280
841199451,22cyclictest42-21ksoftirqd/323:15:203
8411994512,10cyclictest42-21ksoftirqd/300:30:013
8411994510,19cyclictest42-21ksoftirqd/323:05:023
841199446,15cyclictest42-21ksoftirqd/300:35:033
841199442,20cyclictest42-21ksoftirqd/319:10:303
8406994424,15cyclictest34-21ksoftirqd/200:04:592
8393994428,12cyclictest103352-21cron23:40:000
1145912440,5sleep02197014-21PK-Backend00:10:150
841199439,20cyclictest141rcu_preempt00:15:213
841199439,19cyclictest42-21ksoftirqd/319:50:033
8411994324,14cyclictest11185-21df_inode19:15:163
162399430,35rtkit-daemon860997-21snmpd23:36:353
841199427,18cyclictest42-21ksoftirqd/319:25:003
841199426,20cyclictest42-21ksoftirqd/323:48:533
8406994232,6cyclictest34-21ksoftirqd/223:45:032
8406994232,6cyclictest34-21ksoftirqd/223:45:032
8406994226,11cyclictest438-20systemd-journal00:31:332
8406994225,13cyclictest34-21ksoftirqd/219:10:272
8406994220,13cyclictest34-21ksoftirqd/223:50:532
8406994215,6cyclictest141rcu_preempt23:25:142
8406994213,6cyclictest141rcu_preempt19:40:302
8401994230,8cyclictest1753-21gnome-shell19:40:141
8401994225,13cyclictest1787-21gdbus23:45:021
8401994225,13cyclictest1787-21gdbus23:45:021
8393994213,6cyclictest141rcu_preempt00:00:210
1104522420,19sleep234-21ksoftirqd/200:00:032
841199413,7cyclictest42-21ksoftirqd/323:25:133
8411994118,7cyclictest42-21ksoftirqd/319:25:293
840699417,11cyclictest1617-10pipewire-media-23:00:442
8406994123,14cyclictest34-21ksoftirqd/219:50:032
840199419,6cyclictest141rcu_preempt23:45:311
840199419,6cyclictest141rcu_preempt00:05:261
8401994113,22cyclictest110728-40systemctl00:00:041
841199403,16cyclictest2197014-21PK-Backend20:00:113
8406994027,8cyclictest121098-21sh00:30:012
8406994019,12cyclictest34-21ksoftirqd/200:15:162
8406994010,5cyclictest141rcu_preempt19:35:312
840199406,6cyclictest141rcu_preempt19:35:311
8401994025,11cyclictest123229-21chmod00:35:031
840199401,8cyclictest26-21ksoftirqd/100:15:001
8401994016,10cyclictest121628-21expr00:30:181
8401994010,6cyclictest141rcu_preempt19:55:301
841199397,16cyclictest42-21ksoftirqd/323:51:533
840699392,11cyclictest1753-21gnome-shell23:35:242
8406993915,6cyclictest141rcu_preempt19:35:002
840199396,10cyclictest26-21ksoftirqd/119:53:241
8393993921,13cyclictest25718-21ntpq19:55:270
8393993912,6cyclictest141rcu_preempt19:15:020
839399391,13cyclictest141rcu_preempt23:06:020
840699387,6cyclictest141rcu_preempt23:45:142
8406993826,8cyclictest1069-21runrttasks23:20:132
8406993826,8cyclictest1069-21runrttasks23:20:132
8406993822,11cyclictest22665-21cron19:50:012
8406993814,6cyclictest141rcu_preempt23:30:282
8406993812,6cyclictest141rcu_preempt23:10:272
840199382,32cyclictest1753-21gnome-shell23:33:271
839399389,5cyclictest141rcu_preempt23:45:280
839399388,6cyclictest141rcu_preempt00:00:020
8393993828,6cyclictest161rcuc/019:20:300
8393993820,13cyclictest438-20systemd-journal23:28:440
8406993720,13cyclictest34-21ksoftirqd/200:23:452
8406993720,13cyclictest34-21ksoftirqd/200:23:452
8406993712,6cyclictest141rcu_preempt20:05:232
8401993726,7cyclictest8746-21tr19:10:121
8401993722,11cyclictest28867-21sort20:05:191
8401993713,5cyclictest141rcu_preempt23:05:141
839399375,5cyclictest141rcu_preempt19:50:200
839399375,10cyclictest13-21ksoftirqd/023:24:440
839399375,10cyclictest13-21ksoftirqd/023:24:430
839399373,5cyclictest141rcu_preempt00:07:250
839399372,31cyclictest1753-21gnome-shell00:23:550
839399372,31cyclictest1753-21gnome-shell00:23:540
8393993723,10cyclictest1753-21gnome-shell23:15:060
8393993722,11cyclictest117252-21sshd00:15:440
8393993721,12cyclictest855-21dbus-daemon20:05:010
8393993721,11cyclictest438-20systemd-journal20:09:550
8393993719,13cyclictest13-21ksoftirqd/023:50:150
8393993718,14cyclictest438-20systemd-journal19:35:000
839399371,33cyclictest100146-21/usr/sbin/munin23:30:300
8406993612,6cyclictest141rcu_preempt20:00:132
8401993623,9cyclictest110464-21sed00:00:031
8401993618,14cyclictest116789-21/usr/sbin/munin00:15:251
8401993614,20cyclictest64850irq/126-i91500:28:411
839399367,4cyclictest141rcu_preempt00:38:330
839399365,6cyclictest141rcu_preempt19:35:120
839399365,6cyclictest141rcu_preempt19:25:120
839399365,11cyclictest13-21ksoftirqd/000:29:470
8393993623,9cyclictest10483-21timerandwakeup19:15:030
8401993523,8cyclictest118092-21ls00:20:141
8401993523,8cyclictest118092-21ls00:20:141
8401993522,9cyclictest11805-21grep19:15:291
8401993521,10cyclictest99702-21sshd23:27:441
8401993521,10cyclictest28304-21ls20:05:021
8401993520,11cyclictest14404-21sort19:25:141
8401993518,13cyclictest12983-21if_enp2s019:20:201
8401993516,15cyclictest93598-21df23:10:161
839399351,31cyclictest0-21swapper/019:40:490
8401993421,8cyclictest97088-21entropy23:20:191
8401993421,8cyclictest97088-21entropy23:20:191
8401993420,10cyclictest90499-21ntp_kernel_pll_23:00:241
8401993417,13cyclictest107724-21load23:50:271
8401993320,9cyclictest16545-21sort19:30:221
8401993319,10cyclictest102147-21cat23:35:151
839399336,7cyclictest141rcu_preempt19:47:230
839399331,4cyclictest141rcu_preempt23:15:010
8401993220,8cyclictest96552-21cat23:20:031
839399321,15cyclictest2197014-21PK-Backend00:31:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional