You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 15:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot7s.osadl.org (updated Sat Jul 27, 2024 12:44:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12335210379,8sleep30-21swapper/307:09:193
12066210287,10sleep10-21swapper/107:05:471
1223428057,18sleep00-21swapper/007:07:590
1226127658,14sleep20-21swapper/207:08:212
1246499299,3cyclictest29337-21tr12:00:170
1247899263,5cyclictest25799-21sh11:20:003
1247599230,21cyclictest0-21swapper/212:31:522
12475992222,0cyclictest0-21swapper/210:43:042
12475992222,0cyclictest0-21swapper/210:43:042
1247599220,20cyclictest0-21swapper/209:56:262
12475992113,4cyclictest224-21systemd-journal08:29:592
1247599210,21cyclictest0-21swapper/212:12:152
1246899210,1cyclictest0-21swapper/112:36:361
1247599206,13cyclictest0-21swapper/209:40:002
12468992018,1cyclictest25-21ksoftirqd/110:30:551
12468992018,1cyclictest0-21swapper/107:45:141
54642190,2chrt5462-21apache_accesses09:10:110
1247899191,0cyclictest0-21swapper/311:09:283
1247599193,10cyclictest0-21swapper/212:20:182
12475991918,1cyclictest0-21swapper/211:55:122
1246899191,11cyclictest0-21swapper/112:00:151
12475991817,1cyclictest0-21swapper/210:58:542
1247599180,1cyclictest0-21swapper/211:41:582
1247599180,1cyclictest0-21swapper/211:14:592
1247599180,16cyclictest0-21swapper/211:39:222
12468991818,0cyclictest0-21swapper/110:27:141
323942170,1sleep10-21swapper/111:42:321
12475991716,1cyclictest16532-21kworker/2:011:20:382
12475991716,1cyclictest10673-21kworker/2:209:45:592
12475991714,3cyclictest0-21swapper/211:25:152
12475991713,2cyclictest5022-21diskmemload10:08:302
1247599170,1cyclictest0-21swapper/209:44:022
1247599170,17cyclictest13151-21kworker/2:011:31:382
1247599170,17cyclictest0-21swapper/210:00:102
1246499174,9cyclictest12158-21apt-get09:15:140
12464991715,1cyclictest0-21swapper/011:26:050
1246899166,3cyclictest25-21ksoftirqd/109:40:381
12468991616,0cyclictest0-21swapper/111:50:111
12468991615,1cyclictest25-21ksoftirqd/109:23:391
12468991613,1cyclictest25-21ksoftirqd/111:01:201
1246899161,2cyclictest141rcu_preempt09:45:131
12468991612,3cyclictest12150-21apache_volume09:15:131
1246899160,15cyclictest0-21swapper/111:18:011
12478991514,1cyclictest0-21swapper/311:31:383
1247899151,13cyclictest0-21swapper/309:15:133
12475991514,1cyclictest21453-21kworker/2:010:35:192
12475991513,1cyclictest19912-21awk08:30:252
1247599150,2cyclictest0-21swapper/210:20:342
1247599150,15cyclictest0-21swapper/212:19:012
1247599150,14cyclictest14727-21cstates10:50:162
12468991513,1cyclictest241rcuc/110:35:241
1246499153,6cyclictest16633-21awk10:05:000
1247899148,6cyclictest12540-21apache212:18:293
12478991411,2cyclictest5907-21wc08:00:333
12478991411,2cyclictest39-21ksoftirqd/311:36:503
12478991410,3cyclictest29445-21cat09:50:173
12475991412,1cyclictest3639-21apt-get11:45:112
1247599141,13cyclictest32520-21kworker/2:209:14:292
12475991411,2cyclictest18546-21apt09:20:012
1246899143,4cyclictest25-21ksoftirqd/111:29:591
1246899143,2cyclictest25-21ksoftirqd/107:15:201
12468991412,2cyclictest25-21ksoftirqd/109:35:131
1246899140,1cyclictest141rcu_preempt11:47:161
1246499144,9cyclictest25834-21perf11:20:000
1246499142,6cyclictest1742-21munin-run11:25:000
12464991411,2cyclictest12540-21apache209:41:210
12464991410,3cyclictest17775-21tr07:20:130
12464991410,2cyclictest26374-21tr09:25:160
1246499140,2cyclictest0-21swapper/010:38:390
1247899139,3cyclictest4107-21sed09:05:263
1247899139,3cyclictest20892-21grep07:25:153
1247899139,2cyclictest21762-21grep08:35:203
1247899139,2cyclictest16193-21wc08:25:003
1247899139,2cyclictest12330-21tr11:50:163
1247899134,4cyclictest0-21swapper/310:30:153
12478991310,2cyclictest8034-21grep10:45:173
12478991310,2cyclictest30219-21tr12:20:133
12478991310,2cyclictest29550-21wc11:00:243
12478991310,2cyclictest26061-21chrt09:25:123
12478991310,2cyclictest26043-21grep08:45:153
12478991310,2cyclictest23704-21tr08:40:183
12478991310,2cyclictest23283-21cut12:35:193
12478991310,2cyclictest19451-21grep08:30:193
12478991310,2cyclictest17477-21cron11:14:593
12478991310,2cyclictest16150-21grep12:30:293
12478991310,2cyclictest15318-21head12:10:293
12478991310,2cyclictest13878-21wc12:30:013
12478991310,2cyclictest1276-21grep10:40:173
12478991310,2cyclictest1276-21grep10:40:163
12478991310,2cyclictest10199-21tr10:00:153
1247899130,12cyclictest23622-21grep09:45:303
1247599138,1cyclictest0-21swapper/212:04:022
1247599131,10cyclictest0-21swapper/211:15:122
12475991310,2cyclictest9330-21grep08:10:162
12475991310,2cyclictest26978-21sh07:40:012
12475991310,2cyclictest19784-21sort09:20:202
1247599130,12cyclictest24156-21grep08:40:212
1247599130,0cyclictest0-21swapper/210:45:422
1246899139,3cyclictest6196-21grep12:05:231
1246899136,6cyclictest17292-21cut10:05:131
12468991312,1cyclictest25-21ksoftirqd/109:56:381
12468991311,1cyclictest25-21ksoftirqd/112:30:061
12468991310,2cyclictest6322-21head09:10:181
12468991310,2cyclictest4127-21ls09:05:261
12468991310,2cyclictest3403-21cat07:55:281
12468991310,2cyclictest29799-21grep08:50:251
12468991310,2cyclictest26604-21sed08:45:191
1246899130,2cyclictest0-21swapper/109:50:121
1246899130,11cyclictest0-21swapper/109:33:351
1246499139,3cyclictest9889-21grep11:10:160
1246499139,3cyclictest25279-21ssh11:57:420
1246499139,2cyclictest11086-21grep11:30:140
1246499134,4cyclictest22027-21ssh10:08:210
12464991310,2cyclictest9883-21head08:10:190
12464991310,2cyclictest8893-21ssh09:35:210
12464991310,2cyclictest4515-21grep11:05:290
12464991310,2cyclictest31354-21sed08:55:230
12464991310,2cyclictest30296-21grep09:50:280
12464991310,2cyclictest22002-21cat12:15:150
12464991310,2cyclictest12958-21wc12:10:000
1246499130,12cyclictest15647-21tr12:30:250
248542121,2sleep224878-21ssh12:35:432
166132120,0sleep10-21swapper/110:50:431
1247899129,2cyclictest6968-21sed12:05:323
1247899129,2cyclictest28888-21grep07:40:283
1247899129,2cyclictest26914-21uniq10:35:183
1247899129,2cyclictest26630-21chrt10:58:543
1247899129,2cyclictest24955-21grep10:10:193
1247899129,2cyclictest19092-21grep10:05:253
1247899129,2cyclictest14445-21sed07:10:313
1247899129,2cyclictest11226-21awk08:15:003
1247899128,3cyclictest18045-21taskset10:52:063
1247899128,3cyclictest15412-21tr07:15:163
1247899128,3cyclictest11705-21tr08:15:163
1247899128,2cyclictest25588-21tail07:35:143
1247899127,4cyclictest5424-21munin-node09:10:113
1247899125,5cyclictest2689-21head07:55:163
1247899124,4cyclictest278-21systemd-udevd11:20:223
1247899121,2cyclictest0-21swapper/309:34:593
1247899120,11cyclictest19234-21head07:20:283
1247599129,2cyclictest32518-21cut10:15:212
1247599129,2cyclictest30798-21tr08:55:192
1247599129,2cyclictest30301-21sh08:55:002
1247599129,2cyclictest26912-21sed09:25:192
1247599129,2cyclictest18323-21sed07:20:162
1247599128,2cyclictest3994-21wc11:05:242
1247599128,1cyclictest12558-21cat11:50:162
1247599126,5cyclictest28054-21head07:40:172
1247599121,3cyclictest224-21systemd-journal09:10:012
1247599120,0cyclictest0-21swapper/212:08:282
1246899129,2cyclictest3874-21taskset11:05:221
1246899129,2cyclictest30450-21taskset11:22:051
1246899129,2cyclictest27011-21awk07:40:011
1246899129,2cyclictest18315-21grep07:20:151
1246899129,2cyclictest14604-21ssh10:03:031
1246899129,2cyclictest14411-21sed07:10:301
1246899127,1cyclictest141rcu_preempt12:27:561
1246899126,1cyclictest25-21ksoftirqd/110:12:501
1246899124,8cyclictest5359-21munin-node08:26:351
1246899123,3cyclictest20392-21apt-get11:55:111
12468991211,1cyclictest25-21ksoftirqd/108:35:001
1246899120,0cyclictest0-21swapper/112:15:531
1246499129,2cyclictest8562-21ssh12:26:030
1246499129,2cyclictest32515-21grep10:15:220
1246499129,2cyclictest3157-21ls11:44:590
1246499129,2cyclictest29883-21sed08:50:270
1246499129,2cyclictest29383-21apt07:45:000
1246499129,2cyclictest28182-21ssh09:49:590
1246499129,2cyclictest2236-21head09:00:310
1246499129,2cyclictest21324-21tr08:35:150
1246499129,2cyclictest18795-21chrt10:52:500
1246499129,2cyclictest13437-21cut07:10:190
1246499129,2cyclictest12123-21cut08:15:200
1246499128,3cyclictest790-21cut07:50:240
1246499128,3cyclictest19939-21wc08:30:250
1246499125,6cyclictest19366-21apache_processe10:30:120
1246499124,7cyclictest5359-21munin-node12:13:040
12464991211,1cyclictest12462-21cyclictest12:24:370
1246499120,11cyclictest27022-21perf11:40:000
1247899118,2cyclictest7179-21wc10:20:233
1247899118,2cyclictest4999-21ssh09:55:333
1247899118,2cyclictest31062-21ssh10:15:013
1247899118,2cyclictest27495-21ssh11:40:023
1247899118,2cyclictest2319-21cut11:25:133
1247899118,2cyclictest23044-21sed07:30:143
1247899118,2cyclictest22972-21ssh11:55:543
1247899118,2cyclictest1820-21sh12:03:003
1247899118,2cyclictest17019-21ssh10:27:563
1247899118,2cyclictest16659-21sed09:40:283
1247899118,2cyclictest16410-21cut08:25:173
1247899116,3cyclictest2634-21cron09:05:013
1247899110,9cyclictest28419-21cpuspeed_turbos08:50:123
1247899110,1cyclictest0-21swapper/311:45:443
1247899110,10cyclictest23928-21ssh09:23:203
1247599118,2cyclictest31643-21chrt07:48:522
1247599118,2cyclictest29295-21sort11:00:222
1247599118,2cyclictest2705-21rm09:54:592
1247599118,2cyclictest26517-21perl08:45:192
1247599118,2cyclictest24403-21taskset07:32:332
1247599118,2cyclictest23152-21cron08:40:002
1247599118,2cyclictest20771-21grep07:25:162
1247599118,2cyclictest20312-21basename10:30:192
1247599118,2cyclictest15488-21expr07:15:162
1247599118,2cyclictest13867-21grep07:10:222
1247599118,2cyclictest1314-21sed09:00:182
1247599116,4cyclictest24377-21grep10:10:162
1247599113,4cyclictest224-21systemd-journal10:30:002
1247599110,10cyclictest6237-21ls12:25:162
1246899119,1cyclictest26807-21grep09:25:191
1246899119,1cyclictest23213-21head07:30:151
1246899119,1cyclictest22103-21ssh10:55:191
1246899118,2cyclictest9751-21grep08:10:181
1246899118,2cyclictest8303-21head10:45:201
1246899118,2cyclictest6563-21head10:20:171
1246899118,2cyclictest623-21cut10:40:131
1246899118,2cyclictest623-21cut10:40:121
1246899118,2cyclictest4934-21ssh11:25:591
1246899118,2cyclictest31985-21cat07:50:121
1246899118,2cyclictest31375-21aten2_r7power_p10:15:141
1246899118,2cyclictest21890-21ssh11:35:591
1246899118,2cyclictest21841-21head08:35:201
1246899118,2cyclictest12024-21df08:15:171
1246899115,2cyclictest224-21systemd-journal09:05:001
1246899115,1cyclictest141rcu_preempt12:22:391
1246899110,4cyclictest224-21systemd-journal12:15:011
1246499118,2cyclictest8509-21taskset08:05:570
1246499118,2cyclictest7726-21expr10:45:170
1246499118,2cyclictest6581-21head10:20:170
1246499118,2cyclictest6412-21sshd08:04:320
1246499118,2cyclictest5812-21ssh09:56:210
1246499118,2cyclictest4070-21dump-pmu-power08:00:000
1246499118,2cyclictest3375-21expr09:05:160
1246499118,2cyclictest2970-21ssh09:30:440
1246499118,2cyclictest29069-21ssh11:00:190
1246499118,2cyclictest24793-21wc07:35:000
1246499118,2cyclictest23717-21cut08:40:190
1246499118,2cyclictest23567-21ssh09:22:590
1246499118,2cyclictest23408-21head12:35:190
1246499118,2cyclictest16504-21ssh10:27:250
1246499118,2cyclictest15878-21ssh11:51:500
1246499117,3cyclictest20879-21sed07:25:160
1246499117,3cyclictest15375-21grep07:15:160
1246499114,6cyclictest24163-21apt-get10:10:140
1246499113,8cyclictest28731-21kworker/u8:010:40:590
1246499113,8cyclictest28731-21kworker/u8:010:40:580
1247899108,1cyclictest29647-21aten2_r7power_c07:45:163
1247899107,2cyclictest8525-21ssh09:35:183
1247899107,2cyclictest7506-21cut08:05:193
1247899107,2cyclictest1111-21sed07:50:283
1247599108,1cyclictest7631-21cat08:05:212
1247599108,1cyclictest6281-21chrt08:02:492
1247599108,1cyclictest32381-21cat07:50:152
1247599108,1cyclictest1932-21basename07:55:092
1247599108,1cyclictest15767-21taskset08:21:342
1247599108,1cyclictest1248-21expr09:30:152
1247599108,1cyclictest12231-21tail08:15:202
1246899108,1cyclictest7139-21expr08:05:151
1246899108,1cyclictest5141-21head08:00:191
1246899108,1cyclictest29280-21taskset07:43:591
1246899108,1cyclictest16147-21date08:25:001
1246899107,2cyclictest10743-21sed11:10:231
1246899102,2cyclictest141rcu_preempt08:40:301
1246499108,1cyclictest277392sleep008:47:440
1246499108,1cyclictest25726-21cat07:35:170
1246499107,2cyclictest3777-21ssh11:45:130
1246499107,2cyclictest30671-21wc07:45:230
1246499107,2cyclictest21215-21chrt10:55:050
1246499107,2cyclictest16964-21sort08:25:200
1246499106,1cyclictest13942-21apt-get08:20:130
124789996,2cyclictest30846-21tr08:55:193
124689997,1cyclictest21000-21taskset07:25:181
124689996,2cyclictest31008-21expr08:55:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional