You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 08:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Fri Nov 22, 2024 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2632021470,2sleep11795899cyclictest00:25:091
16921212095,18sleep00-21swapper/019:05:170
17575211896,17sleep30-21swapper/319:05:283
17817211598,12sleep10-21swapper/119:08:341
17631211493,16sleep20-21swapper/219:06:112
209812820,0sleep20-21swapper/222:03:372
47342730,1sleep10-21swapper/121:12:001
127582720,2sleep231-21ksoftirqd/221:20:202
167722680,0sleep10-21swapper/121:59:341
75202310,1sleep0101ktimersoftd/022:55:490
78892290,1sleep30-21swapper/321:15:203
1795799260,24cyclictest0-21swapper/021:20:120
1795799230,3cyclictest29837-21rm21:39:160
1795799230,3cyclictest29837-21rm21:39:160
113492230,0sleep1221ktimersoftd/121:53:341
201782220,1sleep10-21swapper/123:10:091
1795799221,4cyclictest0-21swapper/020:15:010
1795799221,19cyclictest0-21swapper/020:35:100
1795799220,3cyclictest0-21swapper/021:56:000
264392210,1sleep10-21swapper/120:50:131
1796099210,19cyclictest0-21swapper/323:00:103
1795999210,4cyclictest13379-21kworker/2:200:28:032
1795999210,19cyclictest0-21swapper/220:45:102
1795799210,3cyclictest21231-21aten2.4-expect21:30:160
1795799210,3cyclictest1030-21mailstats21:05:240
1795799210,19cyclictest0-21swapper/019:30:110
13132210,1sleep00-21swapper/022:16:060
1796099200,18cyclictest0-21swapper/319:45:123
17958992015,4cyclictest12187-21kworker/u8:022:34:041
1795899200,2cyclictest0-21swapper/120:00:131
1795799208,10cyclictest2857-21ssh00:34:290
1795799200,2cyclictest0-21swapper/022:10:130
17958991915,2cyclictest0-21swapper/100:00:101
1795799194,3cyclictest277-21jbd2/sda1-820:19:590
1795799191,16cyclictest0-21swapper/023:58:450
17957991910,7cyclictest1727-21diskmemload00:09:520
1795799190,17cyclictest0-21swapper/000:15:250
1795799190,11cyclictest0-21swapper/023:49:430
1795999181,2cyclictest0-21swapper/223:12:032
17958991815,2cyclictest29613-21kworker/u8:019:22:051
1795899180,3cyclictest0-21swapper/122:41:051
1795799185,3cyclictest0-21swapper/019:10:180
1795799180,3cyclictest0-21swapper/023:05:140
1795799180,2cyclictest0-21swapper/000:00:590
1795799180,16cyclictest0-21swapper/021:10:130
17959991714,2cyclictest32556-21kworker/u8:021:17:042
17959991714,2cyclictest27650-21kworker/u8:223:26:052
1795999171,2cyclictest0-21swapper/200:15:042
1795999170,2cyclictest0-21swapper/221:13:042
17958991714,2cyclictest6865-21kworker/u8:122:22:031
17958991714,2cyclictest3727-21kworker/u8:100:38:031
17958991714,2cyclictest24141-21kworker/u8:121:44:051
17958991714,2cyclictest14332-21kworker/u8:123:20:031
1795899170,15cyclictest0-21swapper/122:29:051
1795899170,15cyclictest0-21swapper/121:26:021
1795799176,2cyclictest0-21swapper/023:31:030
1795799176,2cyclictest0-21swapper/022:33:420
1795799174,3cyclictest26297-21sort23:50:230
1795799174,3cyclictest22457-21cpuspeed_turbos22:05:150
1795799173,3cyclictest2642-21wc21:45:010
1795799173,3cyclictest23120-21sh22:39:490
1795799171,6cyclictest21269-21cat23:45:010
1795799171,6cyclictest0-21swapper/019:18:010
1795799171,4cyclictest26608-21cpuspeed_turbos00:25:130
1795799171,3cyclictest20842-21munin-run21:30:000
1795799170,6cyclictest0-21swapper/021:50:190
1795799170,3cyclictest28788-21fschecks_time19:35:130
1795799170,3cyclictest23935-21ssh23:13:550
1795799170,2cyclictest0-21swapper/023:22:380
17959991613,2cyclictest29629-21kworker/u8:022:05:042
17959991613,2cyclictest22815-21kworker/u8:022:44:042
17959991611,3cyclictest0-21swapper/222:21:422
1795999160,2cyclictest0-21swapper/222:19:032
1795999160,2cyclictest0-21swapper/200:09:042
1795999160,14cyclictest0-21swapper/220:05:192
17958991613,2cyclictest32556-21kworker/u8:021:18:041
17958991613,2cyclictest32556-21kworker/u8:021:06:041
17958991613,2cyclictest31809-21kworker/u8:200:10:041
17958991613,2cyclictest28506-21kworker/u8:222:55:041
17958991613,2cyclictest24141-21kworker/u8:121:39:041
17958991613,2cyclictest24141-21kworker/u8:121:39:041
17958991613,2cyclictest20278-21kworker/u8:220:27:041
17958991613,2cyclictest20278-21kworker/u8:219:28:041
17958991613,2cyclictest20278-21kworker/u8:219:18:031
17958991613,2cyclictest14332-21kworker/u8:123:18:051
17958991613,2cyclictest14332-21kworker/u8:123:04:031
17958991613,2cyclictest10150-21kworker/u8:020:22:051
1795899161,13cyclictest3104-21ssh22:18:511
1795899160,12cyclictest0-21swapper/120:06:051
1795799165,3cyclictest6865-21kworker/u8:122:40:130
1795799164,3cyclictest3745-21runrttasks22:50:320
1795799164,3cyclictest3745-21runrttasks20:44:590
1795799164,3cyclictest11982-21basename00:10:170
1795799163,4cyclictest3011-21nscd21:50:000
1795799163,3cyclictest4933-21ntpdc19:55:170
1795799163,3cyclictest32359-21sed21:05:000
1795799163,3cyclictest20666-21chrt22:03:060
1795799163,3cyclictest10543-21sh23:00:090
1795799161,5cyclictest0-21swapper/022:45:130
1795799161,4cyclictest21504-21munin-node0
1795799161,3cyclictest7226-21cpuspeed_turbos21:15:130
1795799161,3cyclictest30128-21kworker/u8:200:35:150
1795799161,3cyclictest27678-21tail20:50:260
1795799161,3cyclictest24988-21aten2.4-expect23:15:130
1795799161,3cyclictest2234-21head23:25:110
1795799161,3cyclictest0-21swapper/022:25:140
1795799161,3cyclictest0-21swapper/022:20:130
1795799160,8cyclictest0-21swapper/020:25:120
1795799160,7cyclictest0-21swapper/019:40:010
1796099151,12cyclictest7853-21iostat_ios00:05:183
1796099151,12cyclictest7208-21python20:00:253
1796099151,12cyclictest25281-21df23:50:113
1796099151,12cyclictest16465-21ssh21:59:053
1796099151,12cyclictest0-21swapper/322:19:323
1796099151,12cyclictest0-21swapper/320:15:073
1796099150,2cyclictest1727-21diskmemload23:46:203
1796099150,2cyclictest0-21swapper/323:11:413
1796099150,13cyclictest762-21runrttasks23:23:243
1796099150,13cyclictest0-21swapper/321:05:123
1796099150,12cyclictest0-21swapper/300:11:043
17959991512,2cyclictest4388-21kworker/u8:000:03:032
17959991512,2cyclictest1727-21diskmemload23:55:272
17959991512,2cyclictest11359-21kworker/u8:123:53:042
17959991512,2cyclictest11359-21kworker/u8:123:48:022
17959991511,2cyclictest18120-21ssh21:25:382
1795999150,13cyclictest30684-21ssh22:13:482
1795999150,13cyclictest12173-21sh20:15:042
1795999150,13cyclictest0-21swapper/222:34:582
1795999150,10cyclictest0-21swapper/223:40:052
17958991512,2cyclictest6865-21kworker/u8:122:39:041
17958991512,2cyclictest20278-21kworker/u8:219:55:051
17958991512,2cyclictest20278-21kworker/u8:219:39:031
17958991512,2cyclictest17657-21kworker/u8:119:11:041
17958991512,2cyclictest14332-21kworker/u8:123:08:041
17958991512,2cyclictest13300-21kworker/u8:221:21:051
17958991510,3cyclictest0-21swapper/123:52:091
1795899150,2cyclictest0-21swapper/121:33:041
1795799153,3cyclictest24460-21aten2.4_r3power20:45:120
1795799152,3cyclictest22328-21aten2.4_r3power19:20:130
1795799151,4cyclictest24438-21grep19:25:120
1795799151,3cyclictest6412-21sed20:00:150
1795799151,3cyclictest4368-21gnome-session20:20:050
1795799151,3cyclictest3745-21runrttasks19:45:140
1795799151,3cyclictest0-21swapper/023:35:140
1795799151,3cyclictest0-21swapper/020:55:090
1795799151,3cyclictest0-21swapper/020:30:110
1795799151,3cyclictest0-21swapper/020:05:250
1795799150,14cyclictest0-21swapper/019:50:120
285672140,2sleep30-21swapper/300:26:293
1796099141,2cyclictest22074-21apt-get20:40:063
17960991411,2cyclictest0-21swapper/322:20:443
17960991411,2cyclictest0-21swapper/322:03:323
17960991411,2cyclictest0-21swapper/321:23:213
17960991410,2cyclictest0-21swapper/323:35:333
1796099140,2cyclictest0-21swapper/323:43:403
1796099140,2cyclictest0-21swapper/323:30:203
1796099140,2cyclictest0-21swapper/323:07:213
1796099140,2cyclictest0-21swapper/322:50:023
1796099140,2cyclictest0-21swapper/321:38:123
1796099140,2cyclictest0-21swapper/321:38:113
1796099140,2cyclictest0-21swapper/320:35:113
1796099140,13cyclictest0-21swapper/322:25:123
1796099140,13cyclictest0-21swapper/321:12:513
1796099140,12cyclictest1727-21diskmemload23:18:243
1796099140,12cyclictest0-21swapper/322:30:043
1796099140,12cyclictest0-21swapper/322:05:013
1796099140,12cyclictest0-21swapper/321:31:573
1796099140,12cyclictest0-21swapper/321:25:123
1796099140,12cyclictest0-21swapper/300:36:103
17959991411,2cyclictest19663-21kworker/u8:100:22:042
1795999140,2cyclictest0-21swapper/219:20:112
1795999140,13cyclictest0-21swapper/223:38:482
1795999140,13cyclictest0-21swapper/200:14:472
1795999140,12cyclictest7893-21apt-get21:50:142
1795999140,12cyclictest0-21swapper/221:35:332
1795999140,12cyclictest0-21swapper/221:35:322
1795999140,12cyclictest0-21swapper/200:37:492
17958991411,2cyclictest29629-21kworker/u8:022:09:031
17958991411,2cyclictest28641-21kworker/u8:122:12:031
17958991411,2cyclictest28506-21kworker/u8:222:53:041
17958991411,2cyclictest14143-21kworker/u8:000:21:031
17958991411,2cyclictest11359-21kworker/u8:123:45:031
17958991411,2cyclictest10150-21kworker/u8:020:17:041
1795899140,2cyclictest26053-21copy19:30:051
1795899140,12cyclictest0-21swapper/122:46:541
1795899140,12cyclictest0-21swapper/100:19:041
254042131,8sleep325629-21users19:25:263
17960991310,2cyclictest0-21swapper/323:26:143
17960991310,2cyclictest0-21swapper/322:45:153
17960991310,2cyclictest0-21swapper/322:41:523
1796099130,2cyclictest8885-21open_inodes21:50:203
1796099130,2cyclictest27018-21munin-node3
1796099130,2cyclictest26053-21copy19:30:023
1796099130,2cyclictest0-21swapper/322:37:283
1796099130,2cyclictest0-21swapper/321:45:013
1796099130,2cyclictest0-21swapper/300:21:503
1796099130,12cyclictest0-21swapper/321:40:413
1796099130,12cyclictest0-21swapper/300:31:573
1796099130,11cyclictest0-21swapper/300:16:193
17959991310,2cyclictest17299-21ssh23:06:032
17959991310,2cyclictest10150-21kworker/u8:020:20:052
17959991310,2cyclictest0-21swapper/223:21:582
17959991310,2cyclictest0-21swapper/222:25:312
17959991310,2cyclictest0-21swapper/220:10:172
17959991310,2cyclictest0-21swapper/219:30:112
1795999130,2cyclictest0-21swapper/223:31:582
1795999130,2cyclictest0-21swapper/223:15:302
1795999130,2cyclictest0-21swapper/223:02:072
1795999130,2cyclictest0-21swapper/221:45:242
1795999130,2cyclictest0-21swapper/221:42:192
1795999130,2cyclictest0-21swapper/221:30:302
1795999130,12cyclictest0-21swapper/222:55:502
1795999130,12cyclictest0-21swapper/222:50:362
1795999130,12cyclictest0-21swapper/222:39:282
1795999130,12cyclictest0-21swapper/221:55:292
1795999130,12cyclictest0-21swapper/200:33:002
1795899139,2cyclictest0-21swapper/123:40:061
1795899133,8cyclictest21-21rcuc/123:30:171
17958991310,2cyclictest1727-21diskmemload00:32:401
1795899130,2cyclictest0-21swapper/121:47:501
1795899130,2cyclictest0-21swapper/100:05:041
1795899130,12cyclictest0-21swapper/123:57:441
1795899130,12cyclictest0-21swapper/122:04:181
1796099129,2cyclictest0-21swapper/321:00:163
1796099129,2cyclictest0-21swapper/320:50:163
1796099129,2cyclictest0-21swapper/319:15:143
1796099123,6cyclictest0-21swapper/320:45:223
1796099122,7cyclictest0-21swapper/319:38:583
1796099122,5cyclictest0-21swapper/320:10:003
1796099120,2cyclictest0-21swapper/322:55:513
1796099120,2cyclictest0-21swapper/320:10:193
1796099120,11cyclictest0-21swapper/323:55:373
1796099120,11cyclictest0-21swapper/319:20:123
1796099120,11cyclictest0-21swapper/300:01:053
1795999129,2cyclictest10150-21kworker/u8:020:40:042
1795999129,2cyclictest10150-21kworker/u8:020:35:042
1795999129,2cyclictest10150-21kworker/u8:020:31:052
1795999129,2cyclictest0-21swapper/220:25:222
1795999120,2cyclictest0-21swapper/222:46:052
1795999120,2cyclictest0-21swapper/219:10:172
1795999120,11cyclictest0-21swapper/220:55:102
1795999120,11cyclictest0-21swapper/219:55:172
1795999120,11cyclictest0-21swapper/219:50:002
1795999120,11cyclictest0-21swapper/219:25:112
1795899129,2cyclictest0-21swapper/123:37:101
1795899129,2cyclictest0-21swapper/120:35:091
1795899129,2cyclictest0-21swapper/119:50:181
1795899127,3cyclictest0-21swapper/120:10:161
1795899121,9cyclictest2097-21apt-get23:25:101
1795899120,3cyclictest0-21swapper/120:40:161
1796099118,2cyclictest0-21swapper/319:55:233
1796099110,10cyclictest0-21swapper/320:55:253
1796099110,10cyclictest0-21swapper/320:31:383
1795999118,2cyclictest17657-21kworker/u8:119:15:052
1795999118,2cyclictest0-21swapper/219:45:002
1795999112,6cyclictest0-21swapper/221:00:012
1795999112,6cyclictest0-21swapper/220:50:012
1795999110,2cyclictest0-21swapper/219:41:052
1795899118,2cyclictest0-21swapper/120:45:101
1795899111,8cyclictest30322-21munin-node1
1795899111,2cyclictest32332-21apt-get19:45:121
1795899110,9cyclictest0-21swapper/121:00:151
1795899110,9cyclictest0-21swapper/120:55:161
71062100,2sleep20-21swapper/220:00:222
1796099107,2cyclictest0-21swapper/319:40:043
1795999107,2cyclictest28264-21apt-get19:35:102
1795999105,3cyclictest782-21needreboot21:05:182
1795899106,2cyclictest0-21swapper/120:30:271
179609994,3cyclictest18051-21cron20:29:593
179609983,3cyclictest1948-21basename19:50:123
179609983,3cyclictest18928-21cat19:10:173
179609983,3cyclictest15566-21unixbench_multi20:20:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional