You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-25 - 14:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon Nov 25, 2024 00:43:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26269210389,9sleep30-21swapper/319:07:043
26185210188,9sleep10-21swapper/119:05:581
2629129166,9sleep00-21swapper/019:07:220
2619127868,6sleep20-21swapper/219:06:032
243222570,0sleep00-21swapper/021:12:110
104042450,0sleep10-21swapper/121:50:031
26587993330,2cyclictest94-21kswapd021:45:112
2658799241,2cyclictest8381-21cut20:45:142
2658999231,1cyclictest141rcu_preempt00:20:233
2658799230,22cyclictest0-21swapper/222:41:332
2658799220,18cyclictest0-21swapper/221:42:262
2658699210,1cyclictest0-21swapper/121:47:511
2658599212,15cyclictest0-21swapper/022:40:130
26589992019,1cyclictest0-21swapper/322:22:043
26589991918,1cyclictest0-21swapper/321:34:053
2658799190,19cyclictest0-21swapper/221:57:372
2658799190,18cyclictest0-21swapper/223:31:142
2658799180,1cyclictest0-21swapper/221:52:242
2658799180,15cyclictest0-21swapper/200:22:322
2658599184,7cyclictest20598-21munin-run20:05:000
2658599183,6cyclictest224-21systemd-journal00:19:590
2658799170,17cyclictest0-21swapper/222:48:202
2658699172,9cyclictest918-21apt-get22:55:141
2658999163,2cyclictest2858-21ssh23:41:263
26589991615,1cyclictest0-21swapper/321:13:183
26589991612,3cyclictest12976-21ssh23:50:163
2658999160,14cyclictest0-21swapper/322:40:193
26587991610,5cyclictest2877-21dump-pmu-power20:35:012
2658799160,1cyclictest0-21swapper/223:28:462
2658699161,6cyclictest224-21systemd-journal22:25:011
2658699161,2cyclictest0-21swapper/122:50:171
2658599168,6cyclictest7744-21snmpd21:27:380
195382160,0chrt0-21swapper/000:20:010
26589991514,1cyclictest0-21swapper/322:47:493
2658799154,7cyclictest311rcuc/200:00:012
2658799150,3cyclictest26176-21apt-get00:25:122
2658799150,14cyclictest20031-21diskmemload21:11:352
26586991513,1cyclictest25-21ksoftirqd/123:43:261
26585991512,2cyclictest13-21ksoftirqd/021:37:200
26585991511,3cyclictest22838-21tune2fs23:10:210
2658999142,2cyclictest0-21swapper/323:11:583
26589991411,2cyclictest6516-21munin-plugin-st00:10:013
26589991411,0cyclictest0-21swapper/321:57:063
26587991413,1cyclictest0-21swapper/221:24:592
26587991410,3cyclictest31117-21grep23:15:262
2658799140,2cyclictest0-21swapper/221:15:102
2658799140,14cyclictest0-21swapper/222:07:022
2658799140,14cyclictest0-21swapper/221:27:112
2658799140,13cyclictest0-21swapper/223:01:492
2658699149,1cyclictest20031-21diskmemload22:10:041
26586991411,2cyclictest27042-21ls20:15:231
26586991411,2cyclictest27042-21ls20:15:221
26586991410,3cyclictest25-21ksoftirqd/121:10:141
2658599145,8cyclictest30530-21munin-run22:05:000
26585991410,3cyclictest30014-21grep23:15:170
2658999139,3cyclictest5252-21tr19:30:223
2658999138,3cyclictest18750-21tr20:00:153
26589991312,1cyclictest39-21ksoftirqd/322:06:233
26589991310,2cyclictest5665-21ls20:40:013
26589991310,2cyclictest32631-21tail22:30:213
26589991310,2cyclictest23476-21tr20:10:143
26589991310,2cyclictest13688-21grep00:15:163
26589991310,2cyclictest12969-21grep23:25:273
26589991310,2cyclictest10257-21cat20:50:003
2658999130,12cyclictest19121-21tail23:30:223
2658999130,12cyclictest11624-21ssh00:39:173
2658799139,3cyclictest29778-21chrt22:51:472
2658799133,2cyclictest224-21systemd-journal20:29:382
26587991310,2cyclictest6503-21cron00:10:012
26587991310,2cyclictest31644-21cron23:40:002
26587991310,2cyclictest24915-21grep22:00:172
26587991310,2cyclictest16422-21grep21:30:202
26587991310,2cyclictest1034-21cut22:55:142
2658799130,1cyclictest0-21swapper/200:30:282
2658799130,12cyclictest6455-21munin-plugin-st23:45:002
2658799130,12cyclictest18669-21sleep23:55:012
2658699139,3cyclictest8741-21wc19:40:001
2658699139,3cyclictest6163-21ssh00:09:381
2658699139,3cyclictest11809-21tail23:25:191
2658699135,4cyclictest224-21systemd-journal00:29:591
2658699133,6cyclictest25337-21apt-get20:15:001
26586991310,2cyclictest26827-21chrt00:00:181
26586991310,2cyclictest18180-21cron20:00:001
2658699130,12cyclictest26209-21chrt22:00:291
2658699130,12cyclictest16694-21grep23:05:261
2658599139,3cyclictest1830-21grep23:40:250
2658599132,8cyclictest24715-21ls22:25:010
2658599131,6cyclictest7871-21awk20:45:000
26585991310,2cyclictest8150-21grep19:35:260
26585991310,2cyclictest6483-21ssh00:35:120
26585991310,2cyclictest27305-21tr19:10:160
26585991310,2cyclictest26527-21grep20:15:180
26585991310,2cyclictest26527-21grep20:15:180
26585991310,2cyclictest18283-21grep21:55:170
2658599130,2cyclictest0-21swapper/019:20:130
2658599130,1cyclictest0-21swapper/023:26:470
2658599130,1cyclictest0-21swapper/023:01:340
2658999129,2cyclictest7299-21cron23:00:003
2658999129,2cyclictest4777-21grep21:45:163
2658999129,2cyclictest29980-21tail23:15:173
2658999129,2cyclictest26569-21grep20:15:183
2658999129,2cyclictest26569-21grep20:15:183
2658999129,2cyclictest23695-21head21:35:163
2658999129,2cyclictest22454-21grep20:05:263
2658999128,3cyclictest6828-21grep22:35:173
2658999128,3cyclictest3932-21wc19:29:593
2658999128,3cyclictest28870-21grep20:20:163
2658999128,3cyclictest28020-21ssh21:15:133
2658999128,3cyclictest13863-21grep20:55:153
2658999127,3cyclictest5500-21tr22:10:193
2658999124,4cyclictest224-21systemd-journal20:55:003
26589991210,2cyclictest32159-21ssh00:04:533
2658799129,2cyclictest4062-21grep20:35:202
2658799129,2cyclictest28317-21basename20:20:122
2658799129,2cyclictest23638-21grep21:35:172
2658799129,2cyclictest22140-21ssh23:10:172
2658799129,2cyclictest22085-21grep20:05:212
2658799129,2cyclictest16796-21head19:55:192
2658799129,2cyclictest15415-21wc21:00:002
2658799129,2cyclictest13695-21tr00:15:162
2658799128,3cyclictest30053-21grep19:15:182
2658799127,4cyclictest14852-21ntpq19:50:242
2658799127,4cyclictest11847-21ntpq20:50:212
2658799120,12cyclictest0-21swapper/200:13:432
2658699129,2cyclictest9370-21wc20:45:221
2658699129,2cyclictest7889-21tr00:35:241
2658699129,2cyclictest7429-21expr00:10:161
2658699129,2cyclictest32197-21cut22:05:221
2658699129,2cyclictest29498-21tr23:15:141
2658699129,2cyclictest27782-21grep19:10:191
2658699129,2cyclictest26725-21wc22:25:261
2658699129,2cyclictest22941-21grep23:10:211
2658699129,2cyclictest21195-21wc22:45:221
2658699129,2cyclictest20134-21sed23:30:311
2658699129,2cyclictest19935-21cut23:55:171
2658699129,2cyclictest17559-21tr21:55:141
2658699129,2cyclictest16027-21tr21:30:181
2658699129,2cyclictest15821-21grep19:55:001
2658699129,2cyclictest1219-21grep22:30:311
2658699129,2cyclictest10582-21grep19:40:281
2658699128,3cyclictest17203-21grep21:00:251
2658699128,1cyclictest25-21ksoftirqd/121:18:341
2658599129,2cyclictest6246-21cut22:35:140
2658599129,2cyclictest5498-21cron22:34:590
2658599129,2cyclictest4703-21cut21:45:160
2658599129,2cyclictest3547-21sh21:45:000
2658599129,2cyclictest32426-21grep20:25:270
2658599129,2cyclictest25384-21tr22:25:150
2658599129,2cyclictest25194-21chrt23:35:130
2658599129,2cyclictest2142-21tr21:20:160
2658599129,2cyclictest19898-21grep22:45:140
2658599129,2cyclictest1864-21grep20:30:230
2658599129,2cyclictest1500-21ssh00:05:190
2658599129,2cyclictest14075-21tr20:55:190
2658599129,2cyclictest10774-21grep21:50:190
2658599128,3cyclictest12988-21grep19:45:270
2658599122,5cyclictest20031-21diskmemload00:00:010
2658599121,5cyclictest11419-21perf22:15:010
2658999119,1cyclictest18193-21tr21:05:143
2658999118,2cyclictest9165-21cat21:25:133
2658999118,2cyclictest8396-21taskset19:37:243
2658999118,2cyclictest7100-21aten2_r7power_v00:10:163
2658999118,2cyclictest6437-21ssh21:23:063
2658999118,2cyclictest580-21ls00:30:193
2658999118,2cyclictest3753-21ssh23:20:083
2658999118,2cyclictest31357-21ssh00:29:093
2658999118,2cyclictest20440-21ssh23:08:573
2658999118,2cyclictest17745-21date21:05:003
2658999118,2cyclictest14308-21grep19:50:203
2658999118,2cyclictest12594-21chrt21:50:333
2658999118,2cyclictest1150-21taskset19:20:553
2658999118,2cyclictest10022-21ssh23:00:443
2658999117,3cyclictest27080-21sed23:35:263
2658999114,3cyclictest224-21systemd-journal22:15:283
2658999110,10cyclictest24832-21sed22:00:163
2658799118,2cyclictest7180-21ssh23:45:322
2658799118,2cyclictest6849-21tr00:35:172
2658799118,2cyclictest4996-21wc23:20:202
2658799118,2cyclictest31774-21cat19:20:112
2658799118,2cyclictest30491-21sh00:03:142
2658799118,2cyclictest2907-21ssh22:32:122
2658799118,2cyclictest27753-21ssh22:26:112
2658799118,2cyclictest23423-21rm22:23:402
2658799118,2cyclictest13445-21ssh22:15:262
2658799115,5cyclictest5982-21ssh22:35:132
2658799113,7cyclictest37850irq/131-snd_hda23:05:122
2658799110,10cyclictest7393-21ssh22:10:472
2658699118,2cyclictest8873-21awk21:25:141
2658699118,2cyclictest8629-21sed23:00:181
2658699118,2cyclictest7321-21ssh23:45:411
2658699118,2cyclictest6782-21grep22:35:171
2658699118,2cyclictest5913-21ssh23:20:331
2658699118,2cyclictest4193-21ssh21:20:481
2658699118,2cyclictest418-21chrt20:29:411
2658699118,2cyclictest32755-21grep00:30:181
2658699118,2cyclictest32582-21head19:20:181
2658699118,2cyclictest23684-21grep21:35:171
2658699118,2cyclictest21088-21cut20:05:141
2658699118,2cyclictest2061-21chrt20:30:241
2658699118,2cyclictest19875-21grep00:20:141
2658699118,2cyclictest18744-21tr20:00:151
2658699118,2cyclictest14012-21cat20:55:181
2658699118,2cyclictest13697-21grep22:15:271
2658699117,3cyclictest25-21ksoftirqd/123:35:241
2658699116,3cyclictest14841-21grep22:40:281
2658699115,2cyclictest224-21systemd-journal19:35:001
2658699114,3cyclictest0-21swapper/121:40:181
2658699110,10cyclictest3533-21taskset19:25:551
2658599119,1cyclictest11848-21cut22:15:120
2658599118,2cyclictest988-21head00:30:210
2658599118,2cyclictest5646-21sort19:30:280
2658599118,2cyclictest3712-21ssh23:20:050
2658599118,2cyclictest3621-21tr20:35:170
2658599118,2cyclictest32110-21cat22:05:220
2658599118,2cyclictest28382-21sed22:50:290
2658599118,2cyclictest27527-21chrt21:15:120
2658599118,2cyclictest25731-21ssh00:00:040
2658599118,2cyclictest22827-21ssh23:33:060
2658599118,2cyclictest21587-21grep20:05:170
2658599118,2cyclictest20896-21ssh21:33:470
2658599118,2cyclictest18677-21ssh23:07:150
2658599118,2cyclictest18617-21tail21:05:170
2658599118,2cyclictest18014-21ssh23:54:190
2658599118,2cyclictest16749-21cut19:55:180
2658599118,2cyclictest15383-21sed19:50:300
2658599118,2cyclictest10602-21head19:40:280
2658599117,3cyclictest15759-21grep21:00:130
2658599114,5cyclictest496-21dbus-daemon00:10:300
2658599111,6cyclictest0-21swapper/022:55:140
2658599111,4cyclictest0-21swapper/020:10:130
2658599110,10cyclictest28552-21cat20:20:140
2658999108,1cyclictest9358-21expr19:40:163
2658999108,1cyclictest6907-21ssh23:45:143
2658999108,1cyclictest30654-21if_err_enp2s021:40:183
2658999108,1cyclictest30642-21awk19:15:223
2658999108,1cyclictest27836-21cat19:10:193
2658999108,1cyclictest26397-21ssh22:50:083
2658999108,1cyclictest21710-21ssh23:56:053
2658999107,2cyclictest6215-21expr20:40:163
2658999107,2cyclictest31112-21cat20:25:153
2658999107,2cyclictest25093-21ssh22:25:123
2658999107,2cyclictest11477-21cat19:45:143
2658999106,2cyclictest1291-21runrttasks19:55:203
2658799108,1cyclictest4421-21expr19:30:162
2658799108,1cyclictest23846-21fschecks_count20:10:162
2658799108,1cyclictest18170-21cut21:05:152
2658799108,1cyclictest11574-21cat19:45:152
2658799107,2cyclictest2748-21sed19:25:192
2658799107,2cyclictest26106-21grep20:15:162
2658799107,2cyclictest26106-21grep20:15:162
2658799107,2cyclictest17382-21taskset21:01:472
2658799103,5cyclictest7439-21grep20:40:282
2658799100,9cyclictest9330-21expr19:40:162
2658699108,1cyclictest7700-21taskset20:43:131
2658699108,1cyclictest3496-21egrep20:35:171
2658699108,1cyclictest14057-21ssh00:15:161
2658699108,1cyclictest12249-21grep23:50:121
2658699108,1cyclictest11349-21fschecks_time20:50:161
2658699107,2cyclictest18509-21sort21:05:171
2658699107,2cyclictest12584-21gzip19:45:231
2658699106,2cyclictest29652-21grep19:15:151
2658699100,1cyclictest0-21swapper/120:20:121
2658599108,1cyclictest8314-21ssh23:46:420
2658599107,2cyclictest9953-21taskset20:46:390
2658599107,2cyclictest4045-21wc19:30:000
2658599107,2cyclictest30712-21chrt19:15:230
2658599107,2cyclictest27190-21ssh00:25:170
2658599107,2cyclictest11191-21sed20:50:150
265899996,2cyclictest737-21cat20:30:133
265879997,1cyclictest27034-21seq19:10:142
265879997,1cyclictest18728-21cut20:00:152
265879996,2cyclictest6754-21grep19:35:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional