You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Jul 27, 2024 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15809992721,5cyclictest0-21swapper/200:40:002
1580999270,6cyclictest13510-21munin-run22:55:012
15803992720,6cyclictest5722-21ssh22:47:400
1580399270,23cyclictest6449-21sh21:40:130
1580399270,1cyclictest4375-21df_inode22:12:400
1580399270,0cyclictest9789-21diskmemload23:16:190
15809992623,3cyclictest0-21swapper/200:18:332
15809992622,4cyclictest0-21swapper/221:18:152
1580999260,26cyclictest0-21swapper/223:35:022
1580999260,25cyclictest581-21dbus-daemon19:35:012
1580999260,25cyclictest434-21ssh23:48:032
1580999260,22cyclictest9789-21diskmemload22:43:202
1580999260,1cyclictest0-21swapper/221:45:162
1580999260,0cyclictest0-21swapper/200:10:182
1580699260,25cyclictest0-21swapper/123:37:371
15803992625,1cyclictest62050irq/126-eno123:55:020
1581399250,24cyclictest0-21swapper/300:03:583
15809992521,4cyclictest9789-21diskmemload22:18:122
15809992520,5cyclictest25487-21apt-get20:37:372
1580999250,24cyclictest29193-21ssh21:30:012
1580999250,24cyclictest0-21swapper/223:31:322
1580999250,20cyclictest0-21swapper/222:47:402
1580999250,1cyclictest3869-21sh21:37:432
1580999250,1cyclictest14735-21ssh22:56:532
1580999250,1cyclictest0-21swapper/200:26:182
1580999250,0cyclictest25558-21ssh23:42:172
15806992523,1cyclictest16084-21ssh21:50:221
15803992521,4cyclictest0-21swapper/021:55:480
1580399250,24cyclictest9789-21diskmemload23:45:330
1580399250,24cyclictest0-21swapper/023:08:060
1580399250,21cyclictest9789-21diskmemload22:25:130
1580399250,1cyclictest0-21swapper/000:23:520
1580399250,0cyclictest0-21swapper/023:54:450
15813992421,2cyclictest19232-21ssh23:01:223
1581399240,24cyclictest0-21swapper/300:31:543
15809992422,2cyclictest0-21swapper/223:25:262
15809992422,1cyclictest9810-21idleruntime-cro21:10:012
15809992421,3cyclictest9789-21diskmemload23:22:002
15809992419,4cyclictest13166-21ssh22:22:052
15809992416,4cyclictest9789-21diskmemload22:06:492
1580999240,20cyclictest9789-21diskmemload21:23:122
1580999240,1cyclictest24908-21sh21:25:532
1580999240,1cyclictest11843-21ssh00:33:522
1580999240,0cyclictest0-21swapper/223:59:172
1580999240,0cyclictest0-21swapper/222:36:292
1580999240,0cyclictest0-21swapper/222:36:292
1580999240,0cyclictest0-21swapper/222:01:392
1580999240,0cyclictest0-21swapper/221:41:382
15806992424,0cyclictest9789-21diskmemload21:55:461
15806992422,1cyclictest26078-21ssh00:15:011
1580699240,24cyclictest0-21swapper/122:43:401
1580699240,24cyclictest0-21swapper/121:36:231
1580699240,23cyclictest30285-21ssh21:32:001
1580699240,23cyclictest0-21swapper/122:51:101
1580699240,23cyclictest0-21swapper/122:29:051
1580699240,23cyclictest0-21swapper/122:11:451
1580699240,1cyclictest0-21swapper/123:12:001
15803992421,3cyclictest0-21swapper/023:27:090
15803992421,3cyclictest0-21swapper/022:05:370
15803992420,4cyclictest0-21swapper/000:39:150
15803992420,3cyclictest0-21swapper/000:31:370
1580399240,24cyclictest0-21swapper/022:54:210
1580399240,24cyclictest0-21swapper/000:27:470
1580399240,23cyclictest9789-21diskmemload21:50:160
1580399240,23cyclictest29064-21/usr/sbin/munin22:37:440
1580399240,23cyclictest29064-21/usr/sbin/munin22:37:440
1580399240,23cyclictest0-21swapper/023:12:400
1580399240,23cyclictest0-21swapper/023:01:590
1580399240,20cyclictest0-21swapper/021:30:500
1580399240,1cyclictest0-21swapper/022:43:310
1580399240,0cyclictest0-21swapper/021:36:380
15813992323,0cyclictest0-21swapper/323:55:393
15813992323,0cyclictest0-21swapper/300:27:363
15813992322,1cyclictest15526-21taskset21:49:513
15813992321,2cyclictest25745-21ssh22:00:343
15813992321,1cyclictest31121-21cut23:47:403
1581399230,23cyclictest0-21swapper/322:46:343
1581399230,23cyclictest0-21swapper/322:38:513
1581399230,23cyclictest0-21swapper/322:38:513
1581399230,23cyclictest0-21swapper/322:10:473
1581399230,22cyclictest32110-21tune2fs22:07:403
1581399230,22cyclictest29827-21ssh00:18:363
1581399230,22cyclictest24702-21ssh21:25:313
1581399230,22cyclictest24545-21kworker/u8:021:13:583
1581399230,22cyclictest20403-21latency_hist21:55:013
1581399230,22cyclictest14932-21ssh21:15:143
1581399230,0cyclictest9789-21diskmemload23:05:553
1581399230,0cyclictest9789-21diskmemload22:26:213
1581399230,0cyclictest9789-21diskmemload22:19:293
1581399230,0cyclictest0-21swapper/323:26:293
1581399230,0cyclictest0-21swapper/323:22:043
1581399230,0cyclictest0-21swapper/300:13:523
15809992323,0cyclictest0-21swapper/220:57:382
15809992322,1cyclictest20198-21ssh22:27:542
15809992321,2cyclictest1633-21ssh23:17:082
15809992321,2cyclictest0-21swapper/200:04:522
15809992319,3cyclictest16357-21ssh21:50:512
15809992319,3cyclictest15836-21apt-get20:17:372
15809992319,2cyclictest17404-21rm00:07:122
1580999230,23cyclictest0-21swapper/223:53:012
1580999230,23cyclictest0-21swapper/223:10:092
1580999230,23cyclictest0-21swapper/221:56:102
1580999230,0cyclictest0-21swapper/223:06:452
1580999230,0cyclictest0-21swapper/221:12:142
15806992323,0cyclictest0-21swapper/123:45:381
15806992323,0cyclictest0-21swapper/123:21:241
15806992323,0cyclictest0-21swapper/123:15:191
15806992323,0cyclictest0-21swapper/123:00:141
15806992323,0cyclictest0-21swapper/121:15:051
15806992323,0cyclictest0-21swapper/121:10:071
15806992323,0cyclictest0-21swapper/121:02:371
15806992322,1cyclictest9789-21diskmemload22:21:221
15806992322,0cyclictest0-21swapper/122:30:321
15806992321,1cyclictest0-21swapper/100:30:401
15806992320,2cyclictest12740-21sh00:02:211
1580699230,22cyclictest4351-21ssh22:45:131
1580699230,22cyclictest0-21swapper/121:46:011
1580699230,22cyclictest0-21swapper/100:23:441
1580699230,1cyclictest0-21swapper/123:07:591
1580699230,1cyclictest0-21swapper/122:05:501
1580699230,0cyclictest0-21swapper/123:42:331
1580699230,0cyclictest0-21swapper/123:31:591
1580699230,0cyclictest0-21swapper/122:00:361
15803992323,0cyclictest9789-21diskmemload00:10:130
15803992322,1cyclictest62050irq/126-eno122:33:470
15803992321,2cyclictest0-21swapper/022:56:170
15803992321,1cyclictest0-21swapper/000:15:330
15803992320,3cyclictest0-21swapper/021:27:210
15803992319,4cyclictest9789-21diskmemload22:15:590
15803992319,3cyclictest0-21swapper/022:00:050
1580399230,23cyclictest0-21swapper/022:21:470
1580399230,22cyclictest7794-21dump-pmu-power20:00:000
1580399230,22cyclictest3068-21apt-get20:57:380
1580399230,22cyclictest29852-21munin-run20:45:000
1580399230,22cyclictest19896-21sh21:20:350
1580399230,22cyclictest19630-21sh23:35:170
1580399230,22cyclictest18373-21rm23:33:260
1580399230,22cyclictest11688-21sh00:00:270
1580399230,22cyclictest0-21swapper/023:41:360
1580399230,20cyclictest0-21swapper/021:17:360
1580399230,1cyclictest0-21swapper/000:06:010
15813992222,0cyclictest17513-21ssh00:07:243
15813992222,0cyclictest0-21swapper/323:31:263
15813992222,0cyclictest0-21swapper/322:53:063
15813992222,0cyclictest0-21swapper/322:32:173
15813992220,1cyclictest849-21ssh23:15:453
15813992220,1cyclictest6736-21sh21:40:443
1581399220,22cyclictest0-21swapper/323:52:013
1581399220,22cyclictest0-21swapper/323:40:243
1581399220,22cyclictest0-21swapper/322:21:273
1581399220,21cyclictest9789-21diskmemload22:55:543
1581399220,21cyclictest9789-21diskmemload21:20:133
1581399220,21cyclictest30892-21ssh00:20:033
1581399220,21cyclictest28618-21ssh23:10:583
1581399220,21cyclictest12755-21sh00:35:033
1581399220,21cyclictest0-21swapper/321:30:233
1581399220,21cyclictest0-21swapper/320:32:393
1581399220,1cyclictest2626-21grep20:55:003
1581399220,0cyclictest0-21swapper/322:43:193
1581399220,0cyclictest0-21swapper/321:35:483
1580999220,22cyclictest0-21swapper/222:30:572
1580999220,20cyclictest0-21swapper/219:19:282
1580999220,0cyclictest9789-21diskmemload00:22:302
1580999220,0cyclictest0-21swapper/223:00:542
15806992222,0cyclictest9789-21diskmemload23:50:321
15806992222,0cyclictest0-21swapper/121:25:051
15806992221,1cyclictest0-21swapper/122:38:301
15806992221,1cyclictest0-21swapper/122:38:301
15806992221,1cyclictest0-21swapper/100:05:351
1580699220,22cyclictest21246-21ssh00:10:331
1580699220,22cyclictest0-21swapper/122:16:111
1580699220,22cyclictest0-21swapper/121:21:011
1580699220,21cyclictest9789-21diskmemload23:25:231
1580699220,21cyclictest9789-21diskmemload22:55:201
1580699220,21cyclictest0-21swapper/100:25:211
1580699220,1cyclictest9789-21diskmemload00:37:181
1580699220,1cyclictest0-21swapper/123:59:201
15803992215,0cyclictest0-21swapper/019:27:380
1580399220,2cyclictest25845-21/usr/sbin/munin19:32:380
1580399220,21cyclictest5006-21munin-run19:55:000
1580399220,1cyclictest0-21swapper/021:45:120
1580399220,0cyclictest0-21swapper/023:21:440
1580399220,0cyclictest0-21swapper/021:10:170
1580399220,0cyclictest0-21swapper/020:47:380
15813992120,1cyclictest20585-21munin-run19:20:013
15813992119,1cyclictest8417-21apt-get20:00:023
15813992119,1cyclictest4968-21cron19:55:003
1581399210,21cyclictest0-21swapper/321:55:233
1581399210,0cyclictest9789-21diskmemload23:35:073
15809992121,0cyclictest0-21swapper/219:22:372
15809992120,0cyclictest0-21swapper/220:12:382
1580999210,20cyclictest28269-21apt-get19:37:372
1580999210,0cyclictest0-21swapper/222:10:202
15806992121,0cyclictest0-21swapper/119:17:381
15806992119,1cyclictest0-21swapper/119:22:371
1580699210,21cyclictest0-21swapper/121:40:191
1580699210,20cyclictest0-21swapper/120:47:391
15803992121,0cyclictest8647-21apt-get20:02:380
15803992119,1cyclictest0-21swapper/020:50:020
1580399210,1cyclictest11677-21gltestperf20:07:410
15813992020,0cyclictest21030-21apt-get19:22:373
15813992020,0cyclictest18251-21apt-get20:22:383
15813992020,0cyclictest0-21swapper/321:07:393
15813992020,0cyclictest0-21swapper/319:57:383
15813992020,0cyclictest0-21swapper/319:37:373
15813992018,1cyclictest27811-21perf19:35:013
15813992017,2cyclictest32234-21apt-get20:50:003
1581399200,0cyclictest0-21swapper/320:57:383
15809992020,0cyclictest0-21swapper/221:02:372
1580999200,1cyclictest30299-21apt-get20:47:392
1580999200,19cyclictest0-21swapper/220:42:392
15806992020,0cyclictest0-21swapper/120:37:371
15806992020,0cyclictest0-21swapper/120:00:021
15806992020,0cyclictest0-21swapper/119:32:381
15806992020,0cyclictest0-21swapper/119:12:371
15806992017,2cyclictest2605-21latency_hist20:55:001
15803992020,0cyclictest0-21swapper/020:37:370
15803992020,0cyclictest0-21swapper/020:22:380
15803992020,0cyclictest0-21swapper/019:12:370
15803992019,1cyclictest5480-21apt-get21:02:370
1580399200,1cyclictest23092-21apt-get20:32:390
15813991919,0cyclictest0-21swapper/320:12:383
15813991917,1cyclictest32633-21munin-run19:45:013
15809991919,0cyclictest3054-21apt-get19:52:372
15809991919,0cyclictest0-21swapper/220:52:392
15809991919,0cyclictest0-21swapper/220:00:022
15809991918,0cyclictest0-21swapper/220:22:382
15809991917,1cyclictest616-21gdbus20:10:012
1580999191,17cyclictest16748-21perl19:12:392
15806991919,0cyclictest0-21swapper/120:07:381
15803991919,0cyclictest13433-21apt-get20:12:380
15803991918,1cyclictest20573-21perf19:20:010
15809991818,0cyclictest0-21swapper/220:27:362
15809991818,0cyclictest0-21swapper/219:55:022
15803991816,2cyclictest0-21swapper/019:42:370
15803991816,1cyclictest0-21swapper/019:50:000
1580399180,17cyclictest0-21swapper/020:16:030
15813991717,0cyclictest27904-21apt-get20:42:373
15813991715,1cyclictest12980-21apt20:10:013
1581399170,0cyclictest0-21swapper/320:17:373
15809991717,0cyclictest0-21swapper/220:32:392
15806991717,0cyclictest0-21swapper/120:32:391
15806991717,0cyclictest0-21swapper/120:12:381
15806991717,0cyclictest0-21swapper/119:47:381
15806991715,1cyclictest30681-21apt-get19:42:371
15806991715,1cyclictest0-21swapper/120:22:381
15803991717,0cyclictest0-21swapper/020:27:370
15803991717,0cyclictest0-21swapper/019:22:370
15813991616,0cyclictest0-21swapper/319:47:383
15813991616,0cyclictest0-21swapper/319:27:383
15813991615,0cyclictest0-21swapper/321:02:373
15813991613,2cyclictest20989-21cstates20:27:383
1580999161,15cyclictest0-21swapper/219:47:492
1580999161,14cyclictest23479-21apt-get19:27:382
15806991616,0cyclictest0-21swapper/119:52:381
15806991616,0cyclictest0-21swapper/119:37:371
15806991616,0cyclictest0-21swapper/119:27:381
15806991615,1cyclictest22-21ksoftirqd/119:55:011
15806991614,1cyclictest22621-21cron20:30:011
15806991515,0cyclictest0-21swapper/120:17:371
15803991515,0cyclictest0-21swapper/021:07:380
1580399150,0cyclictest0-21swapper/019:37:370
15813991414,0cyclictest0-21swapper/320:35:023
1581399140,14cyclictest0-21swapper/319:11:093
15806991414,0cyclictest0-21swapper/121:07:051
1580699140,14cyclictest0-21swapper/120:55:101
1580699140,14cyclictest0-21swapper/120:41:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional