You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-23 - 20:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Nov 23, 2024 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30881992421,2cyclictest7781-21turbostat.cron08:35:002
3088099240,23cyclictest17790-21ls08:55:001
30876992422,1cyclictest6189-21basename11:50:010
3087699240,23cyclictest0-21swapper/007:15:460
3087699240,22cyclictest313-21systemd-journal10:00:000
30882992321,1cyclictest20251-21apt-get10:05:033
30882992321,1cyclictest0-21swapper/309:15:003
3088299230,23cyclictest20305-21perf11:10:003
3088299230,1cyclictest7786-21apt-get08:35:183
3088199232,1cyclictest0-21swapper/207:40:462
30881992321,1cyclictest1125-21apt-get07:15:462
3088099230,2cyclictest8578-21idleruntime-cro11:55:001
3088099230,22cyclictest20621-21perf12:20:001
3088099230,22cyclictest15795-21cron12:10:001
30876992320,1cyclictest313-21systemd-journal07:45:000
3087699230,2cyclictest8311-21grep10:45:000
3087699230,22cyclictest28981-21turbostat11:30:000
30882992222,0cyclictest0-21swapper/309:40:473
30882992220,1cyclictest28250-21perf12:35:003
30882992220,1cyclictest2652-21dpkg09:30:013
30882992220,1cyclictest0-21swapper/312:05:003
3088299220,21cyclictest6211-21munin-run11:50:013
3088299220,21cyclictest28350-21perf11:25:003
3088299220,1cyclictest3306-21apt-get08:25:453
3088299220,1cyclictest25410-21munin-run12:30:003
3088299220,1cyclictest20926-21latency_hist07:55:003
3088299220,1cyclictest16127-21sh07:45:003
30881992222,0cyclictest18494-21apt-get08:55:452
30881992220,1cyclictest3806-21perf11:45:002
30881992220,1cyclictest20251-21apt-get10:05:002
30881992219,2cyclictest5373-21latency_hist08:30:002
3088199220,21cyclictest30267-21sendmail-msp09:20:002
3088199220,21cyclictest0-21swapper/212:25:482
3088199220,1cyclictest15821-21apt-get11:00:472
30880992220,1cyclictest20261-21munin-plugin-st10:05:001
3088099220,22cyclictest0-21swapper/107:15:471
3088099220,21cyclictest18544-21perf07:50:011
3088099220,21cyclictest15403-21munin-run08:50:001
3088099220,21cyclictest10870-21apt-get09:45:011
30876992222,0cyclictest0-21swapper/008:35:180
30876992221,1cyclictest21007-21apt-get11:10:490
30876992220,1cyclictest23052-21munin-run10:10:000
30876992220,1cyclictest1395-21munin-run11:40:000
30876992219,2cyclictest13318-21sh07:40:000
3087699220,21cyclictest8611-21apt-get10:45:460
3087699220,21cyclictest3560-21apt-get07:20:470
3087699220,21cyclictest3405-21apt-get10:35:480
3087699220,21cyclictest18508-21apt-get12:15:490
3087699220,21cyclictest18494-21apt-get08:55:460
3087699220,21cyclictest10914-21apt-get08:40:460
3087699220,21cyclictest0-21swapper/010:25:470
3087699220,1cyclictest8018-21apt-get07:30:020
3087699220,1cyclictest564-21idleruntime-cro08:20:000
3087699220,1cyclictest28619-21apt-get10:20:480
3087699220,1cyclictest28267-21ls10:20:000
3087699220,1cyclictest15493-21latency_hist11:00:000
3087699220,0cyclictest0-21swapper/007:10:450
30882992120,1cyclictest13721-21apt-get12:05:493
30882992119,1cyclictest15513-21munin-run11:00:003
30882992119,1cyclictest13365-21cut07:40:003
30882992119,1cyclictest0-21swapper/307:35:003
3088299210,20cyclictest0-21swapper/310:40:473
3088299210,20cyclictest0-21swapper/308:05:463
3088299210,0cyclictest0-21swapper/307:55:453
30881992119,1cyclictest20575-21cron12:20:002
30881992119,1cyclictest0-21swapper/209:50:002
30881992118,2cyclictest593-21dbus-daemon12:00:002
30881992118,2cyclictest15799-21perf12:10:002
3088199210,20cyclictest0-21swapper/209:50:512
3088199210,1cyclictest21007-21apt-get11:10:492
3088199210,0cyclictest31151-21apt-get07:10:462
3088199210,0cyclictest31019-21apt-get10:25:472
3088199210,0cyclictest0-21swapper/210:55:472
30880992121,0cyclictest0-21swapper/110:20:481
30880992121,0cyclictest0-21swapper/110:05:031
30880992120,1cyclictest30871-21cyclictest09:30:541
30880992120,1cyclictest23657-21apt-get08:00:471
30880992120,0cyclictest0-21swapper/112:00:481
3088099210,21cyclictest0-21swapper/108:10:471
3088099210,20cyclictest0-21swapper/112:10:481
3088099210,20cyclictest0-21swapper/111:40:461
3088099210,20cyclictest0-21swapper/110:10:501
3088099210,1cyclictest28315-21lockfile-create10:20:001
3088099210,1cyclictest0-21swapper/109:10:511
3088099210,0cyclictest21242-21apt-get07:55:451
3088099210,0cyclictest0-21swapper/109:50:511
3088099210,0cyclictest0-21swapper/109:15:461
3087699210,21cyclictest0-21swapper/011:05:460
3087699210,21cyclictest0-21swapper/007:50:440
3087699210,20cyclictest0-21swapper/008:00:470
3087699210,1cyclictest7852-21sendmail-msp09:40:000
3087699210,1cyclictest23320-21apt-get12:25:470
3087699210,0cyclictest15705-21apt-get08:50:480
3087699210,0cyclictest0-21swapper/011:00:470
3087699210,0cyclictest0-21swapper/009:25:460
3087699210,0cyclictest0-21swapper/008:30:480
30882992020,0cyclictest0-21swapper/310:15:483
30882992019,0cyclictest897-21apt-get08:20:463
30882992019,0cyclictest0-21swapper/311:35:473
3088299200,20cyclictest0-21swapper/312:10:483
3088299200,20cyclictest0-21swapper/310:00:463
3088299200,20cyclictest0-21swapper/307:10:463
3088299200,0cyclictest28568-21apt-get12:35:503
3088299200,0cyclictest0-21swapper/310:25:483
3088299200,0cyclictest0-21swapper/309:45:013
3088299200,0cyclictest0-21swapper/309:35:463
30881992020,0cyclictest0-21swapper/212:10:492
30881992020,0cyclictest0-21swapper/211:50:472
30881992020,0cyclictest0-21swapper/210:10:502
30881992020,0cyclictest0-21swapper/209:10:512
30881992020,0cyclictest0-21swapper/208:40:462
30881992020,0cyclictest0-21swapper/207:35:462
30881992017,2cyclictest19893-21ntp_states07:50:502
30881992017,2cyclictest0-21swapper/208:20:462
3088199200,20cyclictest0-21swapper/207:45:482
3088199200,0cyclictest13308-21apt-get08:45:452
3088199200,0cyclictest0-21swapper/210:20:482
3088199200,0cyclictest0-21swapper/210:15:482
30880992020,0cyclictest0-21swapper/112:30:021
30880992020,0cyclictest0-21swapper/112:20:481
30880992020,0cyclictest0-21swapper/110:25:481
30880992020,0cyclictest0-21swapper/109:25:461
30880992020,0cyclictest0-21swapper/109:20:461
30880992020,0cyclictest0-21swapper/108:55:011
30880992020,0cyclictest0-21swapper/108:15:481
30880992020,0cyclictest0-21swapper/107:35:461
30880992019,0cyclictest23306-21apt-get09:05:461
30880992019,0cyclictest18842-21apt-get07:50:431
3088099200,0cyclictest0-21swapper/108:35:461
3088099200,0cyclictest0-21swapper/107:10:451
30876992020,0cyclictest0-21swapper/012:30:020
30876992020,0cyclictest0-21swapper/012:05:490
30876992020,0cyclictest0-21swapper/008:10:470
30876992018,1cyclictest28353-21latency_hist11:25:000
30876992018,1cyclictest20927-21apt-get12:20:480
3087699200,0cyclictest0-21swapper/012:35:500
3087699200,0cyclictest0-21swapper/012:00:490
30882991919,0cyclictest0-21swapper/310:35:483
30882991919,0cyclictest0-21swapper/308:40:463
30882991917,1cyclictest0-21swapper/309:10:003
3088299190,18cyclictest0-21swapper/308:00:473
30881991919,0cyclictest0-21swapper/212:30:022
30881991919,0cyclictest0-21swapper/211:35:472
30881991919,0cyclictest0-21swapper/210:40:022
30881991919,0cyclictest0-21swapper/209:40:472
30881991919,0cyclictest0-21swapper/209:05:462
30881991918,0cyclictest8611-21apt-get10:45:462
30881991917,1cyclictest30673-21munin-run12:40:012
30881991917,1cyclictest0-21swapper/207:30:002
30880991919,0cyclictest0-21swapper/111:45:471
30880991919,0cyclictest0-21swapper/111:15:471
30880991919,0cyclictest0-21swapper/111:10:011
30880991919,0cyclictest0-21swapper/110:55:461
30880991919,0cyclictest0-21swapper/110:50:491
30880991919,0cyclictest0-21swapper/109:40:461
30880991917,1cyclictest593-21dbus-daemon07:25:001
30876991919,0cyclictest3363-21apt-get09:30:540
30876991919,0cyclictest10962-21apt-get09:45:460
30876991919,0cyclictest0-21swapper/011:30:440
30876991919,0cyclictest0-21swapper/010:30:500
30876991919,0cyclictest0-21swapper/009:05:460
30876991919,0cyclictest0-21swapper/007:25:450
30876991918,1cyclictest0-21swapper/008:05:450
30876991917,1cyclictest8169-21apt-get09:40:470
30882991818,0cyclictest28499-21apt-get08:10:473
30882991818,0cyclictest0-21swapper/311:50:473
30882991818,0cyclictest0-21swapper/311:40:463
30882991818,0cyclictest0-21swapper/311:00:473
30882991818,0cyclictest0-21swapper/310:20:483
30882991817,0cyclictest8911-21apt-get11:55:483
30882991816,1cyclictest30228-21latency_hist09:20:003
30881991818,0cyclictest5751-21apt-get09:35:462
30881991818,0cyclictest30569-21apt-get09:20:462
30881991818,0cyclictest0-21swapper/212:20:482
30881991818,0cyclictest0-21swapper/211:30:442
30881991818,0cyclictest0-21swapper/210:35:482
30881991818,0cyclictest0-21swapper/209:25:462
30881991818,0cyclictest0-21swapper/208:35:462
30881991818,0cyclictest0-21swapper/207:55:452
30881991817,1cyclictest25750-21perf08:05:002
30881991817,1cyclictest0-21swapper/209:30:542
30881991815,2cyclictest23023-21awk10:10:002
3088199180,2cyclictest28174-21cron08:10:002
30880991818,0cyclictest0-21swapper/110:35:471
30880991818,0cyclictest0-21swapper/109:35:461
30880991818,0cyclictest0-21swapper/107:40:461
30880991817,1cyclictest0-21swapper/111:55:481
30876991818,0cyclictest0-21swapper/008:25:450
30876991818,0cyclictest0-21swapper/008:20:460
30876991818,0cyclictest0-21swapper/007:55:450
30882991717,0cyclictest0-21swapper/312:15:483
30882991717,0cyclictest0-21swapper/310:50:493
30882991717,0cyclictest0-21swapper/308:50:483
30882991717,0cyclictest0-21swapper/308:30:483
30882991716,0cyclictest0-21swapper/309:30:543
30882991715,1cyclictest0-21swapper/307:25:003
3088199172,14cyclictest0-21swapper/210:30:552
30881991717,0cyclictest0-21swapper/212:00:482
30881991717,0cyclictest0-21swapper/208:10:472
30881991716,1cyclictest0-21swapper/211:20:522
30881991716,1cyclictest0-21swapper/209:55:502
30881991716,1cyclictest0-21swapper/207:30:552
30881991715,1cyclictest593-21dbus-daemon11:15:552
30880991717,0cyclictest0-21swapper/112:35:501
30880991717,0cyclictest0-21swapper/112:25:471
30880991717,0cyclictest0-21swapper/110:45:461
3087699170,16cyclictest13136-21latency_hist10:55:000
3087699170,11cyclictest20244-21latency_hist10:05:010
30882991616,0cyclictest5951-21apt-get07:25:453
30882991616,0cyclictest20290-21apt-get11:10:013
30882991614,1cyclictest0-21swapper/309:53:403
3088299160,1cyclictest593-21dbus-daemon08:50:003
30881991614,1cyclictest0-21swapper/208:15:482
30881991611,5cyclictest0-21swapper/209:00:552
3088099168,7cyclictest8837-21munin-node11:27:501
30880991616,0cyclictest0-21swapper/111:20:471
30880991616,0cyclictest0-21swapper/108:25:451
30880991616,0cyclictest0-21swapper/107:30:471
30880991614,1cyclictest22994-21cron09:05:001
3088099160,0cyclictest0-21swapper/111:00:471
30876991616,0cyclictest0-21swapper/012:10:480
30876991616,0cyclictest0-21swapper/009:15:000
30876991614,1cyclictest0-21swapper/008:45:550
30882991515,0cyclictest0-21swapper/312:20:483
30882991515,0cyclictest0-21swapper/311:30:443
30882991515,0cyclictest0-21swapper/309:00:473
30882991514,0cyclictest23358-21apt-get10:10:503
30882991513,1cyclictest0-21swapper/310:35:003
30882991513,1cyclictest0-21swapper/309:55:523
3088299151,10cyclictest313-21systemd-journal07:15:113
3088299150,0cyclictest16455-21apt-get07:45:483
30881991515,0cyclictest0-21swapper/210:50:492
30881991514,1cyclictest4115-21apt-get11:45:472
30881991514,1cyclictest0-21swapper/208:50:562
30881991514,1cyclictest0-21swapper/207:20:562
30880991515,0cyclictest18212-21apt-get11:05:461
30880991515,0cyclictest0-21swapper/108:20:461
3088099150,1cyclictest6992-21grep08:30:551
3088099150,1cyclictest15968-21cstates09:55:471
3088099150,15cyclictest0-21swapper/107:26:171
30876991514,0cyclictest0-21swapper/007:45:480
30876991512,2cyclictest11013-21munin-run12:00:000
3087699150,1cyclictest23964-21gltestperf10:10:520
3087699150,14cyclictest0-21swapper/009:00:560
30882991414,0cyclictest0-21swapper/311:28:033
30882991414,0cyclictest0-21swapper/308:55:463
3088299140,14cyclictest0-21swapper/311:15:563
3088299140,14cyclictest0-21swapper/310:45:373
3088299140,14cyclictest0-21swapper/308:15:123
3088299140,0cyclictest0-21swapper/309:20:153
3088199140,14cyclictest0-21swapper/211:27:152
3088199140,14cyclictest0-21swapper/211:05:162
30880991414,0cyclictest0-21swapper/111:30:441
3088099140,14cyclictest0-21swapper/108:40:021
3088099140,14cyclictest0-21swapper/108:09:261
30876991414,0cyclictest0-21swapper/011:51:210
30876991414,0cyclictest0-21swapper/011:40:120
30876991414,0cyclictest0-21swapper/011:15:150
30876991414,0cyclictest0-21swapper/009:50:220
3087699140,14cyclictest0-21swapper/009:15:340
3087699140,0cyclictest0-21swapper/009:20:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional