You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 08:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Fri Nov 22, 2024 00:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
932499270,1cyclictest30671-21perf00:20:002
9324992522,3cyclictest15391-21apt-get22:40:392
9318992522,2cyclictest28507-21cron19:50:000
932599240,1cyclictest30616-21idleruntime-cro21:00:003
932499240,20cyclictest0-21swapper/223:15:392
9321992422,1cyclictest7097-21munin-run22:25:001
9325992321,1cyclictest20200-21munin-run20:40:003
932599230,22cyclictest9567-21apt-get19:10:383
932499230,22cyclictest3381-21perf20:05:002
932499230,0cyclictest12972-21apt-get22:35:392
932199230,2cyclictest20204-21sendmail-msp00:00:001
932199230,22cyclictest948-21apt20:00:001
931899230,22cyclictest15075-21apt22:40:000
931899230,1cyclictest21309-21latency_hist19:35:010
9325992220,1cyclictest0-21swapper/323:00:003
932599220,21cyclictest969-21awk20:00:003
932599220,21cyclictest5464-21munin-run00:35:003
932599220,21cyclictest27388-21apt-get23:05:413
932599220,21cyclictest20583-21apt-get00:00:383
932599220,21cyclictest0-21swapper/321:30:433
932599220,21cyclictest0-21swapper/320:25:013
932599220,1cyclictest587-21munin-run21:05:003
932599220,1cyclictest4665-21perf22:20:003
9324992219,3cyclictest9486-21apt-get22:30:022
9321992220,1cyclictest4729-21dump-pmu-power23:25:001
9321992220,1cyclictest4729-21dump-pmu-power23:25:001
9321992220,1cyclictest31875-21awk22:10:001
9321992220,1cyclictest27783-21munin-run20:55:001
9321992220,1cyclictest24659-21cron23:00:001
9321992219,2cyclictest27085-21latency_hist23:05:001
932199220,21cyclictest22587-21perf20:45:001
932199220,21cyclictest17449-21munin-plugin-st21:40:001
932199220,21cyclictest12937-21apt-get21:30:431
932199220,21cyclictest12271-21latency_hist22:35:001
932199220,21cyclictest0-21swapper/100:00:381
932199220,1cyclictest8464-21apt-get20:15:401
932199220,1cyclictest19846-21idleruntime-cro21:45:001
9318992220,1cyclictest5454-21awk00:35:000
9318992219,2cyclictest7876-21perf00:40:000
931899220,21cyclictest31271-21apt-get19:55:380
931899220,21cyclictest19895-21perf22:50:010
931899220,0cyclictest24037-21apt-get19:40:410
931899220,0cyclictest2281-21perf22:15:000
9325992120,1cyclictest20185-21apt-get22:50:423
9325992120,1cyclictest18081-21apt-get23:55:383
9325992119,1cyclictest22226-21turbostat21:50:003
932599210,20cyclictest947-21apt-get00:25:403
932599210,20cyclictest5715-21apt-get21:15:393
932599210,20cyclictest24929-21apt-get21:55:373
932599210,20cyclictest1268-21apt-get20:00:403
932599210,20cyclictest0-21swapper/320:05:383
932599210,0cyclictest0-21swapper/323:15:403
932599210,0cyclictest0-21swapper/322:35:393
932599210,0cyclictest0-21swapper/322:30:023
932599210,0cyclictest0-21swapper/300:10:023
9324992120,1cyclictest10889-21apt-get20:20:392
9324992116,4cyclictest0-21swapper/219:55:002
932499210,20cyclictest30938-21apt-get21:00:392
932499210,20cyclictest21618-21apt-get19:35:372
932499210,20cyclictest0-21swapper/222:50:422
932499210,20cyclictest0-21swapper/222:05:402
932499210,20cyclictest0-21swapper/221:50:412
932499210,20cyclictest0-21swapper/219:30:372
932499210,20cyclictest0-21swapper/200:10:452
932499210,1cyclictest17787-21apt-get22:45:382
932499210,1cyclictest0-21swapper/221:45:372
9321992121,0cyclictest0-21swapper/122:25:381
9321992121,0cyclictest0-21swapper/121:55:371
9321992121,0cyclictest0-21swapper/120:25:371
9321992119,1cyclictest3013-21dump-pmu-power21:10:011
9321992119,1cyclictest23712-21cron19:40:001
932199210,3cyclictest151rcu_preempt23:30:021
932199210,20cyclictest0-21swapper/122:50:421
932199210,20cyclictest0-21swapper/100:10:451
932199210,1cyclictest8145-21cron20:15:001
932199210,1cyclictest7808-21dump-pmu-power21:20:001
932199210,0cyclictest20140-21apt-get21:45:371
932199210,0cyclictest0-21swapper/100:25:401
9318992121,0cyclictest0-21swapper/000:05:400
9318992121,0cyclictest0-21swapper/000:05:400
9318992119,1cyclictest30944-21cron19:55:000
931899210,20cyclictest0-21swapper/023:25:420
931899210,20cyclictest0-21swapper/019:10:380
931899210,0cyclictest0-21swapper/023:20:400
931899210,0cyclictest0-21swapper/023:20:400
931899210,0cyclictest0-21swapper/022:55:360
931899210,0cyclictest0-21swapper/020:35:370
9325992020,0cyclictest0-21swapper/323:25:003
9325992020,0cyclictest0-21swapper/323:25:003
9325992020,0cyclictest0-21swapper/322:00:393
9325992020,0cyclictest0-21swapper/321:05:393
9325992018,1cyclictest27099-21munin-plugin-st23:05:003
9325992017,2cyclictest2271-21idleruntime-cro22:15:003
932599200,20cyclictest0-21swapper/323:25:383
932599200,20cyclictest0-21swapper/300:25:013
932599200,19cyclictest5759-21apt-get00:35:393
932599200,0cyclictest0-21swapper/321:35:363
932599200,0cyclictest0-21swapper/319:30:373
932599200,0cyclictest0-21swapper/300:05:403
932599200,0cyclictest0-21swapper/300:05:403
9324992020,0cyclictest0-21swapper/221:25:402
9324992020,0cyclictest0-21swapper/219:10:382
9324992020,0cyclictest0-21swapper/200:35:392
9324992019,1cyclictest26430-21apt-get19:45:402
9324992019,1cyclictest2574-21apt-get22:15:392
9324992019,1cyclictest0-21swapper/200:30:002
9324992019,0cyclictest0-21swapper/222:55:372
932499200,20cyclictest0-21swapper/223:35:402
932499200,0cyclictest24989-21apt-get23:00:372
932499200,0cyclictest0-21swapper/221:30:432
932499200,0cyclictest0-21swapper/220:40:382
932499200,0cyclictest0-21swapper/220:30:382
9321992020,0cyclictest22887-21apt-get20:45:371
9321992020,0cyclictest0-21swapper/122:00:391
9321992020,0cyclictest0-21swapper/119:30:371
9321992020,0cyclictest0-21swapper/119:25:381
9321992020,0cyclictest0-21swapper/100:35:391
9321992020,0cyclictest0-21swapper/100:20:401
9321992020,0cyclictest0-21swapper/100:05:021
9321992020,0cyclictest0-21swapper/100:05:021
9321992019,0cyclictest29777-21apt-get23:10:371
9321992019,0cyclictest10228-21apt-get23:35:391
9321992019,0cyclictest0-21swapper/121:10:391
9321992018,1cyclictest24599-21cron21:55:001
932199200,20cyclictest0-21swapper/122:10:011
932199200,20cyclictest0-21swapper/100:30:431
932199200,0cyclictest3672-21apt-get20:05:381
932199200,0cyclictest28822-21apt-get19:50:381
9318992020,0cyclictest0-21swapper/021:45:370
9318992020,0cyclictest0-21swapper/021:30:430
9318992020,0cyclictest0-21swapper/020:50:400
9318992020,0cyclictest0-21swapper/020:25:370
9318992020,0cyclictest0-21swapper/020:20:390
9318992020,0cyclictest0-21swapper/000:25:400
9318992020,0cyclictest0-21swapper/000:20:400
9318992019,1cyclictest7846-21awk21:20:000
9318992019,1cyclictest26079-21apt-get00:10:020
9318992019,1cyclictest0-21swapper/022:15:390
9318992019,1cyclictest0-21swapper/020:45:370
9318992019,0cyclictest0-21swapper/021:40:380
9318992019,0cyclictest0-21swapper/020:05:380
9318992018,1cyclictest3382-21munin-run20:05:000
931899200,20cyclictest0-21swapper/000:00:380
931899200,19cyclictest0-21swapper/000:15:450
931899200,0cyclictest15679-21apt-get23:50:360
931899200,0cyclictest0-21swapper/021:00:390
9325991919,0cyclictest14433-21apt-get19:20:383
9325991919,0cyclictest12644-21apt-get23:40:393
9325991919,0cyclictest0-21swapper/321:10:393
9325991919,0cyclictest0-21swapper/320:25:373
9325991919,0cyclictest0-21swapper/319:50:383
9325991919,0cyclictest0-21swapper/319:15:393
9325991919,0cyclictest0-21swapper/319:15:393
9325991917,1cyclictest4993-21apt-get22:20:403
9325991917,1cyclictest31925-21apt-get23:15:003
9324991919,0cyclictest5030-21apt-get23:25:382
9324991919,0cyclictest0-21swapper/222:25:382
9324991919,0cyclictest0-21swapper/220:50:022
9321991919,0cyclictest0-21swapper/120:55:021
9321991919,0cyclictest0-21swapper/120:20:381
9321991919,0cyclictest0-21swapper/119:20:381
9321991919,0cyclictest0-21swapper/119:15:381
9321991919,0cyclictest0-21swapper/119:15:381
9321991918,1cyclictest12270-21apt-get22:35:021
9321991918,1cyclictest0-21swapper/120:00:401
9318991919,0cyclictest21618-21apt-get19:35:370
9318991919,0cyclictest0-21swapper/023:35:400
9318991919,0cyclictest0-21swapper/022:25:380
9318991919,0cyclictest0-21swapper/022:05:400
9318991919,0cyclictest0-21swapper/021:05:390
9318991919,0cyclictest0-21swapper/020:55:360
9318991918,1cyclictest24624-21turbostat.cron21:55:000
9325991818,0cyclictest0-21swapper/323:30:433
9325991818,0cyclictest0-21swapper/321:50:413
9325991818,0cyclictest0-21swapper/320:50:003
9325991818,0cyclictest0-21swapper/320:15:393
9324991818,0cyclictest0-21swapper/222:15:002
9324991818,0cyclictest0-21swapper/221:20:392
9324991818,0cyclictest0-21swapper/219:40:412
9324991816,1cyclictest0-21swapper/200:25:012
932499180,18cyclictest0-21swapper/220:25:372
9321991818,0cyclictest0-21swapper/123:15:391
9321991818,0cyclictest0-21swapper/122:15:381
9321991818,0cyclictest0-21swapper/120:35:371
9318991818,0cyclictest0-21swapper/023:00:370
9318991818,0cyclictest0-21swapper/020:15:400
9318991817,0cyclictest0-21swapper/022:20:390
931899180,17cyclictest0-21swapper/022:55:000
9325991717,0cyclictest8135-21apt-get21:20:393
9325991717,0cyclictest0-21swapper/322:40:393
9325991717,0cyclictest0-21swapper/320:10:393
9325991717,0cyclictest0-21swapper/319:45:403
9325991717,0cyclictest0-21swapper/319:40:413
9325991715,1cyclictest30684-21awk00:20:003
932599170,16cyclictest20496-21apt-get20:40:383
9324991717,0cyclictest0-21swapper/223:50:362
9324991717,0cyclictest0-21swapper/200:00:382
9324991716,1cyclictest31271-21apt-get19:55:372
9324991715,1cyclictest5459-21turbostat00:35:002
9321991717,0cyclictest0-21swapper/123:40:391
9321991717,0cyclictest0-21swapper/123:25:381
932199171,15cyclictest19913-21cat22:50:011
932199170,1cyclictest549-21cron21:05:001
9318991717,0cyclictest10192-21apt-get22:30:430
9318991717,0cyclictest0-21swapper/023:05:410
9318991717,0cyclictest0-21swapper/021:20:390
9318991716,0cyclictest0-21swapper/023:55:380
9318991715,1cyclictest16826-21apt-get19:25:380
9325991616,0cyclictest0-21swapper/323:35:393
9325991616,0cyclictest0-21swapper/322:05:403
9325991615,1cyclictest7384-21apt-get22:25:383
9325991614,1cyclictest18905-21latency_hist19:30:003
9324991616,0cyclictest0-21swapper/220:35:372
9324991616,0cyclictest0-21swapper/200:05:022
9324991616,0cyclictest0-21swapper/200:05:022
9324991615,0cyclictest0-21swapper/220:45:382
9324991614,1cyclictest11963-21apt-get19:15:382
9324991614,1cyclictest11963-21apt-get19:15:382
9321991616,0cyclictest0-21swapper/123:05:411
9321991614,1cyclictest0-21swapper/120:30:401
9318991616,0cyclictest0-21swapper/023:50:000
9318991616,0cyclictest0-21swapper/020:30:380
9318991615,0cyclictest0-21swapper/023:15:020
9318991613,3cyclictest10539-21apt-get21:25:400
9318991613,3cyclictest0-21swapper/023:30:440
931899160,15cyclictest0-21swapper/023:10:380
931899160,0cyclictest0-21swapper/022:40:140
9325991515,0cyclictest0-21swapper/320:30:383
9325991514,0cyclictest0-21swapper/321:40:383
9324991515,0cyclictest0-21swapper/221:15:392
9324991514,0cyclictest17747-21apt-get21:40:382
9324991513,2cyclictest0-21swapper/220:15:382
932499150,15cyclictest0-21swapper/222:20:392
932499150,15cyclictest0-21swapper/221:05:012
9321991515,0cyclictest0-21swapper/100:15:391
9321991513,1cyclictest0-21swapper/119:45:401
932199150,1cyclictest9567-21apt-get19:10:381
9318991514,1cyclictest17394-21cron21:40:000
9318991514,1cyclictest0-21swapper/019:20:380
932599140,14cyclictest0-21swapper/323:45:463
932599140,14cyclictest0-21swapper/321:25:163
932599140,14cyclictest0-21swapper/320:52:013
932599140,14cyclictest0-21swapper/319:35:023
932599140,0cyclictest0-21swapper/323:50:543
9324991414,0cyclictest0-21swapper/223:31:072
9324991414,0cyclictest0-21swapper/220:12:192
9324991413,1cyclictest20186-40mandb00:00:002
932499140,14cyclictest0-21swapper/223:44:592
932499140,14cyclictest0-21swapper/223:11:202
932499140,14cyclictest0-21swapper/222:02:082
932499140,14cyclictest0-21swapper/221:55:412
932499140,14cyclictest0-21swapper/221:35:262
932499140,14cyclictest0-21swapper/220:05:312
932499140,14cyclictest0-21swapper/219:25:352
932499140,14cyclictest0-21swapper/219:24:492
932499140,0cyclictest0-21swapper/221:10:282
932499140,0cyclictest0-21swapper/220:55:322
9321991414,0cyclictest0-21swapper/123:50:371
9318991414,0cyclictest0-21swapper/023:40:390
9318991414,0cyclictest0-21swapper/022:00:040
9318991414,0cyclictest0-21swapper/021:55:590
9318991414,0cyclictest0-21swapper/020:10:190
931899140,14cyclictest0-21swapper/021:11:380
931899140,14cyclictest0-21swapper/020:40:120
9325991313,0cyclictest0-21swapper/322:45:483
9324991313,0cyclictest0-21swapper/223:45:442
9321991313,0cyclictest0-21swapper/123:45:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional