You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-01 - 01:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Aug 31, 2024 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4013168995645,9cyclictest88326-21latency_hist11:30:011
4013170994744,2cyclictest4018313-21kworker/u8:0+flush-259:009:20:012
4013173994640,4cyclictest4038803-21latency_hist07:35:003
4013173994336,5cyclictest4095898-21latency_hist08:30:003
4013170994238,3cyclictest4106443-21apt-get08:40:122
4013173993933,4cyclictest4178726-21latency_hist09:50:013
4013163993934,4cyclictest46923-21apt-get10:50:100
4013163993933,4cyclictest4085564-21latency_hist08:20:010
4013163993933,4cyclictest4085564-21latency_hist08:20:000
4013170993532,2cyclictest4142266-21kworker/u8:1+events_unbound09:20:232
4013168993529,4cyclictest129981-21latency_hist12:10:001
401316399350,33cyclictest1-21systemd07:13:390
4013163993431,2cyclictest4018313-21kworker/u8:0+flush-259:009:10:170
4013163993430,3cyclictest36291-21kworker/u8:2+flush-259:010:55:190
4013163993429,4cyclictest57199-21/usr/sbin/munin11:00:140
4013173993329,3cyclictest104269-21kworker/u8:0+flush-259:011:55:253
4013170993328,3cyclictest4064772-21kworker/u8:1+flush-259:008:40:012
401316399330,31cyclictest1-21systemd10:34:400
4013173993229,2cyclictest4018313-21kworker/u8:0+flush-259:008:40:253
4013170993228,3cyclictest3960272-21kworker/u8:2+flush-259:007:25:172
4013163993227,3cyclictest156081-21apt-get12:35:120
4013170993128,2cyclictest4183892-21kworker/u8:1+flush-259:010:30:192
4013170993127,3cyclictest4064772-21kworker/u8:1+events_unbound08:10:192
4013168993126,3cyclictest4183892-21kworker/u8:1+flush-259:010:00:151
4013163993126,3cyclictest104269-21kworker/u8:0+flush-259:011:55:160
4013163993122,5cyclictest254-21systemd-journal08:00:000
4013173993025,3cyclictest4080258-21store08:15:003
4013170993026,3cyclictest4183892-21kworker/u8:1+flush-259:010:35:242
4013163993025,3cyclictest104269-21kworker/u8:0+flush-259:012:25:000
4013173992925,3cyclictest4183892-21kworker/u8:1+flush-259:010:25:243
4013173992923,4cyclictest4152754-21cp09:25:003
4013170992925,3cyclictest4018313-21kworker/u8:0+flush-259:009:15:002
4013168992923,4cyclictest254-21systemd-journal08:43:391
4013168992923,4cyclictest145687-21/usr/sbin/munin12:25:201
4013173992825,2cyclictest4122333-21kworker/u8:3+flush-259:009:00:253
4013170992825,2cyclictest56981-21kworker/u8:1+events_unbound11:20:232
4013170992823,3cyclictest56981-21kworker/u8:1+events_unbound11:15:132
4013163992824,3cyclictest254-21systemd-journal08:34:400
4013173992724,2cyclictest4059428-21kworker/u8:3+flush-259:007:55:243
4013173992723,3cyclictest67513-21kworker/u8:2+flush-259:011:50:013
4013170992723,3cyclictest4064772-21kworker/u8:1+events_unbound08:50:152
4013170992720,5cyclictest254-21systemd-journal07:55:002
4013170992622,3cyclictest124840-21kworker/u8:1+events_unbound12:18:422
4013163992622,3cyclictest3960272-21kworker/u8:2+flush-259:007:25:080
4013163992621,4cyclictest254-21systemd-journal08:35:250
4013163992620,5cyclictest4142668-21diskstats09:15:140
4013163992619,5cyclictest4053387-21ntpq07:45:230
4013173992517,6cyclictest254-21systemd-journal07:40:003
4013170992521,3cyclictest4132194-21kworker/u8:4+events_unbound09:39:402
4013170992521,3cyclictest4116153-21kworker/u8:2+events_unbound10:01:392
4013170992521,3cyclictest4116153-21kworker/u8:2+events_unbound08:45:392
4013170992520,4cyclictest4132194-21kworker/u8:4+events_unbound09:29:412
4013168992520,3cyclictest4132194-21kworker/u8:4+flush-259:009:40:001
4013163992520,3cyclictest4189224-21kworker/u8:3+events_unbound10:15:150
4013163992518,5cyclictest4177878-21ntpq09:45:220
401316399250,23cyclictest1-21systemd07:15:390
4013173992421,2cyclictest4116153-21kworker/u8:2+flush-259:009:30:233
4013170992421,2cyclictest4064772-21kworker/u8:1+flush-259:008:05:202
4013170992421,2cyclictest3955068-21kworker/u8:1+flush-259:007:20:212
4013170992420,3cyclictest4132194-21kworker/u8:4+events_unbound09:08:402
4013170992420,3cyclictest4090951-21kworker/u8:4+events_unbound08:31:402
4013170992419,4cyclictest104269-21kworker/u8:0+events_unbound12:05:202
4013170992418,2cyclictest56981-21kworker/u8:1+flush-259:011:15:002
4013163992420,3cyclictest56981-21kworker/u8:1+events_unbound11:15:150
4013163992417,5cyclictest5441-21latency_hist10:10:010
4013173992318,3cyclictest88440-21gpgv11:30:003
4013170992319,3cyclictest67513-21kworker/u8:2+events_unbound11:46:032
4013170992319,3cyclictest4183892-21kworker/u8:1+events_unbound10:22:402
4013170992319,3cyclictest4116153-21kworker/u8:2+flush-259:009:45:252
4013170992316,5cyclictest254-21systemd-journal10:09:592
4013168992317,4cyclictest4131110-21ntpq09:00:231
4013163992320,2cyclictest56981-21kworker/u8:1+events_unbound11:30:010
4013163992319,3cyclictest4183892-21kworker/u8:1+events_unbound10:35:140
4013163992318,4cyclictest36532-21apt-get10:40:100
4013163992317,4cyclictest25109-21ntpq10:25:190
4013173992217,4cyclictest4116153-21kworker/u8:2+flush-259:009:05:173
4013173992217,4cyclictest4074875-21kworker/u8:2+flush-259:008:20:003
4013173992217,4cyclictest4074875-21kworker/u8:2+flush-259:008:20:003
4013170992219,2cyclictest4132194-21kworker/u8:4+events_unbound09:55:232
4013170992219,2cyclictest4018313-21kworker/u8:0+flush-259:008:55:152
4013170992218,3cyclictest88347-21kworker/u8:0+events_unbound11:31:422
4013170992218,3cyclictest78230-21kworker/u8:4+events_unbound11:39:412
4013170992218,3cyclictest4189224-21kworker/u8:3+events_unbound10:45:112
4013170992218,3cyclictest4189224-21kworker/u8:3+events_unbound10:19:392
4013170992218,3cyclictest36291-21kworker/u8:2+events_unbound10:55:012
4013170992217,4cyclictest67513-21kworker/u8:2+events_unbound11:56:062
4013170992217,4cyclictest56982-21kworker/u8:4+flush-259:011:05:162
4013170992217,4cyclictest56981-21kworker/u8:1+events_unbound11:28:402
4013170992216,4cyclictest4080625-21kworker/u8:3+events_unbound08:15:182
4013170992216,4cyclictest4080625-21kworker/u8:3+events_unbound08:15:182
4013168992218,3cyclictest104269-21kworker/u8:0+flush-259:012:35:001
4013168992217,4cyclictest4028808-21/usr/sbin/munin07:25:141
4013163992218,2cyclictest78230-21kworker/u8:4+flush-259:011:35:170
4013163992216,4cyclictest4064816-21apt-get08:00:130
4013163992215,5cyclictest72608-21cp11:15:010
4013170992118,2cyclictest4116153-21kworker/u8:2+events_unbound09:40:402
4013170992118,2cyclictest160962-21latency_hist12:40:012
4013170992117,3cyclictest4183892-21kworker/u8:1+flush-259:010:25:262
4013170992117,3cyclictest4018313-21kworker/u8:0+events_unbound08:20:242
401317099211,19cyclictest1-21systemd08:00:002
4013168992115,4cyclictest4106181-21latency_hist08:40:001
4013168992115,4cyclictest140529-21apt-get12:20:131
401316899211,18cyclictest4101076-21awk08:35:011
4013163992117,3cyclictest4153221-21kworker/u8:3+flush-259:009:25:180
4013163992116,4cyclictest67513-21kworker/u8:2+flush-259:012:01:080
4013163992116,4cyclictest254-21systemd-journal12:35:000
4013163992116,3cyclictest4116153-21kworker/u8:2+flush-259:009:55:150
4013163992115,4cyclictest254-21systemd-journal11:24:400
4013170992017,2cyclictest4116153-21kworker/u8:2+events_unbound09:30:162
4013170992017,2cyclictest3960272-21kworker/u8:2+flush-259:007:45:202
4013170992016,3cyclictest4183892-21kworker/u8:1+events_unbound10:12:412
4013170992016,3cyclictest3955068-21kworker/u8:1+events_unbound07:10:132
4013170992015,4cyclictest4090959-21/usr/sbin/munin08:25:142
4013168992017,2cyclictest93879-21sh11:35:011
4013168992015,4cyclictest4116153-21kworker/u8:2+flush-259:009:35:001
4013163992016,3cyclictest4183892-21kworker/u8:1+events_unbound10:10:230
4013163992016,3cyclictest4064772-21kworker/u8:1+flush-259:008:05:240
4013163992016,3cyclictest4018313-21kworker/u8:0+flush-259:007:35:180
4013163992013,5cyclictest150609-21latency_hist12:29:590
4013173991916,2cyclictest56981-21kworker/u8:1+events_unbound11:15:233
4013173991916,2cyclictest4018313-21kworker/u8:0+flush-259:007:45:283
4013173991916,2cyclictest36291-21kworker/u8:2+events_unbound11:00:213
4013173991916,2cyclictest140493-21kworker/u8:4+flush-259:012:35:253
4013173991915,3cyclictest78230-21kworker/u8:4+flush-259:011:20:263
4013173991915,3cyclictest4064772-21kworker/u8:1+flush-259:008:00:153
4013173991914,4cyclictest4111624-21apt-get08:45:103
4013173991914,4cyclictest4096166-21apt-get08:30:123
4013173991913,4cyclictest4121693-21latency_hist08:55:013
4013170991916,2cyclictest4064772-21kworker/u8:1+flush-259:008:00:252
4013170991915,3cyclictest3960272-21kworker/u8:2+events_unbound07:34:402
401316899199,9cyclictest254-21systemd-journal12:39:401
4013168991911,5cyclictest67520-21/usr/sbin/munin11:10:131
4013163991915,3cyclictest3955068-21kworker/u8:1+flush-259:007:25:000
4013173991815,2cyclictest129986-21munin-plugin-st12:09:593
4013173991814,3cyclictest124840-21kworker/u8:1+flush-259:012:10:273
4013173991813,4cyclictest119971-21apt-get12:00:133
4013173991813,3cyclictest254-21systemd-journal10:05:453
4013170991814,3cyclictest4018313-21kworker/u8:0+events_unbound07:35:232
4013170991814,3cyclictest140493-21kworker/u8:4+events_unbound12:20:212
4013168991815,2cyclictest4074875-21kworker/u8:2+flush-259:008:20:161
4013168991813,3cyclictest254-21systemd-journal11:50:411
4013168991812,5cyclictest254-21systemd-journal09:45:011
4013168991812,4cyclictest4137133-21latency_hist09:10:011
401316899181,1cyclictest0-21swapper/109:55:141
401316899181,16cyclictest1-21systemd12:15:401
4013168991811,5cyclictest1-21systemd08:15:011
4013163991814,3cyclictest56981-21kworker/u8:1+events_unbound11:25:150
4013163991813,3cyclictest4132194-21kworker/u8:4+events_unbound09:35:170
4013163991811,5cyclictest254-21systemd-journal12:06:400
4013173991713,3cyclictest4018313-21kworker/u8:0+flush-259:008:20:243
4013173991713,3cyclictest140493-21kworker/u8:4+flush-259:012:30:163
4013168991713,3cyclictest4013368-21/usr/sbin/munin07:10:191
4013168991713,2cyclictest4074832-21sh08:10:001
4013168991712,4cyclictest4153255-21apt-get09:25:131
4013168991711,4cyclictest254-21systemd-journal07:35:391
4013168991710,5cyclictest4080625-21kworker/u8:3+flush-259:008:15:161
4013168991710,5cyclictest4080625-21kworker/u8:3+flush-259:008:15:161
401316899170,9cyclictest104007-21runrttasks11:42:151
4013163991714,2cyclictest4116153-21kworker/u8:2+events_unbound09:05:180
4013163991714,2cyclictest4064772-21kworker/u8:1+flush-259:008:25:240
4013163991713,3cyclictest67513-21kworker/u8:2+flush-259:011:45:150
4013163991713,3cyclictest4122333-21kworker/u8:3+events_unbound09:00:240
4013173991612,3cyclictest10599-21kworker/u8:0+flush-259:010:45:253
4013173991612,3cyclictest104269-21kworker/u8:0+flush-259:012:25:183
4013170991613,2cyclictest67513-21kworker/u8:2+flush-259:012:01:102
4013170991612,3cyclictest67513-21kworker/u8:2+events_unbound11:50:022
4013170991612,3cyclictest36291-21kworker/u8:2+events_unbound11:04:422
4013170991612,3cyclictest140493-21kworker/u8:4+events_unbound12:26:082
4013170991612,2cyclictest4132194-21kworker/u8:4+events_unbound09:50:172
401316899168,7cyclictest503-21systemd-logind10:40:011
4013168991612,2cyclictest0-21swapper/107:24:591
401316899161,13cyclictest4137798-21sed09:10:161
4013168991610,4cyclictest10599-21kworker/u8:0+flush-259:010:45:251
401316899160,4cyclictest4023287-21seq07:18:341
401316899160,14cyclictest4152878-21latency_hist09:25:011
401316899160,14cyclictest4095911-21awk08:30:001
401316899160,13cyclictest94340-21cut11:35:141
401316899160,13cyclictest587-21tr10:05:121
401316899160,13cyclictest4179081-21cpuspeed_turbos09:50:141
401316899160,13cyclictest4122032-21cpuspeed_turbos08:55:121
401316899160,13cyclictest4049502-21cpuspeed_turbos07:45:151
401316899160,13cyclictest130323-21tr12:10:131
401316399169,5cyclictest4089883-21ntpq08:20:210
4013163991613,2cyclictest60650irq/133-enp0s29f1:rx-009:20:020
4013163991613,2cyclictest4132194-21kworker/u8:4+events_unbound09:50:220
4013163991611,4cyclictest67288-21latency_hist11:10:000
4013163991611,4cyclictest124840-21kworker/u8:1+events_unbound12:15:190
4013163991611,3cyclictest4049112-21turbostat.cron07:45:010
4013173991512,2cyclictest4183892-21kworker/u8:1+flush-259:010:30:173
4013173991512,2cyclictest4018313-21kworker/u8:0+flush-259:008:35:263
4013173991511,3cyclictest56981-21kworker/u8:1+flush-259:011:30:163
4013170991511,3cyclictest4122333-21kworker/u8:3+events_unbound09:00:212
401316899158,6cyclictest0-21swapper/110:40:131
401316899153,10cyclictest516-21gdbus12:05:001
401316899151,13cyclictest4065225-21sed08:00:181
401316899151,13cyclictest254-21systemd-journal11:25:001
401316899151,13cyclictest119714-21apt12:00:011
401316899151,12cyclictest56953-21idleruntime-cro10:59:591
401316899151,12cyclictest509-21rs:main1
401316899151,12cyclictest4117803-21ps08:50:161
401316899151,12cyclictest4060081-21ps07:55:171
401316899151,12cyclictest4054942-21irqcore07:50:161
401316899151,12cyclictest4044354-21cpuspeed_turbos07:40:121
401316899151,12cyclictest14903-21tr10:15:211
401316899151,12cyclictest104727-21cut11:45:131
401316899151,11cyclictest26391-21grep10:30:131
401316899150,6cyclictest4146501-21ntpq09:15:201
401316899150,14cyclictest62146-21idleruntime-cro11:05:001
401316899150,13cyclictest73189-21tr11:15:111
401316899150,13cyclictest66274-21tail11:05:181
401316899150,13cyclictest5799-21grep10:10:131
4013163991514,1cyclictest60650irq/133-enp0s29f1:rx-008:50:220
4013163991513,1cyclictest60650irq/133-enp0s29f1:rx-011:40:470
4013163991513,1cyclictest60650irq/133-enp0s29f1:rx-009:30:200
4013163991513,1cyclictest60650irq/133-enp0s29f1:rx-007:30:190
4013163991512,2cyclictest60650irq/133-enp0s29f1:rx-008:43:100
4013163991512,2cyclictest124840-21kworker/u8:1+flush-259:012:11:110
4013163991511,3cyclictest4116153-21kworker/u8:2+flush-259:009:40:160
401317399149,4cyclictest56063-21ntpq10:55:233
401317399149,4cyclictest3955068-21kworker/u8:1+flush-259:007:20:163
4013173991411,2cyclictest72000-21unixbench_singl11:10:233
4013173991411,2cyclictest50893-21ntp_kernel_err10:50:203
4013173991411,2cyclictest110115-21irqcore11:50:163
401317099149,4cyclictest254-21systemd-journal12:10:402
4013170991411,2cyclictest4049092-21idleruntime-cro07:45:002
401316899141,2cyclictest20001-21ntpq10:20:211
401316899141,11cyclictest21215-21grep10:25:111
401316899140,4cyclictest4174106-21tune2fs09:45:151
401316899140,2cyclictest4111865-21tune2fs08:45:131
401316899140,2cyclictest4013187-21latency_hist07:10:011
401316899140,11cyclictest40339622chrt07:30:121
4013163991412,2cyclictest60650irq/133-enp0s29f1:rx-008:10:050
4013163991412,1cyclictest60650irq/133-enp0s29f1:rx-008:45:220
4013163991411,2cyclictest60650irq/133-enp0s29f1:rx-010:47:130
4013163991411,2cyclictest60650irq/133-enp0s29f1:rx-010:20:160
4013163991411,2cyclictest60650irq/133-enp0s29f1:rx-010:03:370
4013163991411,2cyclictest60650irq/133-enp0s29f1:rx-008:55:210
401317399139,3cyclictest4183892-21kworker/u8:1+flush-259:010:00:233
4013173991310,2cyclictest4049102-21idleruntime-cro07:45:003
4013163991310,2cyclictest60650irq/133-enp0s29f1:rx-011:51:240
4013163991310,2cyclictest60650irq/133-enp0s29f1:rx-007:51:120
401317399129,2cyclictest140952-21sed12:20:163
401317399128,3cyclictest94139-21apt-get11:35:123
401317399127,3cyclictest4054534-21apt-get07:50:123
401317399125,5cyclictest4013112-21latency_hist07:10:023
4013173991210,1cyclictest0-21swapper/310:19:593
401317099128,3cyclictest4183892-21kworker/u8:1+events_unbound10:40:012
401317399118,2cyclictest103798-21unixbench_singl11:40:253
401317399118,2cyclictest0-21swapper/310:20:233
401317399115,4cyclictest4178969-21/usr/sbin/munin09:50:143
401317399111,2cyclictest41448-21awk10:45:003
401317099118,2cyclictest78230-21kworker/u8:4+events_unbound11:40:142
401317399106,3cyclictest4018313-21kworker/u8:0+events_unbound09:10:183
40131739997,1cyclictest0-21swapper/309:40:233
40131739997,1cyclictest0-21swapper/309:35:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional