You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 14:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sat Jul 27, 2024 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1682525995551,3cyclictest73-21kswapd023:55:003
1682518994841,5cyclictest1860447-21latency_hist21:45:011
1682525994541,3cyclictest1744422-21kworker/u8:1+events_unbound20:40:043
1682522994541,3cyclictest73-21kswapd022:40:002
1682522994437,5cyclictest2065270-21cp00:00:012
1682515994440,3cyclictest1890501-21kworker/u8:3+flush-259:022:05:140
1682522994138,2cyclictest1964482-21kworker/u8:0+events_unbound23:05:242
1682522994138,2cyclictest1964482-21kworker/u8:0+events_unbound23:05:232
1682525993936,2cyclictest1898561-21kworker/u8:2+flush-259:022:29:383
1682522993933,4cyclictest1801684-21latency_hist21:05:012
1682518993934,4cyclictest73-21kswapd000:10:001
1682522993834,3cyclictest2055270-21kworker/u8:3+events_unbound00:25:162
1682522993834,3cyclictest2027434-21kworker/u8:2+events_unbound23:50:202
1682522993832,4cyclictest2019903-21latency_hist23:30:002
1682522993731,4cyclictest1891023-21latency_hist22:05:002
1682515993734,2cyclictest17288962sleep019:55:000
1682515993632,3cyclictest1838051-21apt-get21:30:140
1682515993531,3cyclictest1972610-21kworker/u8:1+flush-259:023:04:240
1682522993431,2cyclictest2776117-21kworker/u8:0+events_unbound19:40:132
1682518993429,3cyclictest1791611-21apt-get20:55:131
1682515993431,2cyclictest1944271-21kworker/u8:4+flush-259:022:50:000
1682522993330,2cyclictest1914057-21kworker/u8:1+flush-259:022:20:232
1682522993330,2cyclictest1818989-21kworker/u8:3+flush-259:021:20:182
1682522993330,2cyclictest1818989-21kworker/u8:3+events_unbound21:25:292
1682522993330,2cyclictest1718427-21kworker/u8:3+events_unbound20:59:312
1682522993329,3cyclictest1898561-21kworker/u8:2+events_unbound22:53:332
1682525993229,2cyclictest73-21kswapd000:15:003
1682522993229,2cyclictest2055270-21kworker/u8:3+flush-259:000:35:232
1682522993229,2cyclictest1844548-21kworker/u8:2+events_unbound21:35:142
1682522993229,2cyclictest1828752-21kworker/u8:1+flush-259:021:40:262
1682515993229,2cyclictest2086741-21kworker/u8:2+flush-259:000:35:280
1682525993128,2cyclictest73-21kswapd023:40:003
1682522993128,2cyclictest1890501-21kworker/u8:3+flush-259:022:25:282
1682522993128,2cyclictest1818989-21kworker/u8:3+events_unbound21:30:172
1682522993126,3cyclictest1857351-21kworker/u8:0+events_unbound22:05:262
1682518993127,3cyclictest2776117-21kworker/u8:0+flush-259:019:55:321
1682518993126,4cyclictest255-21systemd-journal19:30:311
1682515993128,2cyclictest2039465-21kworker/u8:0+flush-259:000:05:030
1682515993126,4cyclictest255-21systemd-journal21:25:000
1682525993026,3cyclictest255-21systemd-journal23:27:313
1682525993025,3cyclictest1729338-21diskstats19:55:143
1682522993027,2cyclictest2055270-21kworker/u8:3+events_unbound00:16:302
1682522993027,2cyclictest1857351-21kworker/u8:0+flush-259:021:55:232
1682522993027,2cyclictest1718427-21kworker/u8:3+flush-259:020:10:262
1682522993026,3cyclictest2003031-21kworker/u8:0+events_unbound23:20:542
1682522993026,3cyclictest1865922-21kworker/u8:2+flush-259:021:46:132
1682522993026,3cyclictest1677540-21kworker/u8:1+flush-259:020:00:132
1682525992923,5cyclictest1972610-21kworker/u8:1+flush-259:023:15:263
1682525992923,4cyclictest1749589-21latency_hist20:15:013
1682522992927,2cyclictest2055270-21kworker/u8:3+flush-259:000:10:132
1682522992927,2cyclictest2039465-21kworker/u8:0+flush-259:000:05:172
1682522992927,2cyclictest1718427-21kworker/u8:3+events_unbound20:25:012
1682522992926,3cyclictest2039465-21kworker/u8:0+flush-259:023:46:332
1682522992926,2cyclictest1806906-21kworker/u8:1+flush-259:021:14:262
1682522992925,3cyclictest73-21kswapd022:35:002
1682522992925,3cyclictest2086741-21kworker/u8:2+events_unbound00:22:212
1682522992925,3cyclictest2003031-21kworker/u8:0+events_unbound23:30:342
1682522992925,3cyclictest1972610-21kworker/u8:1+flush-259:023:10:012
1682522992925,3cyclictest1857351-21kworker/u8:0+flush-259:022:10:462
1682518992923,4cyclictest509-21rs:main1
1682515992927,1cyclictest2086741-21kworker/u8:2+flush-259:000:25:200
1682515992926,2cyclictest73-21kswapd023:00:000
1682515992926,2cyclictest2077900-21kworker/u8:1+flush-259:000:15:000
1682522992825,2cyclictest2039465-21kworker/u8:0+flush-259:000:03:212
1682522992825,2cyclictest1890501-21kworker/u8:3+events_unbound22:57:492
1682522992824,3cyclictest1944271-21kworker/u8:4+events_unbound22:40:232
1682522992824,3cyclictest1775633-21kworker/u8:0+events_unbound20:40:242
1682522992824,3cyclictest1667115-21kworker/u8:3+events_unbound19:10:172
1682522992823,4cyclictest1974910-21kworker/u8:4+events_unbound23:40:092
1682522992823,3cyclictest1828752-21kworker/u8:1+events_unbound21:53:172
1682515992825,2cyclictest2110990-21apt-get00:30:020
1682515992825,2cyclictest1972610-21kworker/u8:1+flush-259:023:05:240
1682515992825,2cyclictest1972610-21kworker/u8:1+flush-259:023:05:240
1682525992723,3cyclictest1964482-21kworker/u8:0+flush-259:023:05:373
1682525992723,3cyclictest1964482-21kworker/u8:0+flush-259:023:05:363
1682525992721,4cyclictest1964482-21kworker/u8:0+flush-259:023:00:243
1682522992724,2cyclictest2027434-21kworker/u8:2+flush-259:023:35:322
1682522992723,3cyclictest2003031-21kworker/u8:0+flush-259:023:18:212
1682522992723,3cyclictest1718427-21kworker/u8:3+events_unbound21:05:202
1682522992722,3cyclictest1956914-21ssh22:45:302
1682518992718,7cyclictest0-21swapper/121:30:011
1682518992717,9cyclictest1677540-21kworker/u8:1+flush-259:019:40:001
1682515992724,2cyclictest1857351-21kworker/u8:0+events_unbound21:44:460
1682515992724,2cyclictest1718427-21kworker/u8:3+flush-259:020:40:260
1682525992623,2cyclictest2027434-21kworker/u8:2+flush-259:023:48:033
1682525992622,3cyclictest2039465-21kworker/u8:0+events_unbound00:00:243
1682522992623,2cyclictest2776117-21kworker/u8:0+events_unbound19:33:442
1682522992623,2cyclictest2116642-21kworker/u8:0+flush-259:000:31:042
1682522992623,2cyclictest1890501-21kworker/u8:3+flush-259:022:18:412
1682518992623,2cyclictest73-21kswapd023:30:001
1682518992622,3cyclictest1936259-21ssh22:34:141
1682515992623,2cyclictest1890501-21kworker/u8:3+flush-259:022:10:200
1682525992522,2cyclictest1718427-21kworker/u8:3+flush-259:020:50:203
1682525992521,3cyclictest2003031-21kworker/u8:0+events_unbound23:20:213
1682522992522,2cyclictest1964482-21kworker/u8:0+flush-259:023:00:322
1682522992522,2cyclictest1718427-21kworker/u8:3+flush-259:020:15:242
1682522992522,2cyclictest1718427-21kworker/u8:3+flush-259:020:15:232
1682522992521,3cyclictest1718427-21kworker/u8:3+flush-259:020:45:282
1682518992521,3cyclictest1937061-21/usr/sbin/munin22:35:311
1682518992521,3cyclictest1827195-21ssh21:20:301
1682525992421,2cyclictest1972610-21kworker/u8:1+events_unbound23:30:143
1682525992421,2cyclictest1890501-21kworker/u8:3+flush-259:022:30:133
1682525992421,2cyclictest1718427-21kworker/u8:3+flush-259:020:45:253
1682522992421,2cyclictest1791391-21kworker/u8:2+flush-259:021:15:422
1682522992421,2cyclictest1744422-21kworker/u8:1+flush-259:020:30:252
1682522992420,3cyclictest1744422-21kworker/u8:1+events_unbound20:35:262
1682515992422,2cyclictest1972610-21kworker/u8:1+flush-259:023:44:520
1682515992421,2cyclictest2039465-21kworker/u8:0+flush-259:023:49:510
1682515992421,2cyclictest1898561-21kworker/u8:2+events_unbound22:35:340
1682515992421,2cyclictest1738864-21kworker/u8:2+events_unbound20:10:130
1682515992420,3cyclictest2776117-21kworker/u8:0+flush-259:019:45:280
1682515992420,3cyclictest1744422-21kworker/u8:1+events_unbound20:35:240
1682525992320,2cyclictest1890501-21kworker/u8:3+flush-259:022:10:223
1682525992320,2cyclictest1857351-21kworker/u8:0+flush-259:022:00:143
1682525992319,3cyclictest2039465-21kworker/u8:0+flush-259:023:44:573
1682525992319,3cyclictest1836620-21ssh21:28:543
1682525992319,3cyclictest1677540-21kworker/u8:1+events_unbound19:53:313
1682522992320,2cyclictest1677540-21kworker/u8:1+events_unbound19:15:202
1682522992318,3cyclictest1744397-21latency_hist20:10:002
1682518992319,3cyclictest2025683-21ssh23:31:521
1682518992318,3cyclictest73-21kswapd022:50:101
1682518992316,5cyclictest1974896-21apt-get23:00:111
1682515992320,2cyclictest1865922-21kworker/u8:2+flush-259:021:45:360
1682525992219,2cyclictest73-21kswapd022:59:193
1682525992219,2cyclictest1857351-21kworker/u8:0+flush-259:022:05:263
1682525992218,3cyclictest1744422-21kworker/u8:1+events_unbound20:25:153
1682525992217,4cyclictest1952139-21ssh22:45:093
1682525992217,3cyclictest1868150-21ssh21:49:323
1682522992215,5cyclictest1712550-21ntpq19:35:242
1682518992219,2cyclictest1860446-21apt-get21:45:031
1682518992217,3cyclictest2096260-21apt-get00:20:101
1682515992220,2cyclictest1718427-21kworker/u8:3+events_unbound21:05:140
1682515992219,2cyclictest2776117-21kworker/u8:0+flush-259:019:20:320
1682515992219,2cyclictest2055270-21kworker/u8:3+flush-259:000:15:180
1682515992219,2cyclictest2003031-21kworker/u8:0+flush-259:023:30:240
1682525992119,1cyclictest1865922-21kworker/u8:2+events_unbound21:55:223
1682525992118,2cyclictest2055270-21kworker/u8:3+events_unbound00:21:353
1682525992118,2cyclictest1964482-21kworker/u8:0+events_unbound23:10:253
1682525992118,2cyclictest1898561-21kworker/u8:2+flush-259:022:38:363
1682525992118,2cyclictest1828752-21kworker/u8:1+events_unbound21:40:163
1682525992117,3cyclictest1873755-21ssh21:51:013
1682525992117,3cyclictest1828752-21kworker/u8:1+flush-259:021:38:333
1682525992117,2cyclictest0-21swapper/321:25:013
1682522992118,2cyclictest2776117-21kworker/u8:0+flush-259:019:55:282
1682522992115,4cyclictest255-21systemd-journal19:55:002
1682518992117,3cyclictest73-21kswapd023:50:001
1682518992117,3cyclictest2086508-21ssh00:12:231
1682518992114,5cyclictest509-21rs:main1
1682518992114,4cyclictest2057663-21latency_hist23:55:001
1682518992114,4cyclictest1990008-21ssh23:10:111
1682515992119,1cyclictest73-21kswapd022:55:000
1682515992119,1cyclictest73-21kswapd021:30:010
1682515992118,2cyclictest2055270-21kworker/u8:3+events_unbound00:25:010
1682515992118,2cyclictest1972610-21kworker/u8:1+events_unbound23:25:110
1682515992118,2cyclictest1744422-21kworker/u8:1+flush-259:020:20:180
1682515992118,2cyclictest1744422-21kworker/u8:1+flush-259:020:20:180
1682515992118,2cyclictest1718427-21kworker/u8:3+events_unbound20:45:220
1682515992118,2cyclictest1677540-21kworker/u8:1+events_unbound19:30:150
1682515992117,3cyclictest1972610-21kworker/u8:1+events_unbound23:59:050
1682525992017,2cyclictest2055270-21kworker/u8:3+flush-259:000:27:303
1682525992017,2cyclictest1898561-21kworker/u8:2+flush-259:022:16:573
1682525992015,4cyclictest1739232-21latency_hist20:05:003
1682522992017,2cyclictest1775633-21kworker/u8:0+events_unbound20:50:192
1682522992017,2cyclictest1677540-21kworker/u8:1+flush-259:019:20:292
1682518992016,3cyclictest1926704-21ssh22:25:331
1682518992016,3cyclictest1904658-21ssh22:11:441
1682518992015,3cyclictest1677540-21kworker/u8:1+flush-259:019:45:001
1682515992017,2cyclictest1828752-21kworker/u8:1+events_unbound21:38:010
1682515992017,2cyclictest1744422-21kworker/u8:1+flush-259:020:25:200
1682525991916,2cyclictest2055270-21kworker/u8:3+events_unbound00:09:333
1682525991916,2cyclictest1890501-21kworker/u8:3+flush-259:022:52:323
1682525991916,2cyclictest1677540-21kworker/u8:1+events_unbound19:45:243
1682525991916,2cyclictest1677540-21kworker/u8:1+events_unbound19:34:313
1682525991915,3cyclictest255-21systemd-journal21:14:553
1682525991915,3cyclictest1818989-21kworker/u8:3+flush-259:021:33:363
1682525991914,4cyclictest1914388-21apt-get22:20:103
1682525991914,4cyclictest1770248-21latency_hist20:35:013
1682525991913,4cyclictest1806877-21latency_hist21:10:013
1682518991911,3cyclictest1882273-21ssh21:57:461
1682515991917,2cyclictest1974910-21kworker/u8:4+flush-259:023:39:100
1682515991917,1cyclictest73-21kswapd022:05:000
1682515991916,2cyclictest1890501-21kworker/u8:3+flush-259:022:27:360
1682515991916,2cyclictest1890501-21kworker/u8:3+flush-259:022:20:260
1682515991915,3cyclictest1944271-21kworker/u8:4+events_unbound22:40:170
1682525991815,2cyclictest2086741-21kworker/u8:2+flush-259:000:15:223
1682525991814,3cyclictest1667115-21kworker/u8:3+flush-259:019:13:373
1682522991815,2cyclictest1718427-21kworker/u8:3+events_unbound20:20:142
1682522991815,2cyclictest1718427-21kworker/u8:3+events_unbound20:20:142
1682522991814,3cyclictest2776117-21kworker/u8:0+events_unbound19:45:122
1682522991812,2cyclictest1677540-21kworker/u8:1+events_unbound19:25:222
1682518991814,3cyclictest73-21kswapd000:30:021
1682518991812,4cyclictest1972610-21kworker/u8:1+flush-259:023:20:171
1682515991815,2cyclictest2776117-21kworker/u8:0+flush-259:019:58:380
1682515991815,2cyclictest1806906-21kworker/u8:1+events_unbound21:10:560
1682515991815,2cyclictest1775633-21kworker/u8:0+flush-259:020:50:150
1682515991812,4cyclictest1691884-21ntpq19:15:200
1682525991715,2cyclictest2055270-21kworker/u8:3+flush-259:023:59:203
1682525991715,2cyclictest1744422-21kworker/u8:1+flush-259:020:35:243
1682525991714,2cyclictest2776117-21kworker/u8:0+events_unbound19:40:203
1682525991714,2cyclictest2116642-21kworker/u8:0+flush-259:000:37:473
1682525991714,2cyclictest1890501-21kworker/u8:3+events_unbound22:41:283
1682525991714,2cyclictest1750277-21irqcore20:15:183
1682525991714,2cyclictest1750277-21irqcore20:15:183
1682525991713,3cyclictest2055270-21kworker/u8:3+events_unbound00:33:273
1682525991713,3cyclictest1814872-21apt-get21:15:113
1682518991713,3cyclictest1964482-21kworker/u8:0+events_unbound23:05:201
1682518991713,3cyclictest1964482-21kworker/u8:0+events_unbound23:05:191
1682518991711,4cyclictest1744661-21apt-get20:10:111
1682518991711,4cyclictest1697895-21latency_hist19:25:001
1682515991715,2cyclictest1791391-21kworker/u8:2+flush-259:021:15:370
1682515991714,2cyclictest60650irq/128-enp0s29f1:rx-020:30:220
1682515991714,2cyclictest2055270-21kworker/u8:3+flush-259:000:00:260
1682515991714,2cyclictest1972610-21kworker/u8:1+events_unbound23:19:580
1682515991714,2cyclictest1898561-21kworker/u8:2+flush-259:022:32:290
1682515991714,2cyclictest1828752-21kworker/u8:1+flush-259:021:52:310
1682515991714,2cyclictest1738864-21kworker/u8:2+events_unbound20:05:230
1682515991714,1cyclictest0-21swapper/023:55:000
1682515991713,3cyclictest2005118-21apt-get23:20:120
1682515991713,3cyclictest1890501-21kworker/u8:3+flush-259:022:17:470
1682525991614,1cyclictest1677540-21kworker/u8:1+flush-259:019:20:253
1682525991613,2cyclictest1718427-21kworker/u8:3+events_unbound20:56:323
1682518991613,2cyclictest0-21swapper/122:25:001
1682518991612,2cyclictest0-21swapper/120:45:201
1682518991611,4cyclictest2055270-21kworker/u8:3+flush-259:023:55:171
1682518991611,4cyclictest1813031-21ssh21:12:261
1682518991611,4cyclictest1775633-21kworker/u8:0+flush-259:020:53:021
1682515991613,2cyclictest1964482-21kworker/u8:0+flush-259:023:10:120
1682515991613,2cyclictest1828752-21kworker/u8:1+flush-259:021:55:520
1682525991512,2cyclictest1744422-21kworker/u8:1+events_unbound20:20:233
1682525991512,2cyclictest1744422-21kworker/u8:1+events_unbound20:20:233
1682525991512,2cyclictest1738864-21kworker/u8:2+events_unbound20:05:193
1682525991511,3cyclictest1677540-21kworker/u8:1+events_unbound19:35:203
168251899159,4cyclictest1723959-21/usr/sbin/munin19:50:141
1682518991512,2cyclictest0-21swapper/123:20:011
1682518991511,3cyclictest2111321-21apt-key00:30:021
1682518991511,3cyclictest1957801-21ssh22:47:091
1682518991511,2cyclictest1721124-21kthreadcore19:45:171
1682515991513,1cyclictest60650irq/128-enp0s29f1:rx-019:36:460
1682515991512,2cyclictest1775633-21kworker/u8:0+events_unbound21:03:330
1682515991512,2cyclictest1677540-21kworker/u8:1+events_unbound20:00:270
1682515991511,3cyclictest1667115-21kworker/u8:3+flush-259:019:10:150
1682525991411,2cyclictest1718427-21kworker/u8:3+flush-259:021:00:173
1682525991410,3cyclictest2776117-21kworker/u8:0+flush-259:019:15:313
1682518991411,2cyclictest1682511-21cyclictest22:40:181
168251899140,7cyclictest1775427-21munin-run20:40:001
1682515991412,1cyclictest60650irq/128-enp0s29f1:rx-019:40:130
168252299137,4cyclictest1682430-21latency_hist19:10:002
168251899139,3cyclictest1818989-21kworker/u8:3+flush-259:021:16:441
168251899138,4cyclictest1890772-21ssh22:04:351
168251899138,3cyclictest1667115-21kworker/u8:3+flush-259:019:10:021
168251899131,4cyclictest1718427-21kworker/u8:3+flush-259:020:30:001
1682518991312,1cyclictest311ktimers/100:18:091
1682518991310,2cyclictest311ktimers/123:42:551
1682518991310,2cyclictest311ktimers/100:37:471
1682518991310,2cyclictest2032588-21ssh23:35:411
1682518991310,2cyclictest1750054-21fschecks_count20:15:151
1682518991310,2cyclictest1750054-21fschecks_count20:15:151
1682515991312,1cyclictest60650irq/128-enp0s29f1:rx-020:15:290
1682515991312,1cyclictest60650irq/128-enp0s29f1:rx-020:15:290
168251899129,1cyclictest0-21swapper/100:00:111
168251899128,3cyclictest1873352-21unixbench-2d21:50:291
168251899128,3cyclictest1780149-21ntp_kernel_pll_20:40:261
168251899127,1cyclictest0-21swapper/121:40:001
1682518991210,1cyclictest0-21swapper/121:05:001
168251599125,5cyclictest1702182-21ntpq19:25:240
1682515991210,1cyclictest60650irq/128-enp0s29f1:rx-020:55:140
168251899117,2cyclictest1770247-21turbostat.cron20:35:001
168251899116,4cyclictest508-21in:imuxsock19:30:001
16825189995,2cyclictest1692751-21cron19:20:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional