You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 14:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Fri Nov 22, 2024 12:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1283191993635,0cyclictest0-21swapper/710:57:387
1283191992220,1cyclictest0-21swapper/708:04:337
1283191992113,7cyclictest0-21swapper/710:34:257
128318399210,20cyclictest0-21swapper/410:58:014
128319199200,6cyclictest0-21swapper/708:40:157
1283185992018,2cyclictest1767126-21kworker/u16:6+flush-8:012:33:035
1283188991916,2cyclictest1450594-21kworker/u16:4+flush-8:008:28:156
1283188991914,4cyclictest1737186-21latency_hist10:52:386
1283185991917,2cyclictest1706074-21kworker/u16:1+events_unbound11:28:565
1283185991917,2cyclictest1706074-21kworker/u16:1+events_unbound11:28:565
1283185991917,2cyclictest1605590-21kworker/u16:3+events_unbound10:58:085
1283185991816,2cyclictest1758148-21kworker/u16:0+flush-8:011:03:065
1283185991815,2cyclictest1501677-21kworker/u16:2+flush-8:008:57:385
1283185991815,2cyclictest1328126-21kworker/u16:0+events_unbound08:03:075
128319199174,5cyclictest0-21swapper/710:20:157
128318899170,6cyclictest0-21swapper/609:07:526
128318899170,13cyclictest0-21swapper/608:43:026
1283185991715,2cyclictest1501677-21kworker/u16:2+events_unbound10:32:595
1283185991715,2cyclictest1501677-21kworker/u16:2+events_unbound10:12:535
128319199168,7cyclictest0-21swapper/707:30:347
1283185991614,2cyclictest1605590-21kworker/u16:3+events_unbound10:52:105
1283185991614,2cyclictest1328126-21kworker/u16:0+flush-8:008:27:535
1283185991614,2cyclictest1281383-21kworker/u16:3+flush-8:007:28:115
1283185991513,2cyclictest1378056-21kworker/u16:2+events_unbound08:19:385
128318399150,15cyclictest0-21swapper/408:38:004
128319199140,13cyclictest0-21swapper/710:50:017
128319199140,13cyclictest0-21swapper/708:53:027
128318899140,1cyclictest0-21swapper/608:37:526
128318399146,8cyclictest0-21swapper/410:25:094
128318399140,14cyclictest0-21swapper/409:08:254
128318399140,14cyclictest0-21swapper/408:44:564
128318399140,13cyclictest0-21swapper/411:03:064
128318399140,13cyclictest0-21swapper/410:49:484
1283191991312,0cyclictest0-21swapper/708:18:317
1283185991311,2cyclictest1605590-21kworker/u16:3+events_unbound10:19:145
1283185991311,2cyclictest1501677-21kworker/u16:2+events_unbound10:42:355
1283185991311,2cyclictest1328126-21kworker/u16:0+flush-8:008:45:375
1283185991311,2cyclictest1328126-21kworker/u16:0+events_unbound08:13:545
128318399130,13cyclictest0-21swapper/410:18:264
128318399130,12cyclictest1439663-21chrt08:22:354
128318399130,11cyclictest1779347-21cstates11:12:494
1283191991210,1cyclictest0-21swapper/710:26:387
128319199120,1cyclictest0-21swapper/710:13:007
128319199120,10cyclictest0-21swapper/711:17:327
1283188991211,1cyclictest0-21swapper/611:02:366
128318899120,1cyclictest0-21swapper/608:53:026
1283185991210,2cyclictest1605590-21kworker/u16:3+events_unbound10:25:185
128318599120,12cyclictest0-21swapper/508:38:475
128318399127,4cyclictest1293173-21latency_hist07:12:394
128318399124,8cyclictest0-21swapper/408:57:314
128318399123,8cyclictest1698340-21tr10:32:524
128318399122,9cyclictest1457253-21cut08:27:584
1283183991211,1cyclictest0-21swapper/410:15:264
128318399120,12cyclictest0-21swapper/410:40:214
128318399120,12cyclictest0-21swapper/408:13:524
128319199119,1cyclictest0-21swapper/712:34:427
128319199119,1cyclictest0-21swapper/708:47:127
128319199119,1cyclictest0-21swapper/708:14:377
1283191991110,0cyclictest0-21swapper/711:02:087
128319199110,10cyclictest0-21swapper/710:38:077
128318899116,4cyclictest1666979-21cat10:17:526
1283188991110,1cyclictest0-21swapper/610:15:276
1283188991110,1cyclictest0-21swapper/608:20:316
128318599119,2cyclictest1501677-21kworker/u16:2+flush-8:010:07:395
128318599119,2cyclictest1328126-21kworker/u16:0+events_unbound09:11:005
128318599118,2cyclictest1328126-21kworker/u16:0+flush-8:009:27:385
128318399113,6cyclictest1840231-21cat11:42:384
128318399112,8cyclictest1403586-21cut08:02:574
1283183991110,1cyclictest0-21swapper/407:31:464
128318399110,11cyclictest0-21swapper/412:33:154
128318399110,10cyclictest1818900-21chrt11:29:584
128318399110,10cyclictest1818900-21chrt11:29:584
128318899109,1cyclictest0-21swapper/611:14:446
128318899107,2cyclictest1166-21snmpd11:05:046
128318899107,2cyclictest0-21swapper/608:03:006
128318899107,2cyclictest0-21swapper/607:12:396
128318899100,9cyclictest0-21swapper/609:32:386
128318899100,4cyclictest1676445-21/usr/sbin/munin10:23:066
128318899100,2cyclictest0-21swapper/611:27:586
128318899100,2cyclictest0-21swapper/611:27:586
128318599108,2cyclictest1706074-21kworker/u16:1+events_unbound11:15:495
128318599108,1cyclictest0-21swapper/509:32:385
128318399102,7cyclictest1778884-21latency_hist11:12:384
128318399100,9cyclictest1696846-21cut10:32:384
128318399100,8cyclictest1911844-21wc12:17:384
128318399100,8cyclictest1748112-21cut10:57:384
128318399100,8cyclictest1625301-21latency_hist09:57:394
128318399100,8cyclictest1531624-21latency_hist09:07:384
128318399100,8cyclictest1399497-21cat08:02:394
12831889990,8cyclictest1696952-21/usr/sbin/munin10:32:586
12831889990,8cyclictest1328134-21/usr/sbin/munin07:28:066
12831889990,2cyclictest1942649-21/usr/sbin/munin12:32:536
12831839990,8cyclictest1932159-21latency_hist12:27:384
12831839990,8cyclictest1881093-21latency_hist12:02:384
12831839990,8cyclictest1870999-21sed11:57:384
12831839990,8cyclictest1860674-21cat11:52:384
12831839990,8cyclictest1850224-21grep11:47:384
12831839990,8cyclictest1829877-21latency_hist11:37:384
12831839990,8cyclictest1809500-21latency_hist11:27:384
12831839990,8cyclictest1809500-21latency_hist11:27:384
12831839990,8cyclictest1655719-21latency_hist10:12:384
12831839990,8cyclictest1521726-21exim409:02:384
12831839990,8cyclictest1379122-21cat07:52:394
12831839990,8cyclictest1327903-21date07:27:394
12831839990,8cyclictest1283223-21cat07:07:394
12831839990,8cyclictest1283223-21cat07:07:394
12831839990,8cyclictest0-21swapper/412:07:384
12831839990,8cyclictest0-21swapper/410:02:384
12831839990,8cyclictest0-21swapper/409:52:384
12831919980,2cyclictest17677952sleep711:06:077
12831889986,1cyclictest0-21swapper/611:52:386
12831889984,3cyclictest1891067-21latency_hist12:07:386
12831889983,4cyclictest923-21gdbus11:47:386
12831889983,3cyclictest1138-21rs:main6
12831889980,7cyclictest0-21swapper/607:47:396
12831859987,1cyclictest0-21swapper/509:47:395
12831859986,2cyclictest1706074-21kworker/u16:1+events_unbound11:37:385
12831859986,1cyclictest0-21swapper/508:52:385
12831859986,1cyclictest0-21swapper/508:37:385
12831859984,3cyclictest1603604-21date09:42:385
12831839985,2cyclictest1368624-21dump-pmu-power07:47:384
12831839983,4cyclictest1603411-21turbostat09:42:384
12831839980,7cyclictest1942267-21grep12:32:384
12831839980,7cyclictest1921969-21date12:22:384
12831839980,7cyclictest1836-21crond10:07:384
12831839980,7cyclictest1836-21crond07:22:384
12831839980,7cyclictest1799140-21latency_hist11:22:384
12831839980,7cyclictest1726938-21latency_hist10:47:384
12831839980,7cyclictest1604662-21cat09:47:384
12831839980,7cyclictest1582899-21pluginstate09:32:384
12831839980,7cyclictest1572796-21awk09:27:384
12831839980,7cyclictest1552368-21latency_hist09:17:384
12831839980,7cyclictest1552368-21latency_hist09:17:384
12831839980,7cyclictest1501441-21cut08:52:384
12831839980,7cyclictest1470670-21sendmail08:37:384
12831839980,7cyclictest1419925-21cat08:12:384
12831839980,7cyclictest1389359-21cat07:57:394
12831839980,7cyclictest1358164-21latency_hist07:42:394
12831839980,7cyclictest1348197-21crond07:37:384
12831919975,1cyclictest0-21swapper/712:27:387
12831919971,4cyclictest1911601-21turbostat12:17:387
12831919971,4cyclictest0-21swapper/711:28:567
12831919971,4cyclictest0-21swapper/711:28:567
12831919970,6cyclictest0-21swapper/708:30:337
12831889975,1cyclictest0-21swapper/610:57:386
12831889975,1cyclictest0-21swapper/607:07:396
12831889975,1cyclictest0-21swapper/607:07:396
12831889974,2cyclictest1655606-21idleruntime-cro10:12:386
12831889974,2cyclictest1614731-21munin-plugin-st09:52:386
12831889973,3cyclictest1921653-21latency_hist12:22:386
12831889973,3cyclictest1911622-21basename12:17:386
12831889973,3cyclictest1562936-21cat09:22:386
12831889973,3cyclictest1420319-21expr08:12:516
12831889973,2cyclictest639-21systemd-journal09:27:386
12831889971,5cyclictest0-21swapper/610:37:566
12831859975,1cyclictest0-21swapper/512:32:385
12831859975,1cyclictest0-21swapper/512:07:385
12831859975,1cyclictest0-21swapper/511:47:385
12831859975,1cyclictest0-21swapper/510:12:385
12831859975,1cyclictest0-21swapper/510:02:385
12831859975,1cyclictest0-21swapper/509:22:385
12831859975,1cyclictest0-21swapper/507:22:385
12831859974,2cyclictest639-21systemd-journal09:52:385
12831859973,3cyclictest1358741-21cat07:42:395
12831859970,3cyclictest1593525-21cat09:37:385
12831839970,6cyclictest1901700-21sed12:12:384
12831839970,6cyclictest1593531-21latency_hist09:37:384
12831839970,6cyclictest1450565-21latency_hist08:27:384
12831919965,0cyclictest0-21swapper/709:07:567
12831919961,3cyclictest0-21swapper/712:32:387
12831919961,3cyclictest0-21swapper/707:37:397
12831919960,5cyclictest0-21swapper/710:07:387
12831919960,4cyclictest1880954-21chrt12:02:387
12831919960,4cyclictest1655632-21sleep10:12:387
12831889965,1cyclictest0-21swapper/607:57:396
12831889964,1cyclictest0-21swapper/610:07:386
12831889963,3cyclictest1727618-21sendmail10:47:386
12831889963,2cyclictest1399234-21idleruntime-cro08:02:396
12831889963,2cyclictest1348200-21turbostat07:37:386
12831889962,3cyclictest1881200-21cat12:02:386
12831889962,3cyclictest1778735-21cat11:12:386
12831889962,3cyclictest1358721-21sed07:42:396
12831889960,4cyclictest1603679-21taskset09:42:386
12831889960,4cyclictest1532185-21cat09:07:386
12831859965,1cyclictest0-21swapper/509:17:385
12831859965,1cyclictest0-21swapper/509:17:385
12831859965,1cyclictest0-21swapper/507:12:395
12831859964,1cyclictest0-21swapper/512:27:385
12831859964,1cyclictest0-21swapper/511:22:375
12831859964,1cyclictest0-21swapper/507:07:395
12831859964,1cyclictest0-21swapper/507:07:395
12831859962,3cyclictest1922050-21sed12:22:385
12831859962,3cyclictest1778728-21cat11:12:385
12831859962,3cyclictest1532193-21latency_hist09:07:385
12831859962,3cyclictest1348332-21cat07:37:395
12831859961,1cyclictest681rcuog/409:02:385
12831859960,5cyclictest1911835-21cat12:17:385
12831859960,3cyclictest1809534-21cat11:27:385
12831859960,3cyclictest1625209-21cat09:57:385
12831859960,3cyclictest1419893-21date08:12:385
12831859960,2cyclictest681rcuog/407:47:385
12831839960,5cyclictest1562874-21sed09:22:384
12831839960,5cyclictest0-21swapper/407:17:394
12831919954,0cyclictest0-21swapper/710:32:387
12831919954,0cyclictest0-21swapper/709:27:387
12831919953,1cyclictest0-21swapper/709:52:387
12831919952,2cyclictest0-21swapper/711:22:377
12831919952,2cyclictest0-21swapper/707:22:397
12831919951,3cyclictest0-21swapper/711:37:387
12831919951,3cyclictest0-21swapper/709:47:387
12831919950,4cyclictest1850238-21sleep11:47:387
12831919950,4cyclictest1624974-21turbostat09:57:387
12831919950,4cyclictest1521741-21turbostat09:02:387
12831919950,4cyclictest1399236-21turbostat08:02:397
12831919950,4cyclictest1378813-21sleep07:52:387
12831919950,3cyclictest0-21swapper/710:02:397
12831889953,1cyclictest0-21swapper/608:52:386
12831889952,2cyclictest1282622-21cyclictest11:22:376
12831889950,4cyclictest1942249-21sendmail12:32:386
12831889950,4cyclictest1901696-21date12:12:386
12831889950,4cyclictest1870956-21sed11:57:386
12831889950,4cyclictest1840210-21latency_hist11:42:386
12831889950,4cyclictest1829703-21basename11:37:386
12831889950,4cyclictest1809374-21latency_hist11:27:386
12831889950,4cyclictest1809374-21latency_hist11:27:386
12831889950,4cyclictest1696853-21sed10:32:386
12831889950,4cyclictest1635335-21sed10:02:386
12831889950,4cyclictest1624979-21munin-plugin-st09:57:386
12831889950,4cyclictest1604789-21latency_hist09:47:386
12831889950,4cyclictest1552403-21basename09:17:386
12831889950,4cyclictest1552403-21basename09:17:386
12831889950,4cyclictest1521727-21crond09:02:386
12831889950,4cyclictest1470701-21cat08:37:386
12831889950,4cyclictest1450537-21sed08:27:386
12831889950,4cyclictest1419307-21latency_hist08:12:386
12831889950,4cyclictest1378806-21munin-plugin-st07:52:386
12831889950,4cyclictest1327842-21sed07:27:396
12831889950,4cyclictest1317586-21sh07:22:386
12831859954,1cyclictest0-21swapper/511:27:385
12831859953,1cyclictest1399235-21latency_hist08:02:395
12831859953,1cyclictest0-21swapper/512:02:385
12831859952,2cyclictest14250irq/27-ahci[0000:00:1f.2]10:32:385
12831859952,2cyclictest1-21systemd11:52:385
12831859952,2cyclictest0-21swapper/511:57:385
12831859951,3cyclictest1727617-21cut10:47:385
12831859951,3cyclictest1327879-21sed07:27:395
12831859951,1cyclictest201rcu_preempt10:57:385
12831859950,4cyclictest1840232-21date11:42:385
12831859950,4cyclictest1450546-21cat08:27:385
12831859950,4cyclictest1389218-21sendmail07:57:395
12831859950,4cyclictest1378805-21perf07:52:385
12831919941,2cyclictest0-21swapper/712:22:387
12831919941,2cyclictest0-21swapper/712:07:387
12831919941,2cyclictest0-21swapper/711:57:387
12831919941,2cyclictest0-21swapper/711:52:387
12831919941,2cyclictest0-21swapper/711:42:387
12831919941,2cyclictest0-21swapper/711:27:387
12831919941,2cyclictest0-21swapper/711:27:387
12831919941,2cyclictest0-21swapper/711:12:387
12831919941,2cyclictest0-21swapper/710:47:387
12831919941,2cyclictest0-21swapper/709:42:387
12831919941,2cyclictest0-21swapper/709:32:387
12831919941,2cyclictest0-21swapper/709:22:387
12831919941,2cyclictest0-21swapper/709:17:387
12831919941,2cyclictest0-21swapper/709:17:387
12831919941,2cyclictest0-21swapper/709:07:387
12831919941,2cyclictest0-21swapper/708:52:387
12831919941,2cyclictest0-21swapper/708:37:387
12831919941,2cyclictest0-21swapper/708:12:387
12831919941,2cyclictest0-21swapper/707:57:397
12831919941,2cyclictest0-21swapper/707:47:387
12831919941,2cyclictest0-21swapper/707:42:397
12831919941,2cyclictest0-21swapper/707:27:397
12831919941,2cyclictest0-21swapper/707:12:387
12831919941,2cyclictest0-21swapper/707:07:397
12831919941,2cyclictest0-21swapper/707:07:397
12831889941,2cyclictest1307723-21latency_hist07:17:396
12831859940,3cyclictest1901693-21cron12:12:385
12831859940,3cyclictest1307711-21latency_hist07:17:395
12831919930,2cyclictest0-21swapper/712:12:387
12831919930,2cyclictest0-21swapper/709:37:387
12831919930,2cyclictest0-21swapper/708:27:387
12831919930,2cyclictest0-21swapper/707:17:397
12831889930,2cyclictest0-21swapper/609:37:386
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional