You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-22 - 09:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot6.osadl.org (updated Fri Nov 22, 2024 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
578519993625,9cyclictest0-21swapper/522:10:005
578523993225,5cyclictest0-21swapper/721:52:407
578517993224,5cyclictest0-21swapper/321:12:353
578510993115,2cyclictest1162-21containerd22:56:390
57851099311,15cyclictest670762-21kworker/u16:0+rpciod00:15:140
57851999303,23cyclictest655718-21sed23:00:015
578519993026,2cyclictest1164-21containerd22:16:415
578523992924,2cyclictest672689-21latency_hist23:55:027
578519992924,3cyclictest630788-21aten_rapower_en21:45:115
578519992923,4cyclictest650074-21wc22:40:255
57851999292,24cyclictest606020-21grep20:30:155
57851999291,1cyclictest0-21swapper/500:20:295
57851999282,19cyclictest605221-21kworker/u16:7+xprtiod20:37:325
578519992813,11cyclictest645749-21apt-get22:30:145
578518992824,2cyclictest1167-21dockerd19:54:294
57851799281,24cyclictest0-21swapper/323:30:183
57851399280,1cyclictest0-21swapper/121:34:401
57851399280,1cyclictest0-21swapper/121:34:401
578522992723,2cyclictest630790-21aten_rapower_en21:45:116
578519992723,2cyclictest0-21swapper/500:15:185
578519992721,3cyclictest1151-21containerd00:27:495
578519992719,5cyclictest2771081-21snmpd20:16:135
578519992718,6cyclictest1434492-21containerd23:32:515
578510992715,3cyclictest666165-21grep23:30:200
57852299266,3cyclictest636925-21sed22:00:266
57852299266,16cyclictest0-21swapper/620:00:266
578522992624,1cyclictest641299-21ld22:15:206
578522992624,1cyclictest0-21swapper/619:55:016
578522992623,1cyclictest0-21swapper/622:56:186
578522992622,2cyclictest3680002-21containerd20:54:346
578522992620,4cyclictest610665-21apt-get20:45:116
578519992620,4cyclictest658138-21collect223:05:235
578519992619,5cyclictest0-21swapper/522:50:335
578519992618,5cyclictest395736-21munin-node21:30:155
578519992618,5cyclictest395736-21munin-node21:30:145
578518992623,2cyclictest651988-21latency_hist22:50:004
57851799269,14cyclictest644248-21/usr/sbin/munin22:25:133
578517992616,4cyclictest1166-21dockerd23:04:453
578517992616,4cyclictest1166-21dockerd23:04:453
578514992623,2cyclictest674609-21/usr/sbin/munin00:00:132
57852399252,16cyclictest684088-21kworker/u16:2-events_freezable_power_00:30:157
57852299256,3cyclictest64-21ksoftirqd/622:39:026
57852299256,3cyclictest64-21ksoftirqd/622:39:016
578522992521,2cyclictest667049-21cron23:35:016
578522992514,7cyclictest1145-21dockerd23:41:046
57851999257,15cyclictest585526-21/usr/sbin/munin19:30:145
57851999253,6cyclictest0-21swapper/519:37:295
578519992521,2cyclictest618431-21grep21:05:255
578519992519,4cyclictest0-21swapper/500:00:205
578519992519,3cyclictest1202620-21dockerd21:59:425
578519992518,5cyclictest1152-21containerd23:41:045
578519992513,7cyclictest589512-21ntpq19:40:235
578518992521,2cyclictest671294-21aten_rapower_po23:50:114
578518992521,2cyclictest628974-21latency_hist21:40:004
57851799251,19cyclictest0-21swapper/322:30:223
578513992523,1cyclictest0-21swapper/123:56:251
578513992521,2cyclictest611892-21kworker/u16:13-rpciod20:52:371
57851099250,16cyclictest574049-21kworker/u16:14+rpciod20:06:200
578523992420,2cyclictest609376-21gltestperf20:40:157
578523992420,2cyclictest609376-21gltestperf20:40:157
578523992420,2cyclictest0-21swapper/719:20:237
578523992419,3cyclictest659340-21df_abs23:10:147
578523992417,3cyclictest0-21swapper/719:12:267
57852399241,18cyclictest591804-21kworker/u16:4+rpciod19:56:327
57852299248,1cyclictest0-21swapper/621:35:196
578522992422,1cyclictest0-21swapper/600:10:336
578522992422,1cyclictest0-21swapper/600:10:326
578522992421,1cyclictest0-21swapper/622:35:016
578522992421,1cyclictest0-21swapper/621:00:256
578522992420,2cyclictest64-21ksoftirqd/622:40:186
578522992420,2cyclictest582810-21/usr/sbin/munin19:20:196
578522992419,3cyclictest0-21swapper/600:26:526
578522992418,4cyclictest664681-21ntpq23:25:186
578522992414,8cyclictest0-21swapper/622:10:016
57851999249,1cyclictest0-21swapper/523:22:045
57851999246,13cyclictest595129-21kworker/u16:9+xprtiod19:56:305
57851999243,17cyclictest677808-21/usr/sbin/munin00:10:235
57851999243,17cyclictest677808-21/usr/sbin/munin00:10:235
578519992420,2cyclictest27950irq/115-46000000.ethernet-tx000:31:405
578519992419,3cyclictest2771081-21snmpd22:24:135
578519992419,3cyclictest1166-21dockerd21:42:395
578519992416,5cyclictest627693-21uniq21:35:135
578519992416,4cyclictest0-21swapper/522:26:435
57851899241,19cyclictest600149-21apt-get20:15:044
57851499245,15cyclictest596317-21find20:00:252
578514992421,1cyclictest622201-21latency_hist21:20:012
578514992419,3cyclictest592368-21aten_rapower_vo19:50:142
57851499241,15cyclictest680744-21kworker/u16:9-rpciod00:20:172
57851499240,20cyclictest2771081-21snmpd20:15:032
578513992418,3cyclictest590307-21latency_hist19:45:011
578513992416,3cyclictest651969-21kworker/u16:12+rpciod22:47:481
57851099244,17cyclictest131ktimers/020:40:140
57851099244,17cyclictest131ktimers/020:40:140
578510992420,2cyclictest0-21swapper/021:40:000
578510992419,2cyclictest639988-21timerandwakeup22:10:270
57852399234,16cyclictest686910-21/usr/sbin/munin00:35:257
57852399234,16cyclictest686910-21/usr/sbin/munin00:35:247
57852399233,17cyclictest595784-21df20:00:177
57852399231,18cyclictest0-21swapper/723:45:307
57852299237,1cyclictest0-21swapper/619:40:226
57852299234,4cyclictest584254-21/usr/sbin/munin19:25:156
578522992320,1cyclictest0-21swapper/623:50:506
578522992320,1cyclictest0-21swapper/623:06:326
578522992320,1cyclictest0-21swapper/621:10:236
578522992320,1cyclictest0-21swapper/619:15:156
578522992320,1cyclictest0-21swapper/619:15:156
578522992319,2cyclictest631ktimers/622:26:296
578522992318,3cyclictest0-21swapper/620:10:176
578522992318,3cyclictest0-21swapper/620:10:176
578522992318,2cyclictest0-21swapper/619:35:126
57852299231,18cyclictest0-21swapper/620:40:156
57852299231,18cyclictest0-21swapper/620:40:156
57852299231,18cyclictest0-21swapper/620:15:046
57852299231,17cyclictest0-21swapper/621:30:036
57852299231,17cyclictest0-21swapper/621:30:036
57852299230,17cyclictest618555-21kworker/u16:10+rpciod21:10:006
57851999235,3cyclictest581808-21sshd19:17:395
57851999235,3cyclictest581808-21sshd19:17:395
57851999235,15cyclictest686132-21/usr/sbin/munin00:35:115
57851999235,15cyclictest686132-21/usr/sbin/munin00:35:115
57851999232,17cyclictest639325-21fschecks_count22:10:185
57851999232,17cyclictest0-21swapper/523:53:575
578519992320,1cyclictest583367-21/usr/sbin/munin19:20:265
578519992318,3cyclictest584310-21grep19:25:155
578519992318,3cyclictest0-21swapper/500:05:295
578519992317,4cyclictest551ktimers/520:20:135
578519992317,4cyclictest551ktimers/520:20:135
578519992317,4cyclictest0-21swapper/521:00:515
578519992314,5cyclictest660809-21apt-get23:15:155
57851999231,18cyclictest612540-21/usr/sbin/munin20:50:155
578518992319,2cyclictest676605-21gltestperf00:05:194
578518992319,2cyclictest634785-21/usr/sbin/munin21:55:234
578518992318,3cyclictest656377-21/usr/sbin/munin23:00:184
578518992318,3cyclictest656377-21/usr/sbin/munin23:00:174
578518992318,3cyclictest654003-21latency_hist22:55:044
578518992318,2cyclictest649148-21/usr/sbin/munin22:40:134
578518992316,2cyclictest611890-21kworker/u16:9+xprtiod20:47:344
57851899231,3cyclictest0-21swapper/400:23:524
57851799235,3cyclictest28050irq/117-46000000.ethernet20:25:013
57851799235,3cyclictest28050irq/117-46000000.ethernet20:25:003
578517992319,2cyclictest683713-21switchtime00:25:273
578517992319,2cyclictest64539-21containerd23:08:463
578517992319,2cyclictest0-21swapper/323:42:483
578517992318,3cyclictest606929-21latency_hist20:35:013
578517992318,3cyclictest587358-21grep19:35:123
578517992318,3cyclictest28050irq/117-46000000.ethernet21:26:343
578514992318,3cyclictest0-21swapper/223:06:462
57851499231,2cyclictest613493-21head20:50:262
57851499231,18cyclictest586210-21ntpq19:30:202
57851499231,18cyclictest30-21rcuc/219:40:292
57851499231,17cyclictest0-21swapper/223:47:032
57851499231,17cyclictest0-21swapper/219:12:022
57851499230,20cyclictest0-21swapper/221:40:182
578513992319,1cyclictest598467-21kworker/u16:10+rpciod20:16:211
578513992316,5cyclictest0-21swapper/122:38:141
578513992316,5cyclictest0-21swapper/122:38:131
57851399231,16cyclictest2159658-21kworker/1:2+events_long00:02:441
57851099236,2cyclictest585106-21sed19:25:260
578510992318,3cyclictest0-21swapper/019:15:180
578510992318,3cyclictest0-21swapper/019:15:180
578523992218,2cyclictest599288-21gltestperf20:10:187
578523992218,2cyclictest599288-21gltestperf20:10:177
578523992218,1cyclictest0-21swapper/721:10:037
578523992217,2cyclictest1-21systemd20:38:067
57852399221,17cyclictest0-21swapper/720:25:107
57852399221,17cyclictest0-21swapper/700:05:127
57852299227,1cyclictest0-21swapper/600:00:216
57852299225,15cyclictest0-21swapper/622:46:236
57852299223,15cyclictest0-21swapper/621:26:306
578522992220,1cyclictest0-21swapper/621:56:206
578522992219,1cyclictest0-21swapper/600:15:186
578522992218,2cyclictest0-21swapper/623:16:246
578522992218,1cyclictest0-21swapper/619:56:266
578522992214,4cyclictest0-21swapper/623:21:116
578522992211,5cyclictest1434492-21containerd21:40:116
57852299220,19cyclictest631ktimers/619:35:006
57851999224,16cyclictest0-21swapper/519:51:225
578519992218,2cyclictest636871-21sed22:00:265
578519992218,2cyclictest633174-21mailstats21:50:275
578519992218,2cyclictest0-21swapper/520:45:265
578519992217,2cyclictest0-21swapper/523:29:585
578519992216,2cyclictest0-21swapper/520:10:125
578519992216,2cyclictest0-21swapper/520:10:125
578519992214,5cyclictest0-21swapper/521:20:245
57851999221,18cyclictest596036-21/usr/sbin/munin20:00:185
57851999220,19cyclictest0-21swapper/520:05:115
578518992219,2cyclictest639247-21df_abs22:10:174
578518992218,2cyclictest590152-21dropbear19:42:554
578518992216,3cyclictest633679-21latency_hist21:55:004
578518992215,3cyclictest1166-21dockerd19:26:044
57851899221,18cyclictest0-21swapper/423:30:174
57851899220,20cyclictest0-21swapper/422:55:194
57851899220,15cyclictest0-21swapper/421:34:274
57851899220,15cyclictest0-21swapper/421:34:264
57851799229,10cyclictest649114-21aten_rapower_cu22:40:133
57851799226,2cyclictest647707-21tune2fs22:35:153
57851799226,2cyclictest647707-21tune2fs22:35:153
57851799224,2cyclictest600917-21fschecks_count20:15:193
57851799222,4cyclictest1152-21containerd22:58:453
578517992220,1cyclictest0-21swapper/321:30:033
578517992220,1cyclictest0-21swapper/321:30:023
578517992216,3cyclictest28050irq/117-46000000.ethernet20:06:183
578517992215,4cyclictest1151-21containerd23:52:213
578517992215,3cyclictest0-21swapper/321:00:133
57851799221,16cyclictest1202620-21dockerd22:00:403
57851799220,16cyclictest615211-21kworker/u16:8+rpciod21:17:323
578514992219,1cyclictest655772-21/usr/sbin/munin23:00:172
578514992219,1cyclictest655772-21/usr/sbin/munin23:00:172
578514992219,1cyclictest584649-21/usr/sbin/munin19:25:242
578514992218,2cyclictest623107-21sendmail_mailtr21:20:262
578514992217,3cyclictest617786-21/usr/sbin/munin21:05:182
57851499221,18cyclictest311ktimers/221:50:232
57851499221,17cyclictest0-21swapper/223:55:122
57851499221,17cyclictest0-21swapper/222:55:202
57851499220,16cyclictest611881-21kworker/u16:1+rpciod20:47:342
57851399223,16cyclictest622859-21/usr/sbin/munin21:20:181
57851399222,17cyclictest231ktimers/100:06:521
578513992218,2cyclictest580313-21latency_hist19:15:011
578513992218,2cyclictest0-21swapper/120:01:351
578513992217,3cyclictest685818-21latency_hist00:35:011
578513992217,3cyclictest629490-21grep21:40:151
578513992217,2cyclictest588372-21grep19:35:241
578513992216,3cyclictest624018-21aten_rapower_cu21:25:151
578513992216,2cyclictest0-21swapper/119:15:131
578513992216,2cyclictest0-21swapper/119:15:131
57851399221,16cyclictest0-21swapper/123:40:531
57851399220,19cyclictest231ktimers/119:23:561
57851099222,17cyclictest650923-21df_abs22:45:190
578510992219,1cyclictest664649-21netstat23:25:170
578510992217,3cyclictest3680002-21containerd00:05:180
57851099221,17cyclictest0-21swapper/021:30:030
57851099221,17cyclictest0-21swapper/021:30:030
57852399212,16cyclictest0-21swapper/719:25:247
578523992118,1cyclictest602283-21/usr/sbin/munin20:20:267
578523992118,1cyclictest602283-21/usr/sbin/munin20:20:257
578523992115,3cyclictest639501-21cut22:10:187
578523992115,3cyclictest621915-21sendmail21:20:007
578523992115,3cyclictest614362-21grep20:55:147
578523992115,3cyclictest614362-21grep20:55:137
578523992115,2cyclictest0-21swapper/722:00:117
57852399211,2cyclictest0-21swapper/721:09:217
57852399211,2cyclictest0-21swapper/719:47:117
57852399211,17cyclictest0-21swapper/722:40:117
57852399211,17cyclictest0-21swapper/719:30:137
57852399211,16cyclictest656405-21ntpq23:00:197
57852399211,16cyclictest656405-21ntpq23:00:187
57852399211,16cyclictest1163-21containerd21:48:087
57852399211,16cyclictest0-21swapper/723:05:247
57852399211,16cyclictest0-21swapper/722:35:207
57852399211,16cyclictest0-21swapper/722:35:197
578522992119,1cyclictest0-21swapper/619:10:216
578522992118,1cyclictest0-21swapper/600:20:276
578522992116,3cyclictest621889-21kworker/u16:14+rpciod21:17:326
578522992115,3cyclictest674122-21idleruntime-cro00:00:016
578522992115,3cyclictest643007-21ntpq22:20:226
578522992115,3cyclictest1917278-21containerd23:46:496
578522992114,5cyclictest607968-21ld20:35:226
57852299211,17cyclictest603468-21tail20:20:266
57852299211,17cyclictest603468-21tail20:20:266
57852299211,17cyclictest0-21swapper/600:35:506
57852299211,17cyclictest0-21swapper/600:35:496
57852299211,16cyclictest656760-21/usr/sbin/munin23:00:246
57852299211,16cyclictest656760-21/usr/sbin/munin23:00:246
57852299210,18cyclictest0-21swapper/619:46:236
57851999219,9cyclictest673111-21fschecks_count23:55:155
57851999213,16cyclictest0-21swapper/523:35:025
57851999213,16cyclictest0-21swapper/523:35:015
578519992117,2cyclictest0-21swapper/522:48:215
578519992117,2cyclictest0-21swapper/521:14:405
578519992116,2cyclictest0-21swapper/519:10:155
578519992115,4cyclictest0-21swapper/521:30:015
578519992115,3cyclictest0-21swapper/523:10:155
578519992115,2cyclictest604001-21apt-get20:25:115
578519992114,6cyclictest0-21swapper/519:47:345
578519992114,4cyclictest2771081-21snmpd20:55:105
578519992114,4cyclictest2771081-21snmpd20:55:105
57851999211,17cyclictest610477-21wc20:45:025
57851999211,17cyclictest610477-21wc20:45:015
57851899213,2cyclictest613808-21latency_hist20:55:024
578518992119,1cyclictest680911-21latency_hist00:20:004
578518992117,2cyclictest662434-21/usr/sbin/munin23:20:244
578518992116,3cyclictest642797-21grep22:20:174
578518992116,3cyclictest0-21swapper/422:07:024
57851899211,17cyclictest1163-21containerd23:10:464
57851899211,17cyclictest0-21swapper/420:26:304
57851899211,16cyclictest64539-21containerd00:00:214
57851899211,16cyclictest591229-21ldconfig19:45:244
57851899211,16cyclictest0-21swapper/423:25:514
57851899210,18cyclictest0-21swapper/420:35:114
57851899210,17cyclictest0-21swapper/421:25:214
578517992117,2cyclictest667436-21find23:35:113
578517992117,2cyclictest667436-21find23:35:113
578517992117,2cyclictest660485-21cat23:15:003
578517992117,2cyclictest632807-21gltestperf21:50:193
578517992117,2cyclictest631082-21fschecks_time21:45:153
578517992117,2cyclictest630212-21grep21:40:243
57851799211,1cyclictest0-21swapper/322:45:123
57851799211,17cyclictest653004-21ntp_offset22:50:283
57851799211,16cyclictest0-21swapper/319:25:243
57851799210,17cyclictest0-21swapper/320:10:123
57851799210,17cyclictest0-21swapper/320:10:113
578514992119,1cyclictest686196-21aten_rapower_cu00:35:132
578514992119,1cyclictest686196-21aten_rapower_cu00:35:122
578514992117,2cyclictest678296-21/usr/sbin/munin00:10:232
578514992117,2cyclictest678296-21/usr/sbin/munin00:10:232
578514992117,2cyclictest624495-21load21:25:192
578514992115,3cyclictest679753-21fschecks_count00:15:162
578514992113,4cyclictest772922-21sendmail-mta21:30:182
578514992113,4cyclictest772922-21sendmail-mta21:30:172
57851499211,18cyclictest628176-21awk21:35:222
57851499211,18cyclictest0-21swapper/221:46:292
57851499210,18cyclictest683517-21ntpq00:25:242
57851499210,17cyclictest659152-21apt-get23:10:112
57851499210,17cyclictest0-21swapper/222:46:192
57851499210,17cyclictest0-21swapper/219:20:202
578513992118,1cyclictest593737-21latency_hist19:55:011
578513992117,2cyclictest664521-21grep23:25:161
578513992117,2cyclictest625232-21kworker/u16:13+xprtiod21:50:001
578513992116,2cyclictest608205-21awk20:35:241
57851399211,3cyclictest0-21swapper/123:10:121
57851399211,17cyclictest636010-21sort22:00:141
57851399211,17cyclictest0-21swapper/123:35:341
57851399211,17cyclictest0-21swapper/123:35:331
57851399211,17cyclictest0-21swapper/119:46:491
57851399211,16cyclictest0-21swapper/120:05:111
57851399210,18cyclictest0-21swapper/121:50:141
57851399210,17cyclictest0-21swapper/122:55:241
57851099213,2cyclictest131ktimers/023:11:270
578510992117,2cyclictest674859-21fschecks_time00:00:160
578510992117,2cyclictest596391-21switchtime20:00:260
578510992116,3cyclictest1154-21containerd00:26:290
578510992116,3cyclictest0-21swapper/000:20:250
57851099211,17cyclictest1349086-21dockerd22:44:430
57851099211,17cyclictest0-21swapper/022:05:210
57851099211,16cyclictest0-21swapper/022:30:150
57851099211,15cyclictest0-21swapper/020:30:120
578523992017,2cyclictest623827-21latency_hist21:25:027
578523992016,2cyclictest644138-21wget22:25:127
578523992015,2cyclictest674208-21latency_hist00:00:007
578523992015,2cyclictest663558-21cat23:20:287
578523992015,2cyclictest654772-21ntp_states22:55:207
578523992015,2cyclictest0-21swapper/721:55:207
578523992014,3cyclictest683494-21ntpq00:25:237
57852399201,2cyclictest0-21swapper/720:30:127
57852399201,15cyclictest0-21swapper/700:01:367
57852399200,17cyclictest611112-21/usr/sbin/munin20:45:177
57852399200,16cyclictest0-21swapper/721:30:317
57852399200,16cyclictest0-21swapper/721:30:317
578522992017,2cyclictest685187-21/usr/sbin/munin00:30:296
578522992016,2cyclictest64-21ksoftirqd/621:20:206
578522992015,3cyclictest0-21swapper/623:35:116
578522992015,3cyclictest0-21swapper/623:35:116
578522992014,3cyclictest632355-21/usr/sbin/munin21:50:216
57852299201,2cyclictest652820-21cut22:50:236
57852299200,17cyclictest0-21swapper/623:10:106
57852299200,16cyclictest606235-21ld20:30:206
578519992017,2cyclictest0-21swapper/523:02:585
578519992017,2cyclictest0-21swapper/523:02:575
578519992017,2cyclictest0-21swapper/521:15:135
57851999201,16cyclictest0-21swapper/522:35:155
57851999201,16cyclictest0-21swapper/522:35:145
578518992017,2cyclictest644182-21aten_rapower_vo22:25:144
578518992017,1cyclictest658091-21ld23:05:224
578518992017,1cyclictest0-21swapper/419:55:124
578518992016,2cyclictest597919-21proc_pri20:05:234
578518992015,3cyclictest64539-21containerd22:30:154
578518992015,2cyclictest0-21swapper/420:21:094
578518992015,2cyclictest0-21swapper/420:21:094
578518992014,3cyclictest615173-21grep20:55:234
578518992014,3cyclictest615173-21grep20:55:234
57851899201,2cyclictest0-21swapper/421:15:124
57851899201,15cyclictest0-21swapper/400:10:194
57851899201,15cyclictest0-21swapper/400:10:184
57851899200,18cyclictest0-21swapper/423:55:124
57851899200,18cyclictest0-21swapper/422:35:084
57851899200,18cyclictest0-21swapper/422:35:074
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional