OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8
Special All - All RT - Optimization - Ethernet - Thumbnails - Next

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rackcslot4s.osadl.org (updated Sat Jul 27, 2024 00:44:21)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
28279	99	34	11,7	cyclictest	15324	-21	fschecks_count	21:45:13	3
28279	99	31	0,3	cyclictest	21360	-21	turbostat.cron	22:05:00	3
28275	99	28	16,8	cyclictest	24423	-21	apt-get	20:35:10	2
28266	99	24	3,14	cyclictest	24746	-21	apt-get	20:35:15	0
28266	99	24	10,4	cyclictest	9	-21	ksoftirqd/0	19:35:14	0
28279	99	23	1,3	cyclictest	29828	-21	munin-node	19:15:01	3
28279	99	22	16,4	cyclictest	5599	-21	http	22:55:00	3
28279	99	22	16,3	cyclictest	22178	-21	python3	23:45:01	3
28279	99	22	15,4	cyclictest	32315	-21	munin-run	21:00:01	3
28275	99	22	18,2	cyclictest	7059	-21	ls	19:40:21	2
28275	99	22	13,7	cyclictest	4410	-21	tune2fs	22:50:12	2
28270	99	22	14,6	cyclictest	26161	-21	df	23:55:14	1
28266	99	22	13,6	cyclictest	5771	-21	date	22:55:01	0
28279	99	21	2,13	cyclictest	22419	-21	apt-get	20:30:13	3
28279	99	21	16,3	cyclictest	25116	-21	ntp_offset	23:50:18	3
28279	99	21	13,6	cyclictest	9879	-21	awk	21:25:33	3
28279	99	21	13,6	cyclictest	30138	-21	awk	20:50:20	3
28279	99	21	13,6	cyclictest	16896	-21	snmp_o5powerctl	23:25:23	3
28279	99	21	0,5	cyclictest	10811	-21	turbostat	23:10:00	3
28279	99	21	0,19	cyclictest	24160	-21	apt-config	20:35:02	3
28275	99	21	2,13	cyclictest	32464	-21	apt-get	00:15:11	2
28275	99	21	13,6	cyclictest	7176	-21	sed	00:35:17	2
28275	99	21	13,6	cyclictest	32714	-21	ls	21:00:13	2
28275	99	21	12,6	cyclictest	2949	-21	grep	00:20:27	2
28275	99	21	12,6	cyclictest	21996	-21	grep	23:40:24	2
28275	99	21	1,14	cyclictest	20805	-21	apt-get	20:25:12	2
28270	99	21	2,14	cyclictest	14329	-21	apt-get	20:05:10	1
28270	99	21	15,4	cyclictest	19717	-21	cron	22:00:00	1
28270	99	21	14,5	cyclictest	424	-21	fw_packets	21:00:17	1
28270	99	21	13,6	cyclictest	5747	-21	sed	22:55:01	1
28270	99	21	13,6	cyclictest	23357	-21	memory	23:45:15	1
28270	99	21	12,7	cyclictest	1294	-21	ntpq	22:40:20	1
28266	99	21	16,3	cyclictest	21722	-21	sendmail_mailqu	20:25:21	0
28266	99	21	15,3	cyclictest	1547	-21	latency_hist	21:05:00	0
28266	99	21	14,5	cyclictest	27888	-21	forks	20:45:14	0
28266	99	21	14,5	cyclictest	13930	-21	awk	23:15:26	0
28266	99	21	1,14	cyclictest	11036	-21	apt-get	23:10:11	0
28266	99	21	0,19	cyclictest	10916	-21	sed	21:30:27	0
28279	99	20	1,3	cyclictest	3804	-21	apt-config	22:50:01	3
28279	99	20	13,5	cyclictest	4315	-21	wc	21:10:15	3
28279	99	20	13,5	cyclictest	32695	-21	tune2fs	00:15:13	3
28279	99	20	13,5	cyclictest	31510	-21	snmp_o5powerctl	19:15:24	3
28279	99	20	13,5	cyclictest	30330	-21	sed	00:05:24	3
28279	99	20	12,6	cyclictest	7006	-21	basename	00:35:16	3
28279	99	20	12,5	cyclictest	21526	-21	nfs4_client	20:25:19	3
28279	99	20	0,7	cyclictest	17344	-21	lsb_release	20:15:00	3
28279	99	20	0,3	cyclictest	6815	-21	lsb_release	21:20:00	3
28279	99	20	0,16	cyclictest	1542	-21	apt-get	21:05:05	3
28279	99	20	0,16	cyclictest	1542	-21	apt-get	21:05:05	3
28279	99	20	0,14	cyclictest	4543	-21	cat	19:35:00	3
28275	99	20	2,14	cyclictest	6175	-21	apt-get	22:55:12	2
28275	99	20	13,5	cyclictest	6654	-21	awk	21:15:28	2
28275	99	20	13,5	cyclictest	27767	-21	expr	00:00:14	2
28275	99	20	13,5	cyclictest	20154	-21	which	20:20:22	2
28275	99	20	12,6	cyclictest	31290	-21	awk	19:15:20	2
28275	99	20	12,6	cyclictest	23368	-21	ls	23:45:15	2
28275	99	20	12,6	cyclictest	15689	-21	cat	21:45:17	2
28275	99	20	1,14	cyclictest	17514	-21	apt-get	23:30:13	2
28275	99	20	1,14	cyclictest	17514	-21	apt-get	23:30:13	2
28275	99	20	10,7	cyclictest	14111	-21	basename	20:05:01	2
28275	99	20	0,14	cyclictest	7028	-21	apt-get	21:20:09	2
28270	99	20	3,9	cyclictest	23805	-21	apt-get	20:35:10	1
28270	99	20	15,3	cyclictest	7567	-21	sendmail	23:00:00	1
28270	99	20	13,5	cyclictest	31017	-21	vmstat	22:30:28	1
28270	99	20	13,5	cyclictest	21845	-21	tune2fs	22:05:17	1
28270	99	20	13,5	cyclictest	18398	-21	proc_pri	20:15:22	1
28270	99	20	12,6	cyclictest	9641	-21	snmp_o5powerctl	21:25:31	1
28270	99	20	12,6	cyclictest	703	-21	sensors_temp	19:20:23	1
28270	99	20	12,6	cyclictest	3285	-21	sed	22:45:27	1
28270	99	20	12,6	cyclictest	20575	-21	memory	22:00:18	1
28270	99	20	11,6	cyclictest	13060	-21	hostname	23:15:15	1
28270	99	20	1,13	cyclictest	9470	-21	apt-get	19:50:10	1
28266	99	20	2,12	cyclictest	16726	-21	apt-get	21:50:12	0
28266	99	20	13,5	cyclictest	9	-21	ksoftirqd/0	00:10:12	0
28266	99	20	13,5	cyclictest	5223	-21	cut	00:30:12	0
28266	99	20	13,5	cyclictest	22898	-21	expr	23:45:11	0
28266	99	20	13,5	cyclictest	22699	-21	grep	22:05:29	0
28266	99	20	13,5	cyclictest	20655	-21	grep	22:00:19	0
28266	99	20	13,4	cyclictest	22796	-21	grep	20:30:15	0
28266	99	20	12,5	cyclictest	32377	-21	threads	22:35:30	0
28266	99	20	12,5	cyclictest	29972	-21	ntp_kernel_pll_	00:05:18	0
28279	99	19	2,16	cyclictest	31076	-21	latency_hist	22:35:01	3
28279	99	19	2,15	cyclictest	6329	-21	latency_hist	00:35:01	3
28279	99	19	15,2	cyclictest	6844	-21	missed_timers	19:40:19	3
28279	99	19	14,3	cyclictest	26199	-21	sh	22:20:00	3
28279	99	19	14,3	cyclictest	26199	-21	sh	22:20:00	3
28279	99	19	13,4	cyclictest	32551	-21	dpkg	21:00:12	3
28279	99	19	1,2	cyclictest	29084	-21	munin-run	20:49:59	3
28279	99	19	12,5	cyclictest	7331	-21	cat	22:55:28	3
28279	99	19	11,6	cyclictest	22030	-21	cat	22:05:18	3
28279	99	19	11,6	cyclictest	19174	-21	awk	21:55:21	3
28279	99	19	11,6	cyclictest	11649	-21	wc	19:55:19	3
28279	99	19	11,6	cyclictest	11649	-21	wc	19:55:18	3
28279	99	19	1,13	cyclictest	19124	-21	apt-get	23:35:10	3
28279	99	19	10,7	cyclictest	3095	-21	cron	00:24:59	3
28279	99	19	0,17	cyclictest	32439	-21	cat	00:14:59	3
28275	99	19	15,2	cyclictest	8560	-21	ntp_offset	23:00:21	2
28275	99	19	13,3	cyclictest	0	-21	swapper/2	19:35:10	2
28275	99	19	12,5	cyclictest	3649	-21	tune2fs	00:25:15	2
28275	99	19	12,5	cyclictest	18484	-21	expr	21:55:12	2
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17