OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #7, slot #2

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system

Characteristics of the 376 highest latencies:

System rack7slot2.osadl.org (updated Wed Apr 02, 2025 00:44:06)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
1170189	99	90	60,20	cyclictest	54	-21	kswapd0	23:00:08	0
1170189	99	84	25,45	cyclictest	256	50	irq/23-42890000.ethernet	00:00:11	0
1170190	99	80	27,16	cyclictest	54	-21	kswapd0	00:25:11	1
1170190	99	79	45,32	cyclictest	374	-21	systemd-timesyn	19:43:09	1
1170190	99	78	33,9	cyclictest	1173975	-21	modprobe	19:25:00	1
1170190	99	77	38,5	cyclictest	1239554	-21	latency_hist	22:05:01	1
1170189	99	77	57,15	cyclictest	1337816	-21	ssh	00:20:53	0
1170190	99	76	59,10	cyclictest	1168930	-21	kworker/u4:8+events_unbound	19:15:15	1
1170189	99	75	7,53	cyclictest	37	-21	kcompactd0	22:25:13	0
1170190	99	72	39,4	cyclictest	1190864	-21	munin-run	20:40:00	1
1170190	99	69	39,4	cyclictest	1195934	-21	sendmail	21:00:00	1
1170190	99	69	36,8	cyclictest	1170187	-21	cyclictest	20:45:00	1
1170190	99	68	32,7	cyclictest	0	-21	swapper/1	22:50:08	1
1170190	99	66	58,6	cyclictest	1193361	-21	kworker/u4:1+events_unbound	21:20:00	1
1170190	99	66	58,6	cyclictest	1193361	-21	kworker/u4:1+events_unbound	21:20:00	1
1170190	99	66	31,8	cyclictest	0	-21	swapper/1	20:20:27	1
1170190	99	65	32,6	cyclictest	1325556	-21	kworker/u4:0-rpciod	00:10:22	1
1170189	99	65	49,13	cyclictest	1170187	-21	cyclictest	21:50:01	0
1170190	99	64	56,6	cyclictest	1319810	-21	fschecks_time	23:55:22	1
1170189	99	62	11,47	cyclictest	1217892	-21	ssh	21:35:26	0
1170190	99	59	7,29	cyclictest	1265778	-21	ssh	22:40:30	1
1170189	99	59	43,11	cyclictest	1184155	-21	modprobe	20:05:00	0
1170189	99	58	45,6	cyclictest	0	-21	swapper/0	23:10:08	0
1170189	99	58	2,53	cyclictest	54	-21	kswapd0	00:25:10	0
1170189	99	56	49,5	cyclictest	1190821	-21	kworker/u4:18+events_unbound	20:36:44	0
1170189	99	56	38,14	cyclictest	1305215	-21	fschecks_time	23:35:22	0
1170189	99	56	32,19	cyclictest	256	50	irq/23-42890000.ethernet	21:10:11	0
1170189	99	56	1,49	cyclictest	1179262	-21	/usr/sbin/munin	19:45:13	0
1170190	99	55	43,8	cyclictest	1273702	-21	kworker/u4:6-rpciod	23:00:48	1
1170189	99	55	45,8	cyclictest	1333943	-21	switchtime	00:15:40	0
1170190	99	54	47,5	cyclictest	1251060	-21	/usr/sbin/munin	22:20:27	1
1170190	99	54	47,5	cyclictest	1251060	-21	/usr/sbin/munin	22:20:27	1
1170189	99	54	48,5	cyclictest	1194617	-21	kworker/u4:2+events_unbound	20:58:43	0
1170189	99	54	33,7	cyclictest	0	-21	swapper/0	22:10:28	0
1170190	99	53	7,36	cyclictest	1199539	-21	diskmemload	21:10:36	1
1170190	99	53	42,7	cyclictest	1304530	-21	latency_hist	23:35:00	1
1170190	99	53	38,9	cyclictest	1237776	-21	kworker/u4:17-rpciod	22:06:53	1
1170190	99	53	18,23	cyclictest	0	-21	swapper/1	22:55:29	1
1170189	99	53	35,16	cyclictest	1208567	-21	ssh	21:21:58	0
1170189	99	53	32,18	cyclictest	1231751	-21	latency_hist	21:55:02	0
1170189	99	52	29,21	cyclictest	1309715	-21	mailstats	23:40:36	0
1170189	99	52	15,29	cyclictest	1300818	-21	ssh	23:30:10	0
1170189	99	51	7,16	cyclictest	1172889	-21	apt-get	19:20:13	0
1170189	99	51	44,5	cyclictest	1176464	-21	kworker/u4:7+events_unbound	19:57:32	0
1170190	99	50	3,33	cyclictest	1232655	-21	ssh	21:55:32	1
1170190	99	50	32,14	cyclictest	0	-21	swapper/1	23:40:13	1
1170189	99	50	43,5	cyclictest	1168928	-21	kworker/u4:4+events_unbound	19:15:00	0
1170189	99	50	33,14	cyclictest	1213367	-21	ssh	21:30:16	0
1170189	99	50	31,5	cyclictest	1281619	-21	kworker/u4:4-rpciod	23:10:00	0
1170190	99	49	39,8	cyclictest	1182678	-21	timerwakeupswit	19:55:39	1
1170189	99	49	32,15	cyclictest	1329487	-21	aten_r7power_po	00:10:13	0
1170189	99	49	29,18	cyclictest	1175427	-21	aten2__	19:30:20	0
1170189	99	49	29,17	cyclictest	0	-21	swapper/0	22:15:42	0
1170189	99	49	21,4	cyclictest	15	1	ktimers/0	23:20:34	0
1170190	99	48	40,6	cyclictest	1180938	-21	gltestperf	19:50:28	1
1170190	99	48	35,9	cyclictest	1193378	-21	modprobe	20:50:01	1
1170189	99	48	42,5	cyclictest	1180344	-21	kworker/u4:11+events_unbound	19:52:23	0
1170189	99	48	42,4	cyclictest	1319601	-21	aten_r7power_vo	23:55:19	0
1170189	99	48	41,5	cyclictest	1342773	-21	aten_r7power_vo	00:30:23	0
1170189	99	48	41,5	cyclictest	0	-21	swapper/0	23:50:01	0
1170190	99	47	42,3	cyclictest	1219426	-21	kworker/u4:12+events_unbound	21:41:10	1
1170190	99	47	39,6	cyclictest	1345993	-21	fschecks_count	00:35:22	1
1170189	99	47	40,5	cyclictest	1182856	-21	kworker/u4:13+events_unbound	20:51:32	0
1170189	99	47	33,10	cyclictest	37	-21	kcompactd0	20:10:25	0
1170190	99	46	40,4	cyclictest	1300581	-21	munin-plugin-st	23:30:01	1
1170190	99	46	39,5	cyclictest	1333000	-21	aten2__	00:15:18	1
1170190	99	46	39,5	cyclictest	1247013	-21	uname	22:15:25	1
1170190	99	46	39,5	cyclictest	1194286	-21	sendmail_mailqu	20:50:42	1
1170190	99	46	35,8	cyclictest	1233758	-21	kworker/u4:4-rpciod	22:10:27	1
1170190	99	46	35,8	cyclictest	0	-21	swapper/1	22:30:12	1
1170189	99	46	39,4	cyclictest	256	50	irq/23-42890000.ethernet	21:15:38	0
1170189	99	46	39,4	cyclictest	256	50	irq/23-42890000.ethernet	21:15:37	0
1170189	99	46	38,6	cyclictest	1186728	-21	unixbench_multi	20:15:41	0
1170189	99	46	37,7	cyclictest	1345846	-21	aten_r7power_po	00:35:19	0
1170189	99	46	35,8	cyclictest	1235591	-21	latency_hist	22:00:01	0
1170189	99	46	28,13	cyclictest	1222294	-21	timerandwakeup	21:40:41	0
1170189	99	46	27,5	cyclictest	0	-21	swapper/0	22:40:00	0
1170189	99	46	25,17	cyclictest	15	1	ktimers/0	22:40:33	0
1170190	99	45	38,5	cyclictest	1337731	-21	sshd	00:20:42	1
1170190	99	45	38,5	cyclictest	1196373	-21	fschecks_count	21:00:20	1
1170190	99	45	37,6	cyclictest	1218510	-21	unixbench_singl	21:35:41	1
1170190	99	45	36,7	cyclictest	1316717	-21	unixbench_singl	23:50:35	1
1170190	99	45	36,7	cyclictest	1304986	-21	sh	23:35:19	1
1170190	99	45	36,7	cyclictest	1295303	-21	unixbench_multi	23:20:42	1
1170190	99	45	34,8	cyclictest	1183765	-21	ps	20:00:38	1
1170190	99	45	33,8	cyclictest	0	-21	swapper/1	00:00:08	1
1170189	99	45	7,15	cyclictest	15	1	ktimers/0	22:50:20	0
1170189	99	45	37,6	cyclictest	1177614	-21	unixbench_multi	19:35:38	0
1170189	99	45	35,7	cyclictest	1193225	-21	unixbench-2d	20:45:38	0
1170189	99	45	28,15	cyclictest	1254019	-21	munin-run	22:25:01	0
1170189	99	45	28,15	cyclictest	1254019	-21	munin-run	22:25:00	0
1170190	99	44	36,6	cyclictest	1188842	-21	fschecks_time	20:30:15	1
1170190	99	44	36,5	cyclictest	1153984	-21	kworker/u4:7-rpciod	19:10:01	1
1170190	99	44	35,6	cyclictest	1216974	-21	awk	21:35:00	1
1170190	99	44	33,7	cyclictest	0	-21	swapper/1	00:33:42	1
1170190	99	43	35,6	cyclictest	1264818	-21	latency_hist	22:40:01	1
1170190	99	43	34,7	cyclictest	1219426	-21	kworker/u4:12-rpciod	21:50:16	1
1170190	99	43	34,7	cyclictest	1175591	-21	df_abs	19:30:21	1
1170190	99	43	33,7	cyclictest	0	-21	swapper/1	23:20:00	1
1170190	99	43	33,7	cyclictest	0	-21	swapper/1	23:20:00	1
1170190	99	43	33,7	cyclictest	0	-21	swapper/1	21:20:28	1
1170189	99	43	7,22	cyclictest	1276555	-21	ssh	22:55:39	0
1170189	99	43	6,18	cyclictest	15	1	ktimers/0	22:08:12	0
1170189	99	43	35,5	cyclictest	1188762	-21	fschecks_count	20:30:15	0
1170189	99	43	35,5	cyclictest	1168928	-21	kworker/u4:4-rpciod	19:15:18	0
1170189	99	43	33,8	cyclictest	1290416	-21	basename	23:15:21	0
1170189	99	43	33,8	cyclictest	1290416	-21	basename	23:15:21	0
1170189	99	43	33,7	cyclictest	1236368	-21	/usr/sbin/munin	22:00:20	0
1170189	99	43	29,10	cyclictest	37	-21	kcompactd0	20:40:27	0
1170190	99	42	32,6	cyclictest	0	-21	swapper/1	23:10:25	1
1170189	99	42	35,5	cyclictest	1213029	-21	awk	21:30:00	0
1170189	99	42	35,5	cyclictest	1188313	-21	kworker/u4:16+events_unbound	21:06:10	0
1170190	99	41	32,7	cyclictest	1286074	-21	latency_hist	23:10:01	1
1170190	99	41	32,7	cyclictest	0	-21	swapper/1	19:35:20	1
1170190	99	41	29,9	cyclictest	0	-21	swapper/1	21:05:39	1
1170189	99	41	35,4	cyclictest	1257793	-21	netstat	22:30:23	0
1170189	99	41	33,6	cyclictest	1300673	-21	latency_hist	23:30:02	0
1170189	99	41	33,5	cyclictest	0	-21	swapper/0	19:25:21	0
1170189	99	41	24,14	cyclictest	19	1	rcuc/0	20:08:42	0
1170190	99	40	29,8	cyclictest	1188341	-21	idleruntime-cro	20:30:00	1
1170190	99	40	29,8	cyclictest	0	-21	swapper/1	20:10:23	1
1170189	99	40	33,5	cyclictest	1315775	-21	grep	23:50:15	0
1170189	99	40	33,5	cyclictest	1182856	-21	kworker/u4:13+events_unbound	20:28:10	0
1170189	99	40	31,6	cyclictest	1177999	-21	aten_r7power_en	19:40:17	0
1170190	99	39	31,6	cyclictest	1170367	-21	aten2__	19:10:17	1
1170190	99	39	30,7	cyclictest	1213073	-21	grep	21:30:01	1
1170190	99	39	21,9	cyclictest	17	1	rcu_preempt	00:08:09	1
1170189	99	39	31,6	cyclictest	1187153	-21	aten_r7power_po	20:20:17	0
1170190	99	38	30,6	cyclictest	1227820	-21	latency_hist	21:50:02	1
1170190	99	38	29,7	cyclictest	1315228	-21	idleruntime-cro	23:50:00	1
1170190	99	38	28,8	cyclictest	0	-21	swapper/1	19:45:42	1
1170190	99	38	27,8	cyclictest	0	-21	swapper/1	19:25:18	1
1170190	99	38	26,9	cyclictest	0	-21	swapper/1	20:15:20	1
1170190	99	38	23,4	cyclictest	17	1	rcu_preempt	22:27:36	1
1170189	99	38	31,5	cyclictest	1269762	-21	kworker/u4:1+events_unbound	22:49:42	0
1170189	99	38	24,12	cyclictest	0	-21	swapper/0	19:10:00	0
1170190	99	37	30,5	cyclictest	1244625	-21	kworker/u4:5+rpciod	22:50:02	1
1170189	99	37	6,29	cyclictest	15	1	ktimers/0	21:00:30	0
1170189	99	35	24,8	cyclictest	19	1	rcuc/0	00:05:30	0
1170190	99	32	20,10	cyclictest	1184345	-21	cron	20:10:00	1
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17