OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of system in rack #9, slot #3

Click here to display the system's profile data or here to proceed to next system.

Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system

Characteristics of the 100 highest latencies:

System rack9slot3.osadl.org (updated Wed Oct 02, 2019 00:45:06)
Delayed (victim)					Switcher (culprit)			Timestamp	CPU
PID	Prio	Total latency (µs)	T^(,W^*) latency (µs)	Cmd	PID	Prio	Cmd	Timestamp	CPU
3545	2	246	171,19	sleep2	0	-21	swapper/2	19:07:56	2
3616	2	244	170,45	sleep1	0	-21	swapper/1	19:08:50	1
3618	2	233	169,54	sleep3	0	-21	swapper/3	19:08:52	3
3513	2	223	193,19	sleep0	0	-21	swapper/0	19:07:30	0
6944	2	51	0,8	sleep2	32	1	ktimersoftd/2	23:10:19	2
3912	99	49	19,6	cyclictest	33	-21	ksoftirqd/2	21:35:04	2
3912	99	48	44,2	cyclictest	33	-21	ksoftirqd/2	21:20:14	2
3912	99	47	42,3	cyclictest	33	-21	ksoftirqd/2	21:10:17	2
3912	99	47	2,3	cyclictest	12	1	rcu_preempt	21:18:23	2
3912	99	46	21,3	cyclictest	33	-21	ksoftirqd/2	22:55:01	2
3912	99	46	0,7	cyclictest	33	-21	ksoftirqd/2	22:38:37	2
3912	99	45	41,3	cyclictest	33	-21	ksoftirqd/2	00:10:01	2
3912	99	45	40,3	cyclictest	33	-21	ksoftirqd/2	19:45:13	2
3912	99	45	40,2	cyclictest	33	-21	ksoftirqd/2	20:20:17	2
3912	99	45	23,8	cyclictest	33	-21	ksoftirqd/2	22:15:09	2
3912	99	44	40,2	cyclictest	33	-21	ksoftirqd/2	23:39:59	2
3912	99	44	39,3	cyclictest	33	-21	ksoftirqd/2	20:50:06	2
3912	99	44	20,3	cyclictest	33	-21	ksoftirqd/2	21:40:16	2
3912	99	44	14,6	cyclictest	33	-21	ksoftirqd/2	21:10:00	2
3912	99	44	0,5	cyclictest	12	1	rcu_preempt	22:05:17	2
3911	99	44	39,3	cyclictest	25	-21	ksoftirqd/1	22:10:06	1
3912	99	43	39,2	cyclictest	33	-21	ksoftirqd/2	20:35:09	2
3912	99	43	38,3	cyclictest	33	-21	ksoftirqd/2	22:20:18	2
3912	99	43	38,3	cyclictest	33	-21	ksoftirqd/2	20:00:12	2
3912	99	43	20,4	cyclictest	33	-21	ksoftirqd/2	21:33:35	2
3912	99	43	20,3	cyclictest	33	-21	ksoftirqd/2	21:50:08	2
3912	99	43	18,3	cyclictest	33	-21	ksoftirqd/2	23:46:38	2
3912	99	43	17,5	cyclictest	33	-21	ksoftirqd/2	00:03:40	2
3912	99	43	14,3	cyclictest	33	-21	ksoftirqd/2	22:30:10	2
3912	99	43	14,3	cyclictest	33	-21	ksoftirqd/2	22:30:09	2
3912	99	42	38,3	cyclictest	33	-21	ksoftirqd/2	00:32:04	2
3912	99	42	21,9	cyclictest	33	-21	ksoftirqd/2	20:55:19	2
3912	99	42	18,4	cyclictest	33	-21	ksoftirqd/2	22:56:00	2
3912	99	42	16,4	cyclictest	33	-21	ksoftirqd/2	19:15:13	2
25679	2	42	0,1	sleep1	0	-21	swapper/1	23:55:17	1
3912	99	41	37,2	cyclictest	33	-21	ksoftirqd/2	23:05:16	2
3912	99	41	36,3	cyclictest	33	-21	ksoftirqd/2	20:41:09	2
3912	99	41	17,3	cyclictest	33	-21	ksoftirqd/2	23:35:00	2
3912	99	41	17,3	cyclictest	33	-21	ksoftirqd/2	22:28:14	2
3912	99	41	17,3	cyclictest	33	-21	ksoftirqd/2	22:28:13	2
3912	99	41	16,3	cyclictest	33	-21	ksoftirqd/2	23:50:07	2
3912	99	41	15,7	cyclictest	33	-21	ksoftirqd/2	23:55:18	2
3913	99	40	1,2	cyclictest	0	-21	swapper/3	22:05:06	3
3912	99	40	36,3	cyclictest	33	-21	ksoftirqd/2	23:25:07	2
3912	99	40	15,5	cyclictest	33	-21	ksoftirqd/2	19:27:50	2
3912	99	40	14,4	cyclictest	33	-21	ksoftirqd/2	19:55:15	2
3912	99	40	14,4	cyclictest	33	-21	ksoftirqd/2	19:55:15	2
3912	99	40	13,6	cyclictest	33	-21	ksoftirqd/2	00:20:00	2
3912	99	39	6,3	cyclictest	33	-21	ksoftirqd/2	20:30:10	2
3912	99	39	5,7	cyclictest	33	-21	ksoftirqd/2	00:05:15	2
3912	99	39	17,7	cyclictest	33	-21	ksoftirqd/2	20:05:08	2
3912	99	39	17,3	cyclictest	33	-21	ksoftirqd/2	19:35:36	2
3912	99	39	14,3	cyclictest	33	-21	ksoftirqd/2	19:30:15	2
3912	99	39	13,3	cyclictest	33	-21	ksoftirqd/2	20:10:07	2
3910	99	39	33,3	cyclictest	9	-21	ksoftirqd/0	21:55:06	0
3910	99	39	31,5	cyclictest	9	-21	ksoftirqd/0	19:40:08	0
3912	99	38	9,5	cyclictest	33	-21	ksoftirqd/2	22:40:20	2
3912	99	38	29,2	cyclictest	33	-21	ksoftirqd/2	23:40:14	2
3912	99	38	29,2	cyclictest	33	-21	ksoftirqd/2	21:45:05	2
3912	99	38	19,7	cyclictest	33	-21	ksoftirqd/2	20:47:03	2
3912	99	38	17,3	cyclictest	33	-21	ksoftirqd/2	00:27:04	2
3912	99	38	14,3	cyclictest	33	-21	ksoftirqd/2	23:17:09	2
3913	99	37	1,5	cyclictest	23841	-21	chrt	22:35:07	3
3913	99	37	0,3	cyclictest	16475	-21	grep	22:15:15	3
3912	99	37	1,5	cyclictest	33	-21	ksoftirqd/2	19:40:09	2
3912	99	37	15,3	cyclictest	33	-21	ksoftirqd/2	19:50:18	2
3912	99	37	15,3	cyclictest	33	-21	ksoftirqd/2	19:12:38	2
3912	99	37	12,5	cyclictest	33	-21	ksoftirqd/2	23:25:00	2
3912	99	37	1,1	cyclictest	0	-21	swapper/2	23:00:10	2
3912	99	37	10,4	cyclictest	33	-21	ksoftirqd/2	22:10:03	2
3910	99	37	31,3	cyclictest	9	-21	ksoftirqd/0	23:24:59	0
3913	99	36	14,19	cyclictest	4221	-21	sed	23:05:09	3
3913	99	36	12,21	cyclictest	20891	-21	cat	19:50:11	3
3913	99	36	11,21	cyclictest	8993	-21	grep	19:20:13	3
3912	99	36	6,10	cyclictest	33	-21	ksoftirqd/2	22:00:07	2
3912	99	36	33,2	cyclictest	33	-21	ksoftirqd/2	22:45:08	2
3912	99	36	31,3	cyclictest	33	-21	ksoftirqd/2	20:15:11	2
3911	99	36	14,18	cyclictest	1370	-21	snmpd	19:57:01	1
3911	99	36	14,18	cyclictest	1370	-21	snmpd	19:57:00	1
3913	99	35	12,20	cyclictest	17207	-21	sed	23:35:17	3
3913	99	35	0,3	cyclictest	22816	-21	sendmail_mailqu	22:30:17	3
3913	99	35	0,3	cyclictest	22816	-21	sendmail_mailqu	22:30:16	3
3913	99	34	17,13	cyclictest	10119	-21	apt-get	19:25:04	3
3913	99	34	16,15	cyclictest	1370	-21	snmpd	19:13:37	3
3912	99	34	4,2	cyclictest	33	-21	ksoftirqd/2	00:37:37	2
3912	99	34	23,2	cyclictest	33	-21	ksoftirqd/2	21:25:12	2
3912	99	34	17,9	cyclictest	33	-21	ksoftirqd/2	00:21:32	2
3910	99	34	6,3	cyclictest	12	1	rcu_preempt	22:15:18	0
3910	99	34	28,4	cyclictest	20610	-21	fschecks_count	23:45:10	0
3913	99	33	9,21	cyclictest	23519	-21	fschecks_count	21:15:10	3
3913	99	33	27,4	cyclictest	41	-21	ksoftirqd/3	23:10:27	3
3913	99	33	14,16	cyclictest	1141	-21	/usr/sbin/munin	00:15:11	3
3912	99	33	30,2	cyclictest	33	-21	ksoftirqd/2	21:00:16	2
3912	99	33	0,5	cyclictest	33	-21	ksoftirqd/2	21:57:08	2
3911	99	33	28,3	cyclictest	24	1	ktimersoftd/1	20:40:01	1
3910	99	33	3,4	cyclictest	9	-21	ksoftirqd/0	21:15:18	0
3910	99	33	2,7	cyclictest	9	-21	ksoftirqd/0	22:55:10	0
3910	99	33	2,7	cyclictest	12	1	rcu_preempt	23:40:09	0
3913	99	32	4,7	cyclictest	7884	-21	gltestperf	21:55:11	3
3913	99	32	26,4	cyclictest	27990	-21	cat	21:25:14	3
^Timer ^*Wakeup (latency=timer+wakeup+contextswitch)


Access: public		Short link to this page
This page was last modified on 2019-05-28 - 00:17