From: John Ogness <john.ogness@linutronix.de> Date: Wed, 7 Feb 2024 14:47:01 +0106 Subject: [PATCH 03/48] printk: Avoid non-panic CPUs writing to ringbuffer Commit 13fb0f74d702 ("printk: Avoid livelock with heavy printk during panic") introduced a mechanism to silence non-panic CPUs if too many messages are being dropped. Aside from trying to workaround the livelock bugs of legacy consoles, it was also intended to avoid losing panic messages. However, if non-panic CPUs are writing to the ringbuffer, then reacting to dropped messages is too late. Another motivation is that non-finalized messages already might be skipped in panic(). In other words, random messages from non-panic CPUs might already get lost. It is better to ignore all to avoid confusion. To avoid losing panic CPU messages, silence non-panic CPUs immediately on panic. Signed-off-by: John Ogness <john.ogness@linutronix.de> Reviewed-by: Petr Mladek <pmladek@suse.com> Link: https://lore.kernel.org/r/20240207134103.1357162-13-john.ogness@linutronix.de Signed-off-by: Petr Mladek <pmladek@suse.com> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> --- kernel/printk/printk.c | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) Index: linux-6.8.2-rt10/kernel/printk/printk.c =================================================================== @ linux-6.8.2-rt10/kernel/printk/printk.c:465 @ static int console_msg_format = MSG_FORM static DEFINE_MUTEX(syslog_lock); #ifdef CONFIG_PRINTK -/* - * During panic, heavy printk by other CPUs can delay the - * panic and risk deadlock on console resources. - */ -static int __read_mostly suppress_panic_printk; - DECLARE_WAIT_QUEUE_HEAD(log_wait); /* All 3 protected by @syslog_lock. */ /* the next printk record to read by syslog(READ) or /proc/kmsg */ @ linux-6.8.2-rt10/kernel/printk/printk.c:2319 @ asmlinkage int vprintk_emit(int facility if (unlikely(suppress_printk)) return 0; - if (unlikely(suppress_panic_printk) && other_cpu_in_panic()) + /* + * The messages on the panic CPU are the most important. If + * non-panic CPUs are generating any messages, they will be + * silently dropped. + */ + if (other_cpu_in_panic()) return 0; if (level == LOGLEVEL_SCHED) { @ linux-6.8.2-rt10/kernel/printk/printk.c:2809 @ void console_prepend_dropped(struct prin bool printk_get_next_message(struct printk_message *pmsg, u64 seq, bool is_extended, bool may_suppress) { - static int panic_console_dropped; - struct printk_buffers *pbufs = pmsg->pbufs; const size_t scratchbuf_sz = sizeof(pbufs->scratchbuf); const size_t outbuf_sz = sizeof(pbufs->outbuf); @ linux-6.8.2-rt10/kernel/printk/printk.c:2836 @ bool printk_get_next_message(struct prin pmsg->seq = r.info->seq; pmsg->dropped = r.info->seq - seq; - /* - * Check for dropped messages in panic here so that printk - * suppression can occur as early as possible if necessary. - */ - if (pmsg->dropped && - panic_in_progress() && - panic_console_dropped++ > 10) { - suppress_panic_printk = 1; - pr_warn_once("Too many dropped messages. Suppress messages on non-panic CPUs to prevent livelock.\n"); - } - /* Skip record that has level above the console loglevel. */ if (may_suppress && suppress_message_printing(r.info->level)) goto out;