Commit b64dfcde authored Dec 17, 2021 by Borislav Petkov

x86/mm: Prevent early boot triple-faults with instrumentation



Commit in Fixes added a global TLB flush on the early boot path, after
the kernel switches off of the trampoline page table.

Compiler profiling options enabled with GCOV_PROFILE add additional
measurement code on clang which needs to be initialized prior to
use. The global flush in x86_64_start_kernel() happens before those
initializations can happen, leading to accessing invalid memory.
GCOV_PROFILE builds with gcc are still ok so this is clang-specific.

The second issue this fixes is with KASAN: for a similar reason,
kasan_early_init() needs to have happened before KASAN-instrumented
functions are called.

Therefore, reorder the flush to happen after the KASAN early init
and prevent the compilers from adding profiling instrumentation to
native_write_cr4().

Fixes: f154f290 ("x86/mm/64: Flush global TLB on boot and AP bringup")
Reported-by: "J. Bruce Fields" <bfields@fieldses.org>
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Tested-by: Carel Si <beibei.si@intel.com>
Tested-by: "J. Bruce Fields" <bfields@fieldses.org>
Link: https://lore.kernel.org/r/20211209144141.GC25654@xsang-OptiPlex-9020

parent 35fa7452

arch/x86/kernel/cpu/common.c

+1 −1

Original line number	Diff line number	Diff line
		@@ -384,7 +384,7 @@ void native_write_cr0(unsigned long val)
		}
		EXPORT_SYMBOL(native_write_cr0);

		void native_write_cr4(unsigned long val)
		void __no_profile native_write_cr4(unsigned long val)
		{
		unsigned long bits_changed = 0;

arch/x86/kernel/head64.c

+14 −2

Original line number	Diff line number	Diff line
		@@ -483,10 +483,12 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
		/* Kill off the identity-map trampoline */
		reset_early_page_tables();

		__native_tlb_flush_global(native_read_cr4());

		clear_bss();

		/*
		* This needs to happen before kasan_early_init() because latter maps stuff
		* into that page.
		*/
		clear_page(init_top_pgt);

		/*
		@@ -498,6 +500,16 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)

		kasan_early_init();

		/*
		* Flush global TLB entries which could be left over from the trampoline page
		* table.
		*
		* This needs to happen after kasan_early_init() as KASAN-enabled .configs
		* instrument native_write_cr4() so KASAN must be initialized for that
		* instrumentation to work.
		*/
		__native_tlb_flush_global(this_cpu_read(cpu_tlbstate.cr4));

		idt_setup_early_handler();

		copy_bootdata(__va(real_mode_data));