Changbin Du | c2dea5c | 2019-05-08 23:21:20 +0800 | [diff] [blame] | 1 | .. SPDX-License-Identifier: GPL-2.0 |
| 2 | |
| 3 | ============== |
| 4 | Kernel Entries |
| 5 | ============== |
| 6 | |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 7 | This file documents some of the kernel entries in |
James C Boyd | 864d5bb | 2015-07-01 15:13:26 -0500 | [diff] [blame] | 8 | arch/x86/entry/entry_64.S. A lot of this explanation is adapted from |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 9 | an email from Ingo Molnar: |
| 10 | |
| 11 | http://lkml.kernel.org/r/<20110529191055.GC9835%40elte.hu> |
| 12 | |
| 13 | The x86 architecture has quite a few different ways to jump into |
| 14 | kernel code. Most of these entry points are registered in |
James C Boyd | 864d5bb | 2015-07-01 15:13:26 -0500 | [diff] [blame] | 15 | arch/x86/kernel/traps.c and implemented in arch/x86/entry/entry_64.S |
| 16 | for 64-bit, arch/x86/entry/entry_32.S for 32-bit and finally |
| 17 | arch/x86/entry/entry_64_compat.S which implements the 32-bit compatibility |
Luis R. Rodriguez | 96ed4cd | 2014-12-09 14:54:44 -0800 | [diff] [blame] | 18 | syscall entry points and thus provides for 32-bit processes the |
| 19 | ability to execute syscalls when running on 64-bit kernels. |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 20 | |
Luis R. Rodriguez | 96ed4cd | 2014-12-09 14:54:44 -0800 | [diff] [blame] | 21 | The IDT vector assignments are listed in arch/x86/include/asm/irq_vectors.h. |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 22 | |
| 23 | Some of these entries are: |
| 24 | |
| 25 | - system_call: syscall instruction from 64-bit code. |
| 26 | |
Ingo Molnar | 2cd2355 | 2015-06-08 08:28:07 +0200 | [diff] [blame] | 27 | - entry_INT80_compat: int 0x80 from 32-bit or 64-bit code; compat syscall |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 28 | either way. |
| 29 | |
Ingo Molnar | 2cd2355 | 2015-06-08 08:28:07 +0200 | [diff] [blame] | 30 | - entry_INT80_compat, ia32_sysenter: syscall and sysenter from 32-bit |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 31 | code |
| 32 | |
| 33 | - interrupt: An array of entries. Every IDT vector that doesn't |
| 34 | explicitly point somewhere else gets set to the corresponding |
| 35 | value in interrupts. These point to a whole array of |
| 36 | magically-generated functions that make their way to do_IRQ with |
| 37 | the interrupt number as a parameter. |
| 38 | |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 39 | - APIC interrupts: Various special-purpose interrupts for things |
| 40 | like TLB shootdown. |
| 41 | |
| 42 | - Architecturally-defined exceptions like divide_error. |
| 43 | |
| 44 | There are a few complexities here. The different x86-64 entries |
| 45 | have different calling conventions. The syscall and sysenter |
| 46 | instructions have their own peculiar calling conventions. Some of |
| 47 | the IDT entries push an error code onto the stack; others don't. |
| 48 | IDT entries using the IST alternative stack mechanism need their own |
| 49 | magic to get the stack frames right. (You can find some |
| 50 | documentation in the AMD APM, Volume 2, Chapter 8 and the Intel SDM, |
| 51 | Volume 3, Chapter 6.) |
| 52 | |
| 53 | Dealing with the swapgs instruction is especially tricky. Swapgs |
| 54 | toggles whether gs is the kernel gs or the user gs. The swapgs |
| 55 | instruction is rather fragile: it must nest perfectly and only in |
| 56 | single depth, it should only be used if entering from user mode to |
| 57 | kernel mode and then when returning to user-space, and precisely |
| 58 | so. If we mess that up even slightly, we crash. |
| 59 | |
| 60 | So when we have a secondary entry, already in kernel mode, we *must |
| 61 | not* use SWAPGS blindly - nor must we forget doing a SWAPGS when it's |
| 62 | not switched/swapped yet. |
| 63 | |
| 64 | Now, there's a secondary complication: there's a cheap way to test |
| 65 | which mode the CPU is in and an expensive way. |
| 66 | |
| 67 | The cheap way is to pick this info off the entry frame on the kernel |
Changbin Du | c2dea5c | 2019-05-08 23:21:20 +0800 | [diff] [blame] | 68 | stack, from the CS of the ptregs area of the kernel stack:: |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 69 | |
| 70 | xorl %ebx,%ebx |
| 71 | testl $3,CS+8(%rsp) |
| 72 | je error_kernelspace |
| 73 | SWAPGS |
| 74 | |
| 75 | The expensive (paranoid) way is to read back the MSR_GS_BASE value |
Changbin Du | c2dea5c | 2019-05-08 23:21:20 +0800 | [diff] [blame] | 76 | (which is what SWAPGS modifies):: |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 77 | |
| 78 | movl $1,%ebx |
| 79 | movl $MSR_GS_BASE,%ecx |
| 80 | rdmsr |
| 81 | testl %edx,%edx |
| 82 | js 1f /* negative -> in kernel */ |
| 83 | SWAPGS |
| 84 | xorl %ebx,%ebx |
Changbin Du | c2dea5c | 2019-05-08 23:21:20 +0800 | [diff] [blame] | 85 | 1: ret |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 86 | |
Andy Lutomirski | 8b4777a | 2011-06-05 13:50:18 -0400 | [diff] [blame] | 87 | If we are at an interrupt or user-trap/gate-alike boundary then we can |
| 88 | use the faster check: the stack will be a reliable indicator of |
| 89 | whether SWAPGS was already done: if we see that we are a secondary |
| 90 | entry interrupting kernel mode execution, then we know that the GS |
| 91 | base has already been switched. If it says that we interrupted |
| 92 | user-space execution then we must do the SWAPGS. |
| 93 | |
| 94 | But if we are in an NMI/MCE/DEBUG/whatever super-atomic entry context, |
| 95 | which might have triggered right after a normal entry wrote CS to the |
| 96 | stack but before we executed SWAPGS, then the only safe way to check |
| 97 | for GS is the slower method: the RDMSR. |
| 98 | |
Andy Lutomirski | 48e08d0 | 2014-11-11 12:49:41 -0800 | [diff] [blame] | 99 | Therefore, super-atomic entries (except NMI, which is handled separately) |
| 100 | must use idtentry with paranoid=1 to handle gsbase correctly. This |
| 101 | triggers three main behavior changes: |
| 102 | |
| 103 | - Interrupt entry will use the slower gsbase check. |
| 104 | - Interrupt entry from user mode will switch off the IST stack. |
| 105 | - Interrupt exit to kernel mode will not attempt to reschedule. |
| 106 | |
| 107 | We try to only use IST entries and the paranoid entry code for vectors |
| 108 | that absolutely need the more expensive check for the GS base - and we |
| 109 | generate all 'normal' entry points with the regular (faster) paranoid=0 |
| 110 | variant. |