Alex Deucher | 2a8d78d | 2022-04-22 15:26:56 -0400 | [diff] [blame] | 1 | ================= |
| 2 | x86 IOMMU Support |
| 3 | ================= |
| 4 | |
| 5 | The architecture specs can be obtained from the below locations. |
| 6 | |
| 7 | - Intel: http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf |
| 8 | - AMD: https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf |
| 9 | |
| 10 | This guide gives a quick cheat sheet for some basic understanding. |
| 11 | |
| 12 | Basic stuff |
| 13 | ----------- |
| 14 | |
| 15 | ACPI enumerates and lists the different IOMMUs on the platform, and |
| 16 | device scope relationships between devices and which IOMMU controls |
| 17 | them. |
| 18 | |
| 19 | Some ACPI Keywords: |
| 20 | |
| 21 | - DMAR - Intel DMA Remapping table |
| 22 | - DRHD - Intel DMA Remapping Hardware Unit Definition |
| 23 | - RMRR - Intel Reserved Memory Region Reporting Structure |
| 24 | - IVRS - AMD I/O Virtualization Reporting Structure |
| 25 | - IVDB - AMD I/O Virtualization Definition Block |
| 26 | - IVHD - AMD I/O Virtualization Hardware Definition |
| 27 | |
| 28 | What is Intel RMRR? |
| 29 | ^^^^^^^^^^^^^^^^^^^ |
| 30 | |
| 31 | There are some devices the BIOS controls, for e.g USB devices to perform |
| 32 | PS2 emulation. The regions of memory used for these devices are marked |
| 33 | reserved in the e820 map. When we turn on DMA translation, DMA to those |
| 34 | regions will fail. Hence BIOS uses RMRR to specify these regions along with |
| 35 | devices that need to access these regions. OS is expected to setup |
| 36 | unity mappings for these regions for these devices to access these regions. |
| 37 | |
| 38 | What is AMD IVRS? |
| 39 | ^^^^^^^^^^^^^^^^^ |
| 40 | |
| 41 | The architecture defines an ACPI-compatible data structure called an I/O |
| 42 | Virtualization Reporting Structure (IVRS) that is used to convey information |
| 43 | related to I/O virtualization to system software. The IVRS describes the |
| 44 | configuration and capabilities of the IOMMUs contained in the platform as |
| 45 | well as information about the devices that each IOMMU virtualizes. |
| 46 | |
| 47 | The IVRS provides information about the following: |
| 48 | |
| 49 | - IOMMUs present in the platform including their capabilities and proper configuration |
| 50 | - System I/O topology relevant to each IOMMU |
| 51 | - Peripheral devices that cannot be otherwise enumerated |
| 52 | - Memory regions used by SMI/SMM, platform firmware, and platform hardware. These are generally exclusion ranges to be configured by system software. |
| 53 | |
| 54 | How is an I/O Virtual Address (IOVA) generated? |
| 55 | ----------------------------------------------- |
| 56 | |
| 57 | Well behaved drivers call dma_map_*() calls before sending command to device |
| 58 | that needs to perform DMA. Once DMA is completed and mapping is no longer |
| 59 | required, driver performs dma_unmap_*() calls to unmap the region. |
| 60 | |
| 61 | Intel Specific Notes |
| 62 | -------------------- |
| 63 | |
| 64 | Graphics Problems? |
| 65 | ^^^^^^^^^^^^^^^^^^ |
| 66 | |
| 67 | If you encounter issues with graphics devices, you can try adding |
| 68 | option intel_iommu=igfx_off to turn off the integrated graphics engine. |
| 69 | If this fixes anything, please ensure you file a bug reporting the problem. |
| 70 | |
| 71 | Some exceptions to IOVA |
| 72 | ^^^^^^^^^^^^^^^^^^^^^^^ |
| 73 | |
| 74 | Interrupt ranges are not address translated, (0xfee00000 - 0xfeefffff). |
| 75 | The same is true for peer to peer transactions. Hence we reserve the |
| 76 | address from PCI MMIO ranges so they are not allocated for IOVA addresses. |
| 77 | |
| 78 | AMD Specific Notes |
| 79 | ------------------ |
| 80 | |
| 81 | Graphics Problems? |
| 82 | ^^^^^^^^^^^^^^^^^^ |
| 83 | |
| 84 | If you encounter issues with integrated graphics devices, you can try adding |
| 85 | option iommu=pt to the kernel command line use a 1:1 mapping for the IOMMU. If |
| 86 | this fixes anything, please ensure you file a bug reporting the problem. |
| 87 | |
| 88 | Fault reporting |
| 89 | --------------- |
| 90 | When errors are reported, the IOMMU signals via an interrupt. The fault |
| 91 | reason and device that caused it is printed on the console. |
| 92 | |
| 93 | |
| 94 | Kernel Log Samples |
| 95 | ------------------ |
| 96 | |
| 97 | Intel Boot Messages |
| 98 | ^^^^^^^^^^^^^^^^^^^ |
| 99 | |
| 100 | Something like this gets printed indicating presence of DMAR tables |
| 101 | in ACPI: |
| 102 | |
| 103 | :: |
| 104 | |
| 105 | ACPI: DMAR (v001 A M I OEMDMAR 0x00000001 MSFT 0x00000097) @ 0x000000007f5b5ef0 |
| 106 | |
| 107 | When DMAR is being processed and initialized by ACPI, prints DMAR locations |
| 108 | and any RMRR's processed: |
| 109 | |
| 110 | :: |
| 111 | |
| 112 | ACPI DMAR:Host address width 36 |
| 113 | ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000 |
| 114 | ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000 |
| 115 | ACPI DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000 |
| 116 | ACPI DMAR:RMRR base: 0x00000000000ed000 end: 0x00000000000effff |
| 117 | ACPI DMAR:RMRR base: 0x000000007f600000 end: 0x000000007fffffff |
| 118 | |
| 119 | When DMAR is enabled for use, you will notice: |
| 120 | |
| 121 | :: |
| 122 | |
| 123 | PCI-DMA: Using DMAR IOMMU |
| 124 | |
| 125 | Intel Fault reporting |
| 126 | ^^^^^^^^^^^^^^^^^^^^^ |
| 127 | |
| 128 | :: |
| 129 | |
| 130 | DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000 |
| 131 | DMAR:[fault reason 05] PTE Write access is not set |
| 132 | DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000 |
| 133 | DMAR:[fault reason 05] PTE Write access is not set |
| 134 | |
| 135 | AMD Boot Messages |
| 136 | ^^^^^^^^^^^^^^^^^ |
| 137 | |
| 138 | Something like this gets printed indicating presence of the IOMMU: |
| 139 | |
| 140 | :: |
| 141 | |
| 142 | iommu: Default domain type: Translated |
| 143 | iommu: DMA domain TLB invalidation policy: lazy mode |
| 144 | |
| 145 | AMD Fault reporting |
| 146 | ^^^^^^^^^^^^^^^^^^^ |
| 147 | |
| 148 | :: |
| 149 | |
| 150 | AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0007 address=0xffffc02000 flags=0x0000] |
| 151 | AMD-Vi: Event logged [IO_PAGE_FAULT device=07:00.0 domain=0x0007 address=0xffffc02000 flags=0x0000] |