| .. SPDX-License-Identifier: GPL-2.0 |
| |
| Introduction of Uacce |
| --------------------- |
| |
| Uacce (Unified/User-space-access-intended Accelerator Framework) targets to |
| provide Shared Virtual Addressing (SVA) between accelerators and processes. |
| So accelerator can access any data structure of the main cpu. |
| This differs from the data sharing between cpu and io device, which share |
| only data content rather than address. |
| Because of the unified address, hardware and user space of process can |
| share the same virtual address in the communication. |
| Uacce takes the hardware accelerator as a heterogeneous processor, while |
| IOMMU share the same CPU page tables and as a result the same translation |
| from va to pa. |
| |
| :: |
| |
| __________________________ __________________________ |
| | | | | |
| | User application (CPU) | | Hardware Accelerator | |
| |__________________________| |__________________________| |
| |
| | | |
| | va | va |
| V V |
| __________ __________ |
| | | | | |
| | MMU | | IOMMU | |
| |__________| |__________| |
| | | |
| | | |
| V pa V pa |
| _______________________________________ |
| | | |
| | Memory | |
| |_______________________________________| |
| |
| |
| |
| Architecture |
| ------------ |
| |
| Uacce is the kernel module, taking charge of iommu and address sharing. |
| The user drivers and libraries are called WarpDrive. |
| |
| The uacce device, built around the IOMMU SVA API, can access multiple |
| address spaces, including the one without PASID. |
| |
| A virtual concept, queue, is used for the communication. It provides a |
| FIFO-like interface. And it maintains a unified address space between the |
| application and all involved hardware. |
| |
| :: |
| |
| ___________________ ________________ |
| | | user API | | |
| | WarpDrive library | ------------> | user driver | |
| |___________________| |________________| |
| | | |
| | | |
| | queue fd | |
| | | |
| | | |
| v | |
| ___________________ _________ | |
| | | | | | mmap memory |
| | Other framework | | uacce | | r/w interface |
| | crypto/nic/others | |_________| | |
| |___________________| | |
| | | | |
| | register | register | |
| | | | |
| | | | |
| | _________________ __________ | |
| | | | | | | |
| ------------- | Device Driver | | IOMMU | | |
| |_________________| |__________| | |
| | | |
| | V |
| | ___________________ |
| | | | |
| -------------------------- | Device(Hardware) | |
| |___________________| |
| |
| |
| How does it work |
| ---------------- |
| |
| Uacce uses mmap and IOMMU to play the trick. |
| |
| Uacce creates a chrdev for every device registered to it. New queue is |
| created when user application open the chrdev. The file descriptor is used |
| as the user handle of the queue. |
| The accelerator device present itself as an Uacce object, which exports as |
| a chrdev to the user space. The user application communicates with the |
| hardware by ioctl (as control path) or share memory (as data path). |
| |
| The control path to the hardware is via file operation, while data path is |
| via mmap space of the queue fd. |
| |
| The queue file address space: |
| |
| :: |
| |
| /** |
| * enum uacce_qfrt: qfrt type |
| * @UACCE_QFRT_MMIO: device mmio region |
| * @UACCE_QFRT_DUS: device user share region |
| */ |
| enum uacce_qfrt { |
| UACCE_QFRT_MMIO = 0, |
| UACCE_QFRT_DUS = 1, |
| }; |
| |
| All regions are optional and differ from device type to type. |
| Each region can be mmapped only once, otherwise -EEXIST returns. |
| |
| The device mmio region is mapped to the hardware mmio space. It is generally |
| used for doorbell or other notification to the hardware. It is not fast enough |
| as data channel. |
| |
| The device user share region is used for share data buffer between user process |
| and device. |
| |
| |
| The Uacce register API |
| ---------------------- |
| |
| The register API is defined in uacce.h. |
| |
| :: |
| |
| struct uacce_interface { |
| char name[UACCE_MAX_NAME_SIZE]; |
| unsigned int flags; |
| const struct uacce_ops *ops; |
| }; |
| |
| According to the IOMMU capability, uacce_interface flags can be: |
| |
| :: |
| |
| /** |
| * UACCE Device flags: |
| * UACCE_DEV_SVA: Shared Virtual Addresses |
| * Support PASID |
| * Support device page faults (PCI PRI or SMMU Stall) |
| */ |
| #define UACCE_DEV_SVA BIT(0) |
| |
| struct uacce_device *uacce_alloc(struct device *parent, |
| struct uacce_interface *interface); |
| int uacce_register(struct uacce_device *uacce); |
| void uacce_remove(struct uacce_device *uacce); |
| |
| uacce_register results can be: |
| |
| a. If uacce module is not compiled, ERR_PTR(-ENODEV) |
| |
| b. Succeed with the desired flags |
| |
| c. Succeed with the negotiated flags, for example |
| |
| uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA |
| |
| So user driver need check return value as well as the negotiated uacce->flags. |
| |
| |
| The user driver |
| --------------- |
| |
| The queue file mmap space will need a user driver to wrap the communication |
| protocol. Uacce provides some attributes in sysfs for the user driver to |
| match the right accelerator accordingly. |
| More details in Documentation/ABI/testing/sysfs-driver-uacce. |
