| ======================= |
| Linux UVC Gadget Driver |
| ======================= |
| |
| Overview |
| -------- |
| The UVC Gadget driver is a driver for hardware on the *device* side of a USB |
| connection. It is intended to run on a Linux system that has USB device-side |
| hardware such as boards with an OTG port. |
| |
| On the device system, once the driver is bound it appears as a V4L2 device with |
| the output capability. |
| |
| On the host side (once connected via USB cable), a device running the UVC Gadget |
| driver *and controlled by an appropriate userspace program* should appear as a UVC |
| specification compliant camera, and function appropriately with any program |
| designed to handle them. The userspace program running on the device system can |
| queue image buffers from a variety of sources to be transmitted via the USB |
| connection. Typically this would mean forwarding the buffers from a camera sensor |
| peripheral, but the source of the buffer is entirely dependent on the userspace |
| companion program. |
| |
| Configuring the device kernel |
| ----------------------------- |
| The Kconfig options USB_CONFIGFS, USB_LIBCOMPOSITE, USB_CONFIGFS_F_UVC and |
| USB_F_UVC must be selected to enable support for the UVC gadget. |
| |
| Configuring the gadget through configfs |
| --------------------------------------- |
| The UVC Gadget expects to be configured through configfs using the UVC function. |
| This allows a significant degree of flexibility, as many of a UVC device's |
| settings can be controlled this way. |
| |
| Not all of the available attributes are described here. For a complete enumeration |
| see Documentation/ABI/testing/configfs-usb-gadget-uvc |
| |
| Assumptions |
| ~~~~~~~~~~~ |
| This section assumes that you have mounted configfs at `/sys/kernel/config` and |
| created a gadget as `/sys/kernel/config/usb_gadget/g1`. |
| |
| The UVC Function |
| ~~~~~~~~~~~~~~~~ |
| |
| The first step is to create the UVC function: |
| |
| .. code-block:: bash |
| |
| # These variables will be assumed throughout the rest of the document |
| CONFIGFS="/sys/kernel/config" |
| GADGET="$CONFIGFS/usb_gadget/g1" |
| FUNCTION="$GADGET/functions/uvc.0" |
| |
| mkdir -p $FUNCTION |
| |
| Formats and Frames |
| ~~~~~~~~~~~~~~~~~~ |
| |
| You must configure the gadget by telling it which formats you support, as well |
| as the frame sizes and frame intervals that are supported for each format. In |
| the current implementation there is no way for the gadget to refuse to set a |
| format that the host instructs it to set, so it is important that this step is |
| completed *accurately* to ensure that the host never asks for a format that |
| can't be provided. |
| |
| Formats are created under the streaming/uncompressed and streaming/mjpeg configfs |
| groups, with the framesizes created under the formats in the following |
| structure: |
| |
| :: |
| |
| uvc.0 + |
| | |
| + streaming + |
| | |
| + mjpeg + |
| | | |
| | + mjpeg + |
| | | |
| | + 720p |
| | | |
| | + 1080p |
| | |
| + uncompressed + |
| | |
| + yuyv + |
| | |
| + 720p |
| | |
| + 1080p |
| |
| Each frame can then be configured with a width and height, plus the maximum |
| buffer size required to store a single frame, and finally with the supported |
| frame intervals for that format and framesize. Width and height are enumerated in |
| units of pixels, frame interval in units of 100ns. To create the structure |
| above with 2, 15 and 100 fps frameintervals for each framesize for example you |
| might do: |
| |
| .. code-block:: bash |
| |
| create_frame() { |
| # Example usage: |
| # create_frame <width> <height> <group> <format name> |
| |
| WIDTH=$1 |
| HEIGHT=$2 |
| FORMAT=$3 |
| NAME=$4 |
| |
| wdir=$FUNCTION/streaming/$FORMAT/$NAME/${HEIGHT}p |
| |
| mkdir -p $wdir |
| echo $WIDTH > $wdir/wWidth |
| echo $HEIGHT > $wdir/wHeight |
| echo $(( $WIDTH * $HEIGHT * 2 )) > $wdir/dwMaxVideoFrameBufferSize |
| cat <<EOF > $wdir/dwFrameInterval |
| 666666 |
| 100000 |
| 5000000 |
| EOF |
| } |
| |
| create_frame 1280 720 mjpeg mjpeg |
| create_frame 1920 1080 mjpeg mjpeg |
| create_frame 1280 720 uncompressed yuyv |
| create_frame 1920 1080 uncompressed yuyv |
| |
| The only uncompressed format currently supported is YUYV, which is detailed at |
| Documentation/userspace-api/media/v4l/pixfmt-packed.yuv.rst. |
| |
| Color Matching Descriptors |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| It's possible to specify some colometry information for each format you create. |
| This step is optional, and default information will be included if this step is |
| skipped; those default values follow those defined in the Color Matching Descriptor |
| section of the UVC specification. |
| |
| To create a Color Matching Descriptor, create a configfs item and set its three |
| attributes to your desired settings and then link to it from the format you wish |
| it to be associated with: |
| |
| .. code-block:: bash |
| |
| # Create a new Color Matching Descriptor |
| |
| mkdir $FUNCTION/streaming/color_matching/yuyv |
| pushd $FUNCTION/streaming/color_matching/yuyv |
| |
| echo 1 > bColorPrimaries |
| echo 1 > bTransferCharacteristics |
| echo 4 > bMatrixCoefficients |
| |
| popd |
| |
| # Create a symlink to the Color Matching Descriptor from the format's config item |
| ln -s $FUNCTION/streaming/color_matching/yuyv $FUNCTION/streaming/uncompressed/yuyv |
| |
| For details about the valid values, consult the UVC specification. Note that a |
| default color matching descriptor exists and is used by any format which does |
| not have a link to a different Color Matching Descriptor. It's possible to |
| change the attribute settings for the default descriptor, so bear in mind that if |
| you do that you are altering the defaults for any format that does not link to |
| a different one. |
| |
| |
| Header linking |
| ~~~~~~~~~~~~~~ |
| |
| The UVC specification requires that Format and Frame descriptors be preceded by |
| Headers detailing things such as the number and cumulative size of the different |
| Format descriptors that follow. This and similar operations are achieved in |
| configfs by linking between the configfs item representing the header and the |
| config items representing those other descriptors, in this manner: |
| |
| .. code-block:: bash |
| |
| mkdir $FUNCTION/streaming/header/h |
| |
| # This section links the format descriptors and their associated frames |
| # to the header |
| cd $FUNCTION/streaming/header/h |
| ln -s ../../uncompressed/yuyv |
| ln -s ../../mjpeg/mjpeg |
| |
| # This section ensures that the header will be transmitted for each |
| # speed's set of descriptors. If support for a particular speed is not |
| # needed then it can be skipped here. |
| cd ../../class/fs |
| ln -s ../../header/h |
| cd ../../class/hs |
| ln -s ../../header/h |
| cd ../../class/ss |
| ln -s ../../header/h |
| cd ../../../control |
| mkdir header/h |
| ln -s header/h class/fs |
| ln -s header/h class/ss |
| |
| |
| Extension Unit Support |
| ~~~~~~~~~~~~~~~~~~~~~~ |
| |
| A UVC Extension Unit (XU) basically provides a distinct unit to which control set |
| and get requests can be addressed. The meaning of those control requests is |
| entirely implementation dependent, but may be used to control settings outside |
| of the UVC specification (for example enabling or disabling video effects). An |
| XU can be inserted into the UVC unit chain or left free-hanging. |
| |
| Configuring an extension unit involves creating an entry in the appropriate |
| directory and setting its attributes appropriately, like so: |
| |
| .. code-block:: bash |
| |
| mkdir $FUNCTION/control/extensions/xu.0 |
| pushd $FUNCTION/control/extensions/xu.0 |
| |
| # Set the bUnitID of the Processing Unit as the source for this |
| # Extension Unit |
| echo 2 > baSourceID |
| |
| # Set this XU as the source of the default output terminal. This inserts |
| # the XU into the UVC chain between the PU and OT such that the final |
| # chain is IT > PU > XU.0 > OT |
| cat bUnitID > ../../terminal/output/default/baSourceID |
| |
| # Flag some controls as being available for use. The bmControl field is |
| # a bitmap with each bit denoting the availability of a particular |
| # control. For example to flag the 0th, 2nd and 3rd controls available: |
| echo 0x0d > bmControls |
| |
| # Set the GUID; this is a vendor-specific code identifying the XU. |
| echo -e -n "\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10" > guidExtensionCode |
| |
| popd |
| |
| The bmControls attribute and the baSourceID attribute are multi-value attributes. |
| This means that you may write multiple newline separated values to them. For |
| example to flag the 1st, 2nd, 9th and 10th controls as being available you would |
| need to write two values to bmControls, like so: |
| |
| .. code-block:: bash |
| |
| cat << EOF > bmControls |
| 0x03 |
| 0x03 |
| EOF |
| |
| The multi-value nature of the baSourceID attribute belies the fact that XUs can |
| be multiple-input, though note that this currently has no significant effect. |
| |
| The bControlSize attribute reflects the size of the bmControls attribute, and |
| similarly bNrInPins reflects the size of the baSourceID attributes. Both |
| attributes are automatically increased / decreased as you set bmControls and |
| baSourceID. It is also possible to manually increase or decrease bControlSize |
| which has the effect of truncating entries to the new size, or padding entries |
| out with 0x00, for example: |
| |
| :: |
| |
| $ cat bmControls |
| 0x03 |
| 0x05 |
| |
| $ cat bControlSize |
| 2 |
| |
| $ echo 1 > bControlSize |
| $ cat bmControls |
| 0x03 |
| |
| $ echo 2 > bControlSize |
| $ cat bmControls |
| 0x03 |
| 0x00 |
| |
| bNrInPins and baSourceID function in the same way. |
| |
| Configuring Supported Controls for Camera Terminal and Processing Unit |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The Camera Terminal and Processing Units in the UVC chain also have bmControls |
| attributes which function similarly to the same field in an Extension Unit. |
| Unlike XUs however, the meaning of the bitflag for these units is defined in |
| the UVC specification; you should consult the "Camera Terminal Descriptor" and |
| "Processing Unit Descriptor" sections for an enumeration of the flags. |
| |
| .. code-block:: bash |
| |
| # Set the Processing Unit's bmControls, flagging Brightness, Contrast |
| # and Hue as available controls: |
| echo 0x05 > $FUNCTION/control/processing/default/bmControls |
| |
| # Set the Camera Terminal's bmControls, flagging Focus Absolute and |
| # Focus Relative as available controls: |
| echo 0x60 > $FUNCTION/control/terminal/camera/default/bmControls |
| |
| If you do not set these fields then by default the Auto-Exposure Mode control |
| for the Camera Terminal and the Brightness control for the Processing Unit will |
| be flagged as available; if they are not supported you should set the field to |
| 0x00. |
| |
| Note that the size of the bmControls field for a Camera Terminal or Processing |
| Unit is fixed by the UVC specification, and so the bControlSize attribute is |
| read-only here. |
| |
| Custom Strings Support |
| ~~~~~~~~~~~~~~~~~~~~~~ |
| |
| String descriptors that provide a textual description for various parts of a |
| USB device can be defined in the usual place within USB configfs, and may then |
| be linked to from the UVC function root or from Extension Unit directories to |
| assign those strings as descriptors: |
| |
| .. code-block:: bash |
| |
| # Create a string descriptor in us-EN and link to it from the function |
| # root. The name of the link is significant here, as it declares this |
| # descriptor to be intended for the Interface Association Descriptor. |
| # Other significant link names at function root are vs0_desc and vs1_desc |
| # For the VideoStreaming Interface 0/1 Descriptors. |
| |
| mkdir -p $GADGET/strings/0x409/iad_desc |
| echo -n "Interface Associaton Descriptor" > $GADGET/strings/0x409/iad_desc/s |
| ln -s $GADGET/strings/0x409/iad_desc $FUNCTION/iad_desc |
| |
| # Because the link to a String Descriptor from an Extension Unit clearly |
| # associates the two, the name of this link is not significant and may |
| # be set freely. |
| |
| mkdir -p $GADGET/strings/0x409/xu.0 |
| echo -n "A Very Useful Extension Unit" > $GADGET/strings/0x409/xu.0/s |
| ln -s $GADGET/strings/0x409/xu.0 $FUNCTION/control/extensions/xu.0 |
| |
| The interrupt endpoint |
| ~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The VideoControl interface has an optional interrupt endpoint which is by default |
| disabled. This is intended to support delayed response control set requests for |
| UVC (which should respond through the interrupt endpoint rather than tying up |
| endpoint 0). At present support for sending data through this endpoint is missing |
| and so it is left disabled to avoid confusion. If you wish to enable it you can |
| do so through the configfs attribute: |
| |
| .. code-block:: bash |
| |
| echo 1 > $FUNCTION/control/enable_interrupt_ep |
| |
| Bandwidth configuration |
| ~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| There are three attributes which control the bandwidth of the USB connection. |
| These live in the function root and can be set within limits: |
| |
| .. code-block:: bash |
| |
| # streaming_interval sets bInterval. Values range from 1..255 |
| echo 1 > $FUNCTION/streaming_interval |
| |
| # streaming_maxpacket sets wMaxPacketSize. Valid values are 1024/2048/3072 |
| echo 3072 > $FUNCTION/streaming_maxpacket |
| |
| # streaming_maxburst sets bMaxBurst. Valid values are 1..15 |
| echo 1 > $FUNCTION/streaming_maxburst |
| |
| |
| The values passed here will be clamped to valid values according to the UVC |
| specification (which depend on the speed of the USB connection). To understand |
| how the settings influence bandwidth you should consult the UVC specifications, |
| but a rule of thumb is that increasing the streaming_maxpacket setting will |
| improve bandwidth (and thus the maximum possible framerate), whilst the same is |
| true for streaming_maxburst provided the USB connection is running at SuperSpeed. |
| Increasing streaming_interval will reduce bandwidth and framerate. |
| |
| The userspace application |
| ------------------------- |
| By itself, the UVC Gadget driver cannot do anything particularly interesting. It |
| must be paired with a userspace program that responds to UVC control requests and |
| fills buffers to be queued to the V4L2 device that the driver creates. How those |
| things are achieved is implementation dependent and beyond the scope of this |
| document, but a reference application can be found at https://gitlab.freedesktop.org/camera/uvc-gadget |