usb: device_next: new USB Video Class (UVC) implementation #76798
Conversation
|
I am grateful for the work on the USB and Video stacks of Zephyr, as well as the entire Zephyr tree, on the shoulder of which this is built. |
josuah
added
priority: low
|
Force push:
|
|
Force-push:
|
| .dwMinBitRate = sys_cpu_to_le32(15360000), \ | ||
| .dwMaxBitRate = sys_cpu_to_le32(15360000), \ |
There was a problem hiding this comment.
Where are these values coming from?
There was a problem hiding this comment.
These are chosen arbitrarily, and I need to think about what should I do here: pick reasonable defaults? Deduce from other values? Let the user figure out by exposing a devicetree option?
There was a problem hiding this comment.
In future work, it will be possible to ask the sensor for VIDEO_CID_PIXEL_RATE in combination with video_bits_per_pixel() to generate this field.
| .bLength = sizeof(struct usb_ep_descriptor), \ | ||
| .bDescriptorType = USB_DESC_ENDPOINT, \ | ||
| .bEndpointAddress = 0x81, \ | ||
| .bmAttributes = USB_EP_TYPE_BULK, \ |
There was a problem hiding this comment.
I believe UVC can be BULK or ISO.... but I only see BULK supported here. Do you plan to have a way to select which type of EP?
There was a problem hiding this comment.
I forgot to mention that only BULK is supported through this early version. I will need to spend more time investigating the device_next APIs for how ISO is implemented.
I cannot guarantee ETAs as I would do this feature on free time, but will add it to the roadmap: this will be needed at some point.
|
Very grateful for your reviews @XenuIsWatching I will force-push with the changes as soon as I get a chance to do so. |
|
Force-push:
|
josuah
force-pushed
the
pr-usb-uvc
branch
2 times, most recently
from
182ac7c to
cec26cc
Compare
josuah
force-pushed
the
pr-usb-uvc
branch
2 times, most recently
from
02c1087 to
4bdc1ea
Compare
|
force push: rebased on |
|
force push: changed the way descriptors are declared and introduce video controls at the descriptor level (no support for controls commands yet) |
josuah
changed the title
|
force push: implemented the UVC controls for the supported Video class controls. I did not test this yet with the samples, but on the internal fork, we could get an IMX219 sensor with exposure and gain control from the host, format selection at startup (but not runtime, see [1]). I believe that the last step for me is to test the sample on several boards that support |
Known limitations:[1]: there is currently no [2]: [3]: [4]: [5]: There are missing implementations for selector unit, extensions unit, encoding unit. TODO. [6]: UVC introduces dependencies between some controls, i.e. auto-expopsure needs to be off for exposure to be accepted. TODO. [7]: [8]: [9]: [10]: Supports custom header size, but not passing custom header data yet. [11]: Still image capture (capturing one frame at full resolution) not supported. [12]: USB3CV compliance tests not all passing since last rebase. TODO. [13]: Announcing different resolutions/FPS for different connection speed not supported. [14]: Asynchronous controls (the host setting a control, and a notification interrupt alert of completion) not supported. Supported features:[A]: Class API and enumeration [B]: [C]: [D]: [E]: Handling of control commands end-to-end from host to Zephyr video device [F]: Zephyr native Video API that allows to enqueue/dequeue frames like any other Zephyr video device. [G]: Supports fragmented frames as discussed in #66994 and #72827 [H]: [I]: Supports querying the min/max/current values of every controls end-to-end from host to Zephyr video driver. |
josuah
force-pushed
the
pr-usb-uvc
branch
3 times, most recently
from
a7cd6cb to
ea60ec0
Compare
josuah
added
the
In progress
|
Now that there is a Still "in progress" as #88924 needs to be merged first... Or I can also implement just the minimum controls. |
By having an "auto-fallback" mechanism, the VIDEO_SW_GENERATOR did shadow the fact that "platform:mimxrt1064_evk:SHIELD=dvp_fpc24_mt9m114" was missing the full specification, and therefore was not matched at all: the shield was not selected, but the CI still worked. Signed-off-by: Josuah Demangeon <[email protected]>
Make the video software generator a devicetree node, which allows enabling
several instances, and select it as chosen { zephyr,camera = &... }; node.
It can be enabled via a `video-sw-generator` snippet.
Signed-off-by: Josuah Demangeon <[email protected]>
Convert the log module declarationto the new syntax introduced in 5e34681 proposing a more compact syntax. This fixes the log level not being updated in the "capture" sample. Signed-off-by: Josuah Demangeon <[email protected]>
Variables that holds pointers to 'const struct device' typically end-up with '_dev'. The tcpserversink video sample did not have it. Add it. Signed-off-by: Josuah Demangeon <[email protected]>
The zephyr,camera was used in several samples, but was not documented. Add the missing documentation entry for it in the list. Signed-off-by: Josuah Demangeon <[email protected]>
josuah
force-pushed
the
pr-usb-uvc
branch
2 times, most recently
from
fc9d591 to
6235d5d
Compare
Add all the base controls present like they are in Linux into Zephyr, limited to those that can apply in the current system: - Buttons are left as integer for now. - Some description is modified to fit the Zephyr situation. - For the minimum number of buffer, Zephyr uses a different mechanism. - No audio support through the video subsystem. - Homogenize the wording Signed-off-by: Josuah Demangeon <[email protected]>
josuah
removed
the
In progress
Introduce a new USB Video Class (UVC) implementation from scratch. It exposes a native Zephyr Video driver interface, allowing to call the video_enqueue()/video_dequeue() interface. It will query the attached video device to learn about the pipeline capabilities, and use this to configure the USB descriptors. At runtime, this UVC implementation will send this device all the control requests, which it can then dispatch to the rest of the pipeline. The application can poll the format currently selected by the host, but will not be alerted when the host configures a new format, as there is no video.h API for it yet. Signed-off-by: Josuah Demangeon <[email protected]>
The Arduino Nicla Vision board supports the new device_next USB stack and can be used for testing USB Device features such as UVC. Signed-off-by: Josuah Demangeon <[email protected]>
Following the addition of USB Video Class, this adds a sample that makes use of the &zephyr,camera chosen node of any board to stream the video source to the host. A fallback video-emul.overlay is provided for test and debugging purpose for devices without a camera. Signed-off-by: Josuah Demangeon <[email protected]>
|
Force-push:
It is now possible to test without camera attached (again) through a snippet: nRF52840 Dongle FRDM-MCXN947 Arduino Nicla Vision RPi Pico For each board above, this is the output:
|
Dependencies:
drivers: video: introduce "GET" sub-operations #78603replaced by something else to comedrivers: video: allow allocation with a header preceding the buffer #79168not required anymorevideo: emul: virtual driver for an imager and RX engine #79482Downstream:
Build and run: