Portenta H7: Trying to add the Camera/GC2145 support #78
Comments
|
Quick update: Which is called from: the modules\hal\stm32\stm32cure\stm32h7xx\drivers\src\stm32h7xx_hal_dma.c So my read of this is, that in this case we receive that one message, and the DMA stream is stopped, we probably Something like: HAL_DCMI_Start_DMA ? |
|
For the fun (or pain) of it, I thought it might be interesting to try adapting to a different camera that zephyr supports, such as the So far it is not working yet. Some of the issues we have run into include: a) Reset/power down pins - The name of the power down pin is different: powerdown-gpios versus pwdn-gpios b) the port { area - looks like they use remote-endpoint-label instead of remote-endpoint and they have different c) the current OV5640 object does not currently support the size 320x240 size of image. So far I have not gotten past the camera initializing, but will try some more tomorrow. Will also try the OV2640 as well. |
|
Theres a lot more that @KurtE didn't mentioned that we noticed. As mentioned for the OV5640 camera failed to start until we releasized about reset and powerdown pin states via a LA trace that Kurte did.
Digging a bit deeper noticed in driver that we are currently using in the core it is starting the stream with IO as mipi vs dvp? Think we are using DVP for the GC2145. The above errors are coming from video_stm32_dcmi.c Though I am not seeing an error for unable to power up MIPI. The latest driver for the OV5640 actually provides support for both: https://github.com/zephyrproject-rtos/zephyr/blob/main/drivers/video/ov5640.c
|
|
Thanks @mjs513 for filling in a lot of the gaps, Will try 2640 next as it looks like there may be more complete information in the driver. One thing that I keep wondering about, is once more of Arduino integration with zephyr is completed. How do you expect it The chosen{zephyr,camera = &dcmi; Sort of maybe allows you choose to use the hardware camera subsystem, but currently I am assuming that once the real basics of the camera are working, that a lot more of the functionality of the MBED camera library will be added. But now back to trying some of the basics. |
|
Quick update: currently the OV2640 shows more promise than the OV5640. Note: I changed all of the 5640 to 2640 in overlay and config file. Also noted that it has a different I2C address of 0x30, so Build first failed as POWERDOWN pin is not defined in their yaml file... so commented out that line. Images are not good yet, Sort of like I am seeing from GC2145.
The images of the GC2145 are similar. I am not getting the same error on the OV2640... |
|
@KurtE
So don;t think I am going to do anything else on the 5640 until I see if the incoporarte the driver changes. Otherwise I will rewriting it completely |
|
Quick update: OV2640. As a comparison, here is the debug output I just ran on a Teensy Micromod using OV2640 and displayed on an ILI9341 display: Image of setup:
|
|
Did manage to get an image out of the ov7670
Still dies after a while with the errors you see with DCMI failed callback. to get it working to this point
after reading this: however doesn't look like that change made it into 4.1-rc2 |
|
Just as a quick update I switched to using portenta mid-carrier with the ov7670 and image quality improved drastically but still hit the error message when it stops noted in previous posts:
Do have to hit reset a couple of times to get more that a single image to display. Then I will get may 10-12 then it errors out. Did verify that you have to update to {OV7670_COM10, 0x03}, /* COM10 */ otherwise will get no images. |
|
Will have to try it in the morning. Looks a lot better |
|
See this issue as well: #81 |
|
I've sent a fix for the pixel format array: zephyrproject-rtos/zephyr#86810 Only the GC2145 has been tested and known to work well enough with the Giga. The rest of the sensors all have issues, bad regs, bad config, buggy drivers etc... I don't know the state of the H7 support and whether it can use the cameras or not. Note if the colors look wrong, try passing |
|
I have been having issues with the GC2145 on the GIGA and H7 but going to try and sort it out. Basically on Giga looks like its running if I watch the Serial1 spew and on the H7 will see 1 frame in the visualizer. In both cases get this message and then it halts: Again using the H7:
also set clock to 12Mhz and using FIFO_FULL then will run for a couple of minutes but will eventually get the:
|
|
@iabdalkader - @KurtE - @facchinm Did some more experimenting this morning with the GC2145 on the GIGA. First I incorporated the pr (zephyrproject-rtos/zephyr#86810) then went back to the original 2145 settings (12Mhz, 1/4 FIFO). Now for the interesting stuff
I would get some images but seems to be a synch issue going on as well
b. If I then close the visualizer and reopen the visualizer window;
it is slightly out of synch and will keep streaming with the synch changing. |
|
Will have to try it again. Been a bit since I tried it on the GIGA. On H7, I am not getting good images. Will have to try it again |
This is not an error, it just means you're not reading the frames fast enough so the video buffers FIFO is full. The streaming will not stop as the video driver always holds one video buffer. You could try increasing
This is an error, most likely DCMI FIFO overrun. It's possible to recover from this by just restarting the DMA stream, but the Zephyr driver doesn't do that. Try lowering the clock to 10MHz, if it works better we can make this the default.
You can't print anything because the frame is sent over serial, so you'll corrupt the frame if you do and break the sync. This serial frame visualizer has always been unreliable, so you may have to stop/start it a few times, and make sure the board runs first.
I just tested this with and I had no problems with streaming. The colors are wrong, but I already fixed that upstream.
|
|
Thanks @iabdalkader The Arduino_core_merge... build issue, - A few days ago on the other machine I was able to build some of the variants. Personally wish there were easier ways to switch between cameras... Than having to edit the overlay and the config file, rebuild zephyr... On the different cameras, has someone gone through and tried to fix the configurations and maybe add in the missing pieces? For example the OV7670 camera, I don't believe they have anything in it to support the OV7675 camera that Arduino sells. I definitely notice differences of trying it versus some of the older OV7670 cameras I purchased from Amazon or elsewhere. Will try again on GIGA, but need to grab another USB to serial adapter, as GIGA is always now in debug mode... Thanks again |
|
Thanks @iabdalkader for getting back so fast. SOrry it took so long in getting back to you but wanted to try with a few changes based on your feedback.
Yep - I remember that you mentioned that to facchinm in your original PR. Didn;t mean to say that was an error sorry.
@KurtE discussed that in the first post so we are on own on that one.
Yep thats why I mentioned it since I thought it was kind of funny. And yeah have to start and stop it several times to get to work. Glad you confirmed it. If I wait for continuous spew (even with vido buffers = 4)
This is at 10 mhz by the way PS. First if I was going to do anything serious with the camera I would be using OpenMV!!!! Second is there an alternate way to view the images with openmv ide or the arducam windows app? |
|
well made a couple of minor changes and the visualizer seems to be working better. In the sketch change loop to and in the processing sketch in setup kind of a handshake |
|
It could be a serial issue on Windows, maybe it's buffering the data, as it works fine on Linux. Could you try the mbed core? If you get similar results then it's definitely a Windows issue.
I don't know anything about arducam app, but OpenMV uses a debugging protocol over serial, and the IDE is more fault-tolerant, but it will never work with this. There's a websockets viewer in mbed core. The sketch will need to be update to support work with it though (see CameraCaptureWebSerial example). |
|
Sorry I didn't get back to you all yesterday - kind of ran out of steam As for the GC2145 on the GIGA and the Visualizer sketch I never had much luck with it working with mbed on the GIGA thats why I just loaded up OpenMv on it and it worked like a charm. With that said I hack up the camera api to add: wanted to see if I could send a BMP to the arducam windows app (supports jpeg and bmp images) and it kind of worked but not well. So reverted back to the H7 and used a modified sketch that @KurtE generated and seems I am able to get past the
now when I get that warning I will see in the visualizer However with that said I am now getting an error message: that I am not sure how to handle. As for restarting DMA I did notice in video_stm32_dcmi.c a command they use when they start stream: not sure we can use that to restart DMA? |
|
I think I tried to recover by using things like: But I believe it gave me a lot of errors. And/or they were not in the export table... forgot what I ran into. Will try more later. Now I trying to figure out how to force a pin into Analog mode. |
|
Got energetic and ran the GC2145 on the H7 and guess what its working as well as the ov7670, same behavior as described in my last post. No issues with Visualizer.
|
|
When I was testing the camera to display, I had to restart the DMA stream in the Zephyr driver ( I see above that it seems to be working fine. Note the colors issue has been fixed upstream. |
I did try to reduce the clock to 10mhz and it helped with the dropped frames but still eventually stopped just before seeing Now on the Giga I did manage to visualizer working by adding an extra myPort.write(1) in setup. Now seeing corrupt images at 10mhz and fifo_1_4
|
For the fun of it, I thought I would try to add the Camera support that was added to the GIGA to the Portenta H7.
I now have it building and the camera makes it through the begin call, but the camera soon errors out:
In case anyone wishes to play along:
Here are my overlay and config file.
arduino_portenta_h7_m7.zip
I also needed to edit the fixups.c to allow the Portenta to work.
Changed the line that had GIGA and added portenta:
#if (defined(CONFIG_BOARD_ARDUINO_GIGA_R1) || defined(CONFIG_BOARD_ARDUINO_PORTENTA_H7)) && defined(CONFIG_VIDEO)The error code shown in monitor window:
[00:00:14.808,000] <wrn> video_stm32_dcmi: HAL_DCMI_ErrorCallback(41)Note: I edited the file to add the error code, which is error DMA overrun I believe.
Now back to playing
The text was updated successfully, but these errors were encountered: