Close
0%
0%

OpenMV

Python-powered machine vision modules

Similar projects worth following
The OpenMV project aims at making machine vision more accessible to beginners by developing a user-friendly, open-source, low-cost machine vision platform.

OpenMV cameras are programmable in Python3 and come with an extensive set of image processing functions such as face detection, keypoints descriptors, color tracking, QR and Bar codes decoding, AprilTags, GIF and MJPEG recording and more.

Additionally, OpenMV includes a cross-platform IDE (based on Qt Creator) designed specifically to support programmable cameras. The IDE allows viewing the camera's frame buffer, accessing sensor controls, uploading scripts to the camera via serial over USB (or WiFi/BLE if available) and includes a set of image processing tools to generate tags, thresholds, keypoints etc...

The OpenMV project is a THP semifinalist and was successfully funded via Kickstarter back in 2015 and has come a long way since then.

Overview:

  • Scriptable in Python3.
  • On board uSD Card or internal Flash storage for scripts, images and video.
  • RGB, YUV and JPEG Omnivision sensors (OV2640 and OV7725).
  • Recording and Streaming GIF and MJPEG to SD or external WiFi shield.
  • Extension Header breaks out UART, I2C, SPI, PWM, DAC and ADC.
  • User-friendly Python IDE to view the framebuffer and upload scripts to the camera.
  • 16MB SDRAM on-board enables uClinux to run on OpenMV2.
  • Image processing library includes:
    • Line, circle, rectangle detection.
    • Face detection with VJ (compatible with OpenCV's cascades)
    • ORB keypoints detector, descriptor, matching and tracking.
    • QR and Bar codes decoding and AprilTags support.
    • Template matching with Normalized Cross Correlation (NCC) 
    • Misc functions: kmeans, filters, scaling, sub-image, blitting and alpha blending.

The Hardware:

  • Processor: Based on STM32F ARM Cortex-M Digital Signal Controllers (DSCs) running at 168-216MHz. Features a single precision FPU, DSP instructions and a DCMI (Digital Camera Interface). The low-cost, the HW camera interface and the FPU and DSP made this particular controller a perfect match for the project.
  • Image Sensor: OpenMV1 supports many single package lens/sensors, such as the OV965x and OV2640 while OpenMV2 and OpenMV3 support a single sensor with an external lens.
  • PCB: Although it costs more, a 4-layer PCB is used for all cameras for better signal integrity and EMI issues. Additionally, using 4-layers made it possible to fit everything on the 1.0x1.3 inches OpenMV1 board. The first PCB prototypes were all ordered from OSHPark.
  • Debugging and Flashing Firmware: The Serial Wire Debugging (SWD) is broken out on all cameras for debugging with GDB and the DFU is easily accessible to upload new firmware images via USB. Additionally, the camera includes a bootloader that can be used from the IDE to easily upload new firmware images.
  • I/O Headers and Shields: The main 2.54mm headers break out SPI, I2C, USART, PWM, CAN, DAC and ADC. These headers allow interfacing extension boards (or Shields) to OpenMV to extend its capabilities.  For example, using SPI LCD with OpenMV camera to view the framebuffer:


  • The WiFi Shield:  Using the WiFi Shield gives OpenMV the ability to connect to the internet. It features an ATWINC1500 FCC Certified WiFi module which can transmit data at up to 48Mbps making it perfect for streaming video:


The Software:

OpenMV uses a lot of cool open-source software including MicroPython, ChaN's FatFS, ARM's DSP/Math libraries etc.. And it's completely programmable in Python 3! OpenMV can run Python scripts that have access to peripherals (SPI/I2C/UART, CAN, PWM, ADC and DAC), uSD filesystem, wireless, and the image processing library.

The IDE:

OpenMV includes a cross-platform IDE (based on Qt Creator) designed specifically to support programmable cameras. The IDE allows viewing the camera's frame buffer, accessing sensor controls, uploading scripts to the camera via serial over USB (or WiFi/BLE if available) and includes a set of image processing tools to generate tags, thresholds, keypoints etc..

OpenMV 1, 2 and 3:

The OpenMV1 was based on STM32F4 running at 168MHz with very small RAM and Flash. The main advantage of OpenMV1 was small form factor (1.0" x 1.3").  OpenMV2 used the newer STM32F429 runing at 180MHz, with more built-in SRAM, external 16MB SDRAM, external lens for the sensor, 2xIR LEDs, 2xServo headers and 20 I/Os. Finally, OpenMV3 uses the latest Cortex-M7 and is currently in production (see full specs below).


OpenMV1 Specs:

  • MCU (STM32F407): 168MHz, FPU, DSP, DCMI.
  • RAM: 512KB SRAM Flash: 512KBs
  • Image Sensor(s): OV965x (1.3MP) and OV2640 (2MP/JPEG)
  • I/O: USART, SPI, I2C and PWM.
  • USB: 2.0 FS.
  • SD Card: SPI.
  • Power Consumption: 120mA typical.
  • Dimensions: 1.0" x 1.3"

OpenMV2...

Read more »

  • 1 × STM32F4xx MCU ARM Cortex-M4 FPU DSP 168/180MHz
  • 1 × OV2640 CMOS 2MP/RGB/YUV/JPEG
  • 1 × 24LC128 I2C EEPROM 128KBIT 400KHZ 8MSOP
  • 1 × CRYSTAL 12MHz 12PF SMD 2.5x2.0
  • 1 × ADP222ACPZ PM/REG LDO 3.3v/2.5V 16LFCSP

View all 10 components

  • Introducing the New OpenMV-H7

    i.abdalkader09/21/2018 at 01:38 0 comments

    Hey everyone, I would like to introduce you to our next generation cameras, the OpenMV-H7. First, apologies are due for not updating this project more frequently, but it's been really hard keeping track of everything from the website to forums and emails, while keeping the software/firmware releases on schedule.  Anyway, the new OpenMV-H7 is our latest and greatest camera, designed to support prototyping multiple sensors, including the OV7725, MT9V03x Global shutter sensor and the FLIR 1, 2 and 3 thermal sensors! 

    Additionally, we're using a better uSD card socket, a high efficiency switching regulator (up to 1A), a low noise LDO for sensor analog supply and we've added a LiPo battery connector on board. The new OpenMV-H7 is still backward compatible with all the shields designed for OpenMV-F4 and OpenMV-F7. Software-wise, we now have neural network support, WiFi programming support, generally more stable firmware/software and a new UVC (webcam) firmware is coming soon.

    The new OpenMV-H7 will be release in March, if you'd like to support OpenMV and pre-order the new OpenMV-H7 please back our Kickstarter: https://kck.st/2ps2Pgx

  • OpenMV Kickstarter

    i.abdalkader01/26/2015 at 17:54 4 comments

    The OpenMV Kickstarter is finally live! Check it out http://bit.ly/openmvcam

  • OpenMV1 Prototypes

    i.abdalkader09/16/2014 at 22:18 0 comments

    We're making a small batch of OpenMV1 prototypes for beta testing, if anyone's interested, they're available now for preorder on Tindie:

    https://www.tindie.com/products/bot_thoughts/openmv-cam/

    We're only making a few of them (10-30) so the costs are higher, hopefully there will be a bigger run in the future.

  • OpenMV on Quadcopter

    i.abdalkader09/13/2014 at 15:11 4 comments

    Just got my hands on a small quad, I was think this would be the perfect application for a tiny camera like OpenMV, now all I need is an open place to record some videos from the quad :)

  • FAST/FREAK Keypoint Detection

    i.abdalkader09/11/2014 at 12:40 3 comments

    Feature/Keypoints detection is a very interesting and useful tool to have around when doing image processing,  it has endless applications from tracking objects, matching and searching images to more advanced stuff...So far I've been using SURF for this, but other than being complicated and very resource consuming, the algorithm itself is patented in the US and cannot be used for commercial stuff without a license, in addition to that, the only implementation I could find (libopensurf) is GPL'd..

    So I removed all SURF code from the repo and I'm using this relatively new super fast/lightweight algorithm, called FREAK....Note that FREAK is a keypoint descriptor, so it needs something to find keypoints first, for this I'm using a corner detection algorithm called FAST.

    To give you an idea of what you can do with this FAST/FREAK detector, I made this short video:

  • Blob Detection Revised

    i.abdalkader09/06/2014 at 00:37 0 comments

    Made some improvements to the blob detection code, it now assigns unique IDs/Labels to every detected color, based on its index, so you can identify detected blobs with their IDs. It's also faster, and uses memory much more efficiently... Here's a short video of the result:

  • Thermal Imaging/Night Vision

    i.abdalkader08/28/2014 at 00:00 5 comments

    So it seems like there are a few thermal imagers out there that made it through, so for the sake of completeness (and to eliminate the competition :D) I finished up my thermal imaging code...Here's a short video showing thermal/night vision:

    To produce the final image, temperature readings are normalized, then converted to a rainbow (using a lookup table) and then scaled up (using bilinear interpolation) and finally, the thermal image is alpha blended into an RGB image and sent to the LCD...The result looks something like this:

  • OpenMV2 Thermal, LCD and WiFi Shields

    i.abdalkader08/16/2014 at 10:03 0 comments

    So I've been quite busy making some new shields for OMV2, I have not tested everything yet, but it's looking good, so far I made a CC3000, LCD and thermal IR sensor (MLX90620) shields :

    Shields that are small enough, or that have anything to do with imaging (like the thermal shield), are designed to be installed on the front side of the camera, others, like the LCD or battery pack are designed as backpacks. 

    A few of the shields are designed to work together, like the thermal and LCD shields, so it's possible for example, to connect the LCD and have a portable thermo cam:

    Now with the HW out of the way, moving on to more glorious conquests...

  • Running uClinux on OpenMV2

    i.abdalkader08/06/2014 at 19:01 0 comments

    One of the perks of having an SDRAM on board is being able to run full fledged OSes, like uClinux, which require at least a few MBs of RAM to work...uClinux, if you're not familiar with it, is an MMU-less variant of Linux, which means it can run on low-end micros, like the STM32, that don't have an MMU.

    Original support of the STM32 micros has been added by emcraft, and a few contributors on ST forums, the kernel is configured to XIP (execute in place) so it's not relocated to SDRAM and runs directly from flash (the 2MBs of flash hosts the u-boot bootloader, kernel image and the romfs) access to the on-chip flash is pretty fast, it takes, in theory, 0-wait states.

    So with that as my starting point, I made a few changes to u-boot and the kernel, enabled SDIO, and few other interfaces, to make things work on OMV2, I also made a few tweaks to fbtft and built it in the kernel, and here's the result :)

    It takes less than 1 sec to boot, I think that's awesome.. anyway, the LCD is a JD-T1800, I designed this shield for OMV2, it has a light sensor on board which you can read to control the LCD brightness and a few passives...(I'm still waiting for stackable headers, so had to solder the wires to the board):

    I'll keep playing with that for a while, see what more I can do, next, I will probably try to get the DCMI and USB working and maybe I will try to run OpenCV :)...

  • Night Vision, SDIO/SDRAM and Focal Length

    i.abdalkader07/22/2014 at 08:34 0 comments

    So I've been doing some testing with the new OpenMV, I'm very satisfied with the results so far, after replacing the broken sensor, I couldn't wait to test the IR LEDs/Lens, here's a snapshot taken in complete darkness:

    Next, I messed around with the lens trying to see how close I can get to objects (varying the focal length), here's a couple of shots of 402's a few mm from the lens:

    This one is taken under IR:

    With the optics out of the way, I moved on to testing the SDIO/SDRAM.. Unlike OpenMV1, OMV2 uses a 4-bit SDIO running at 48MHz, to interface the uSD, it's pretty fast, I did some testing by recording a video, reading/writing files etc...

    Finally, the SDRAM, fixed the linker script to map the new memory, did some simple tests, writing/reading values, poking with gdb, seems to be working fine, but just to be sure, I'm going to write/find a proper SDRAM test and run it before relocating stuff to SDRAM.

View all 19 project logs

Enjoy this project?

Share

Discussions

Rogerio wrote 11/30/2015 at 16:13 point

hello, I tried to compile version 1, but without success.

What I did was change the Makefile file TARGET field to OPENMV1, but returns the error below:

arm-none-eabi-gcc -O2 -ggdb3 -DNDEBUG -std=gnu99 -Wall -Werror -mlittle-endian -mthumb -nostartfiles -mabi=aapcs-linux -fdata-sections -ffunction-sections -fsingle-precision-constant -Wdouble-promotion -mcpu=cortex-m4 -mfpu=fpv4-sp-d16 -mfloat-abi=hard -DARM_MATH_CM4 -D__FPU_PRESENT=1 -D__FPU_USED=1 -DUSE_USB_FS -DUSE_DEVICE_MODE -DUSE_USB_OTG_ID=0 -DHSE_VALUE=12000000 -DOPENMV1 -DSTM32F407xx -I. -Iinclude -I/root/openmv/src/cmsis/include/ -I/root/openmv/src/cmsis/include/st -I/root/openmv/src/sthal/include/ -I/root/openmv/src/fatfs/include/ -I/root/openmv/src/build/micropython/ -I/root/openmv/src/micropython/ -I/root/openmv/src/micropython/py/ -I/root/openmv/src/micropython/lib/mp-readline -I/root/openmv/src/micropython/stmhal/ -I/root/openmv/src/micropython/stmhal/usbdev/core/inc/ -I/root/openmv/src/micropython/stmhal/usbdev/class/inc/ -I/root/openmv/src/micropython/stmhal/boards/OPENMV1/ -I/root/openmv/src/omv/ -I/root/openmv/src/omv/py/ -I/root/openmv/src/omv/img/ -I/root/openmv/src/winc1500/include/ -c -o /root/openmv/src/build/omv/sdcard_spi.o sdcard_spi.c
sdcard_spi.c:33:20: fatal error: pincfg.h: No such file or directory
 #include "pincfg.h"
                    ^
compilation terminated.
make[1]: *** [/root/openmv/src/build/omv/sdcard_spi.o] Error 1
make[1]: Leaving directory `/root/openmv/src/omv'
make: *** [objs] Error 2

Can you help-me?

Congratulations for the project.

  Are you sure? yes | no

James Khatiblou wrote 11/20/2015 at 21:19 point

Is this still available for purchase?

  Are you sure? yes | no

deʃhipu wrote 11/20/2015 at 22:55 point

They are actually still working on delivering the Kickstarter product, but I think they are now past the big hurdle that they had with it, and we should see our boards soon. Then, I suppose, they can start selling it.

  Are you sure? yes | no

James Khatiblou wrote 12/01/2015 at 18:10 point

Awesome thanks for the info. If you have any updates on the time of release, please let me know. Thanks. 

  Are you sure? yes | no

iamvincentb.bravo wrote 08/25/2015 at 06:05 point

where can I buy this project????

  Are you sure? yes | no

x-labz wrote 08/10/2015 at 13:29 point

Brilliant!!!! 

  Are you sure? yes | no

Roger Boldu wrote 07/17/2015 at 10:51 point

Amazing work!! I'm trying build similar one, but I will like to do some image processing before sending the image. Do you think that STM32F4 it's enough powerful to do some light processing in the middle?  Also, do you think is it possible without the SDRAM, or which SDRAM do you recomend me for a VGA resolution?

  Are you sure? yes | no

k wrote 07/16/2015 at 16:24 point

Any word on when the cameras are available? It seems like soon. I have a project in mind that would use one for eye tracking. Also, is there anyway to use an external camera over usb with your OMV2 boards or extend the distances from the board with a longer ribbon cable on hte  OMV1 boards?

  Are you sure? yes | no

lsojka.gd wrote 07/08/2015 at 12:38 point

Hello!. Very nice project!. We are building similar one. We are trying to find good solution for transmit video stream via wifi. Can you give us suggestions how to choose wifi transmitter? What kind transmitter do you use?

  Are you sure? yes | no

i.abdalkader wrote 07/14/2015 at 05:05 point

Thank you, I'm still in the process of testing a few WiFi module, so I can't recommend anything yet. Although I wouldn't use TI's CC3xxx.

  Are you sure? yes | no

arick wrote 07/04/2015 at 12:15 point

Looks amazing, great work!  Any news on when you will have inventory available on https://openmv.io?

  Are you sure? yes | no

i.abdalkader wrote 07/05/2015 at 04:26 point

Thanks! We're talking to a few distributors, won't be selling directly, I think this is just better for everyone, stay tuned :)

  Are you sure? yes | no

Paul wrote 05/21/2015 at 14:16 point

Fantastic. could this + arduino be used for OCR or would it have to little procesing power?

  Are you sure? yes | no

i.abdalkader wrote 06/19/2015 at 16:00 point

Possibly, OCR is not implemented though.

  Are you sure? yes | no

embed wrote 04/22/2015 at 19:01 point

I have a question regarding the OV2640 could you please respond. Is there any problem with connecting it directly to a FIFO(AL422B) to capture JPEG image frame at 2MP.

I just need to capture one JPEG image frame 

  Are you sure? yes | no

i.abdalkader wrote 04/22/2015 at 19:11 point

Not that I know of.

  Are you sure? yes | no

491082384 wrote 04/20/2015 at 02:20 point

hi  very cool . why u not use support openwrt system mips cpu ? e.g : mtk7620 or more HMZ 

  Are you sure? yes | no

robertwoodliff wrote 04/06/2015 at 17:24 point

Very impressive .......... keep up the great work . 

  Are you sure? yes | no

Xun Han wrote 03/08/2015 at 13:51 point

where are the openmv2's source codes?

  Are you sure? yes | no

i.abdalkader wrote 03/11/2015 at 00:04 point

On github, same repository so far, might move them to another repo.

  Are you sure? yes | no

Antoine wrote 02/27/2015 at 10:15 point

Thank you for starting this brillant project!

Is it still possible to use a non-external lenses (single package sensor) on the newest versions (openmv2-3) like in the pictures of the openmv1?

If yes: I am looking for sensors with physical IR pass filter (that filter out visible light) and haven't been able to find any single-pack-sensor compatible with the OV2640.
I only found filters to add to the external lens (http://peauproductions.com/store/index.php?main_page=index&cPath=2_12)
That would allow to make the best use of the IR leds to isolate and follow IR trackers more accurately (and also reducing the post-processing i suppose): loads of applications in tracking :-)
are you aware of any?

  Are you sure? yes | no

i.abdalkader wrote 02/27/2015 at 13:53 point

Yes the KS cam has an external M12 lens, so those filters should fit.

  Are you sure? yes | no

Antoine wrote 02/27/2015 at 14:11 point

Thank you :)

  Are you sure? yes | no

Alan wrote 02/22/2015 at 08:39 point

This is a neat project, and I hope it ends successfully for you.  I am also using the ov2640 cameras in a project, and I am wondering if the cameras are supposed to output data with the default settings immediately when connected properly, or if they need to be initialized somehow via sccb before they output any data.  The cameras I am using are not outputting any data when I connect them to power (and RST/PWDN).

  Are you sure? yes | no

i.abdalkader wrote 02/23/2015 at 11:00 point

They must be initialized

  Are you sure? yes | no

jamesschen wrote 02/05/2015 at 13:40 point

It is a great project. If we add WiFi shield and LCD shield, could this project perform recording, WIFI transmition, and preview on LCD at the same time?

  Are you sure? yes | no

i.abdalkader wrote 02/06/2015 at 08:02 point

I'm not sure, haven't tried that before, there's only one free SPI so it might be tricky.

  Are you sure? yes | no

jamesschen wrote 02/08/2015 at 10:29 point

Thanks you :)

  Are you sure? yes | no

jacob wrote 02/02/2015 at 02:00 point

It's a very impressive project. Have you calculated the power consumption when this module is working? I wonder if it can be powered buy battery(capacity around 6000mah) , I hope to use it in home security, averagely working about 2 minutes everyday,Thanks!

  Are you sure? yes | no

i.abdalkader wrote 02/02/2015 at 12:21 point

Thank you, yes it depends on what it's doing, but typically around 140mA @5.0v

  Are you sure? yes | no

Albert Latham wrote 01/31/2015 at 00:59 point

It seems that the Eagle BRD files aren't uploading correctly to OSHPark. I checked the OSHPark design rules page and they say nothing about additional files for BRD files. I don't know if I'm doing something wrong, or what.

  Are you sure? yes | no

i.abdalkader wrote 01/31/2015 at 01:48 point

Which one ? Last time I checked they all uploaded successfully

  Are you sure? yes | no

Albert Latham wrote 02/01/2015 at 04:13 point

I tried all of them with no luck. To clarify: I only need to upload the *.brd file for any given board revision, correct?

  Are you sure? yes | no

i.abdalkader wrote 02/01/2015 at 18:23 point

Yes you just need to upload the .brd, I just tried openmv1.brd uploads fine, maybe you should contact OSHPark.

  Are you sure? yes | no

Albert Latham wrote 02/01/2015 at 20:07 point

Thanks for taking your time to check. I will try again and then contact them to see what I can do.

[edit] Seems that this was my fault. Computer issues. I apologize for even bringing this to your attention. [/edit]

  Are you sure? yes | no

i.abdalkader wrote 02/06/2015 at 08:03 point

Never mind ;)

  Are you sure? yes | no

Fab wrote 01/26/2015 at 19:26 point

Contrats for the kickstarter ! I've just pledged for an early believer open MV cam :)

Your project is great, I follow it since a couple of months and I want to support it ! Just curious : is the kickstarter based on the open mv1 or 2 design ? (Edit: According to the technical specs, I' would say it's a v2, right ?)Another suggestion : maybe it would be good for pledgers/interested kickstarters users to explain how the open openMV cam differs from the cmuCam5 pixy (or others), should they had heard of them before ?

  Are you sure? yes | no

i.abdalkader wrote 01/26/2015 at 19:39 point

Hey, it's actually a mix of 1 and 2, it has features from both cams, figured it was the best cost/benefit tradeoff, will explain how it's different if we get any questions about that, I hope you like it and thanks for the support really appreciate it :)

  Are you sure? yes | no

Fab wrote 01/26/2015 at 19:56 point

You're welcome! Thanks for the feed-back. I wish you good luck with your kickstarter's campaign ! No doubt it will be successful :)

  Are you sure? yes | no

Derek Simkowiak wrote 01/03/2015 at 06:32 point

This is awesome! But I am confused by the blog page format... What is the latest news? Is there an actual product "home page" instead of a 4 month old blog post?

I have many questions:

Is version 2 available yet? How many units were made? How much do they cost? Where can I order one?

Is version 1 still available? The provided link to tindie.com says there is a "wait list"... What is the wait time? Will more be produced in the future? Or is this abandoned for the new version 2?

What is the commercial viability of this as a business... Is this just vaporware? Did people who ordered version 1 actually get them? Where can I read some customer reviews? Do you plan to build this into a business with support?

Any info is greatly appreciated. Thanks!

  Are you sure? yes | no

i.abdalkader wrote 01/03/2015 at 08:01 point

Hi, that's actually my personal blog, I wrote about the camera a couple of times, but it's not dedicated to the project, we'll have a working home page eventually, for now, I post major updates here and there's also a google group(http://bit.ly/1Am0ty3) for updates/questions...

We sold some (OMV1) cameras/prototypes for testing and feedback, we got about 100 more backorders, so there's a KS in the works, really soon, working on the content right now.

V2 is not available and will not be used, neither will V1 actually, we'll be using a new one which has more bang for buck (external lens, IR LEDs, more I/O, small form and the 180MHz MCU) will cost roughly the same as OMV1. Continued support depends on the outcome of the KS.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates