Close
0%
0%

World’s first camera stabilized self balance robot

Using your phone’s camera and IMU to position stabilize a self balance robot – no additional sensors needed

serialsensorSerialSensor
Following
Liked Like project

Just one more thing

To make the experience fit your profile, pick a username and tell us what interests you.

Pick an awesome username
hackaday.io/
Your profile's URL: hackaday.io/username. Max 25 alphanumeric characters.
Pick a few interests
Projects that share your interests
People that share your interests

We found and based on your interests.

Choose more interests.

OK, I'm done! Skip
Similar projects worth following
2.2k views
2 comments
33 followers
View Gallery
In this project your phone will be used to fully stabilize a self balance robot. For controlling the position of the robot you need information about the bot’s movement. This information is normally aquired by having motors which are equipped with encoders or using stepper-motors. In this project your phone‘s camera is providing this information via visual odometry – something also used by the mars rover(s).
The performance is strongly depending on the sensors used in your phone, some just delivier better results than others. Addtionally the enviorment needs to be structured, so small movements can be detected between successive images.

  • Position Sensor standalone

    SerialSensor • 07/15/2023 at 10:53 • 0 comments

    Here a video where you can see the position output while driving around

  • Tested phones

    SerialSensor • 04/10/2022 at 12:36 • 0 comments

    Overview over the phones which have been tested. Controller parameters must be fine tuned for each device.

    PhoneAndroid Version
    		Worked?
    Samsung Galaxy S3
    		Android 4.4
    		No, processing power is not suffiecnt
    Google Nexus 4
    		Android 5
    		No, has no USB OTG functionality (and Bluetooth's latency is too high for this application)
    Blackberry PrivAndroid 6Yes
    Samsung Galaxy S6
    		Android 7
    		Yes
    LG VS30
    		Android 8
    		Yes
    Redmi 6
    		Android 9
    		No, has no Gyroscope
    Huawei P10
    		Android 9
    		Yes
    Samsung Galaxy S20
    		Android 10
    		Yes

View all 2 project logs

Enjoy this project?

Share

Discussions

SerialSensor wrote 07/15/2023 at 10:22 • point

For specifically this application latency is everything. You already have a ton of delays until you get the latest data from the internal sensors. Espcially the camera of various phones I've seen 40-100ms latency. So you are already on the edge, since the system is unstable. Therefore you cannot allow additonally high latency, which I would expect the data transmission via screen blinking would introduce.
Do you have some numbers here?

  Are you sure? yes | no

James Newton wrote 07/15/2023 at 04:15 • point

If this could be done in Javascript on a web page in the phone (access to the camera is possible, I'm just not sure it can process the data fast enough) then it would work with this system for interfacing with any smartphone:
https://hackaday.io/project/184720-web-smart-phone-screen-blink-bot

  Are you sure? yes | no

Similar Projects

This project uses a roof mounted camera for path planing and tracking of robots
Project Owner Contributor

Image processing based path planning and tracking

muhammad-sufyan-razaMuhammad Sufyan Raza

A robot that can retrace a route by watching a prerecorded video of it.
Project Owner Contributor

RobleyVision: Route Following From Video

siliconslothSiliconSloth

This is ongoing project of the mobile robot that can move and collect data in rough terrain
Project Owner Contributor

6WD all terrain robot

codersilvercodersilver

This is ongoing project of the mobile robot that can move in a rough terrain
Project Owner Contributor

4WD all terrain robot

codersilvercodersilver

Does this project spark your interest?

Become a member to follow this project and never miss any updates

By using our website and services, you expressly agree to the placement of our performance, functionality, and advertising cookies. Learn More

Yes, delete it Cancel

Report project as inappropriate

You are about to report the project "World’s first camera stabilized self balance robot", please tell us the reason.

Send message

Your application has been submitted.

Remove Member

Are you sure you want to remove yourself as a member for this project?

Project owner will be notified upon removal.