The source code can be found here:

https://github.com/AdroitAnandAI/Gesture-Triggered-Alarm-on-Pi-or-Jetson-Nano

Preparing to Run

To run the object tracking and gesture recognition code,

python3 ball_tracking.py

Note: The twilio account details need to be filled in correctly in sendsms.py file in order to get alert messages on mobile.

To enable alarm simulation with Pimoroni Blinkt! execute

python3 mqtt-blinkt.py

The source code can be found here:

https://github.com/AdroitAnandAI/Gesture-Triggered-Alarm-on-Pi-or-Jetson-Nano

To run Gesture (American Sign Language - ASL) Recognition models using OpenVINO

python3 gesture_recognition_demo.py -m_a OV2021_models/asl-recognition-0004.xml -m_d OV2021_models/person-detection-asl-0001.xml -i 0 -c <omz_dir>/data/dataset_classes/msasl100.json

Supported Models

• asl-recognition-0004
• common-sign-language-0001
• common-sign-language-0002
• person-detection-asl-0001

if you get an error on module import, then add this path: <omz_dir>/demos/common/python to the system path.

The source code can be found here:

https://github.com/AdroitAnandAI/Gesture-Triggered-Alarm-on-Pi-or-Jetson-Nano

How to use?

To run the openvino models in RPI,

python3 object_detection_demo.py -d MYRIAD -i 0 --labels <omz_dir>/data/dataset_classes/voc_20cl_bkgr.txt -m <intel openvino model directory>/face-detection-0200/FP32/face-detection-0200.xml -at ssd

To detect an object of your choice, we need to follow these steps:

• Data Generation: Gather images of similar objects.
• Image Annotation: Label the objects with bounding box.
• API Installation: Install TensorFlow Object Detection API.
• Train & Validate Model: Using annotated images.
• Freeze the Model: To enable mobile deployment.
• Deploy and Run: In mobile or virtual environment.

Now we need to convert the generated XML files to a format suitable for training. Download the project from here and use FoodDetection.ipynb to convert the generated XML files to CSV. Generate TFRecord files using code adapted from this raccoon detector to optimize the data feed. The train &test data are separately handled in the code. Modify the train folder name in the TFRecord generator .py file, if you wish to train other data-sets.

TFRecord is TensorFlows binary storage format. It reduces the training time of your model, as binary data takes up less space and disk read more efficient.

ipython notebook FoodDetection.ipynb
python generate_tfrecord.py
mv test.record data
mv train.record data

API Installation

We will use MobileNet model for the neural network architecture and Single Shot Detection to locate the bounding boxes. Mobilenet-SSD architecture is designed to use in mobile applications. 

To install TensorFlow Object Detection API, download and unzip TensorFlow Models from the repository here and execute the commands below.

cd models/research/
pip install protobuf-compiler
protoc object_detection/protos/*.proto - python_out=.
set PYTHONPATH=<cwd>\models\research;<cwd>\models\research\slim
cd ../../

Train & Validate Model

Download the pre-trained Mobileset SSD model from here and retrain it with your dataset to replace the classes as you desire. Re-training is done to reduce the training time.

Once the environment variable is set, execute the train.py file with a config parameter. Let it train till MAX_STEPS=20,000 or until loss is stabilized.

python train.py -logtostderr -train_dir=data\ -pipeline_config_path=data\ssd_mobilenet_v1_custom.config

Freeze the Model

To serve a model in production, we only need the graph and its weights. We don't need the metadata saved in .meta file, which is mostly useful to retrain the model. TF has a built-in helper function to extract what is needed for inference and create frozen graph_def.

To export the graph for inference, use the latest checkpoint file number stored inside "data" folder. The frozen model file, frozen_inference_graph.pb is generated inside output directory, to be deployed in mobile.

rm -rf object_detection_graph
python models/research/object_detection/export_inference_graph.py -input_type image_tensor -pipeline_config_path data/ssd_mobilenet_v1_custom.config -trained_checkpoint_prefix data/model.ckpt-19945 -output_directory object_detection_graph

Deploy and Run

Download TensorFlow Android examples from here. Using Android Studio, open the project in this path and follow the steps below. Update the tensorflow/WORKSPACE file in root directory with the API level and location of the SDK and NDK.

android_sdk_repository (
name = "androidsdk",
api_level = 23,
build_tools_version = "28.0.3",
path = "C:\Users\Anand\AppData\Local\Android\Sdk",
)
 
android_ndk_repository(
name = "androidndk",
path = "C:\Users\Anand\AppData\Local\Android\Sdk\ndk-bundle",
api_level = 19,
)

• Set "def nativeBuildSystem" in build.gradle to 'none' using Android Studio
• Download quantized Mobilenet-SSD TF Lite model from here and unzip mobilenet_ssd.tflite to assets folder: tensorflow/examples/android/assets/
• Copy frozen_inference_graph.pb generated in the previous step and label_map.pbtxt in \data folder to the "assets" folder above. Edit the label file to reflect the classes to be identified.
• Update the variables TF_OD_API_MODEL_FILE and TF_OD_API_LABELS_FILE in DetectorActivity.java to above filenames with prefix "file:///android_asset/"
• Build the bundle as an APK file using Android Studio.
• Locate APK file and install on your Android mobile. Execute TF-Detect app to start object detection. The camera would turn on and detect objects real-time.

The source code can be found here:

https://github.com/AdroitAnandAI/Multilingual-Text-Inversion-Detection-of-Scanned-Images

Some tweak to find the minimum shape cost matching is required to implement OCR with shape context.

How to run:

docker pull karthik199712/computer_vision:cv

To execute the pipeline with default image:

sudo docker run -it karthik199712/computer_vision:cv main.py

To execute the pipeline with an image in the dataset, give the image path and name after --image flag.

For instance, to execute with 11.png input image, command is as below:

sudo docker run -it karthik199712/computer_vision:cv main.py --image ./data/11.png

The source code can be found here:

https://github.com/AdroitAnandAI/Facial-Attendance-on-Pi-with-LIDAR

The following pretrained OpenVINO models can be used:

• face-detection-retail-0004 and face-detection-adas-0001, to detect faces and predict their bounding boxes
• landmarks-regression-retail-0009, to predict face keypoints
• face-reidentification-retail-0095, to recognize persons

The project has been tested using OpenVINO 2019 (models included in repo) on a Raspberry Pi 4 and Pi Cam with RPLIDAR A1 M8 and Intel Movidius NCS 2.

How to Use?

First, clone this repo to a Raspberry Pi. Set up Intel Movidius NCS 2 so that you can run OpenVINO models in it.

This is enough to run the Face Recognition module. Then connect the LIDAR via USB and install the required packages. Further, you can connect a speaker to RPi after the installation of the eSpeak text-to-speech synthesizer.

pip install -r requirements.txt

python3 ./face_recognition_demo.py -m_fd OV2019-models/face-detection-retail-0004.xml -m_lm OV2019-models/landmarks-regression-retail-0009.xml -m_reid OV2019-models/face-reidentification-retail-0095.xml -d_fd MYRIAD -d_lm MYRIAD -d_reid MYRIAD --verbose -fg "Face_Gallery/"

Press 'n' to increment the date so that next-day attendance registration can be demonstrated.

On the MQTT reception side: If you want to start afresh, you need to create the table in mysql for attendance data insertion. If not, then the database file is already there in this repo (attendance.db).

create table attendance (id INTEGER PRIMARY KEY,name TEXT,date_in,time_in DATE,time_out TIME)

Copy the files inside mqtt-client-code folder to the remote machine. Execute mqtt-attendance.py to receive messages to register attendance. The twilio account details need to be filled in correctly in send-sms.py file in order to get alert messages on mobile.

python3 mqtt-attendance.py
nano send-sms.py

The source code can be found here:

https://github.com/AdroitAnandAI/SLAM-on-Raspberry-Pi

How to use this repo?

git clone https://github.com/simondlevy/BreezySLAM.git
cd to BreezySLAM/examples
sudo python3 setup.py install

git clone https://github.com/simondlevy/PyRoboViz.git
change to PyRoboViz base directory
sudo python3 setup.py install

To execute SLAM on Raspberry Pi,

python3 rpslam-thread.py

The map will be visualized in RPi itself.

To enable LIDAR Map visualization in a remote system, you can execute the MQTT version,

python3 rpslam-mqtt.py

Note: To enable persistence of the visualization map, please replace the init.py file in roboviz directory with the one in this repo and execute PyRoboViz setup script again from the PyRoboViz base directory, using the command below.

sudo python3 setup.py install

Before execution, create a directory named 'gif' inside the base directory of slam script to let the map images saved. These images are saved in time sequence so that a gif animation can be easily created.

If the MQTT visualization is slow, then the bytearray transfer might be the bottleneck. You can either connect the RPi to router using a LAN cable (to improve the speed) or reduce the dimensions of the map to reduce the size of the bytearray. Instead you can reduce the MQTT publish frequency as well.

This project is the prequel to "Solution #1: ADAS - Collision Avoidance System on Indian Cars" project.

The source code can be found here:

https://github.com/AdroitAnandAI/Self-Driving-Car-on-Indian-Roads

The optical flow of the video needs to be generated before training.

python gen_optical_flow_traindata.py

Training the Data

Three driving control parameters viz. Steering Angle, Acceleration and Brake, are trained with different models and are concurrently executed at run time to drive the car.

The model files would be generated inside '/save' directory.

python train_accel.py
python train_brake.py
python train_steer.py

To run the system:

The models for steering, acceleration, brake and gear are simultaneously executed in different sessions with corresponding computation graphs, to predict all driving control values for each frame.

python run_dataset.py

To run the system on new track:

python run_dataset_Test.py

The source code can be found here:

https://github.com/AdroitAnandAI/ADAS-Collision-Avoidance-System-on-Indian-Roads

How to Use?

Please execute the commands in this order, so that by the time the object detection module runs the LIDAR and Flash modules will be up and running. The object detection module is the publish node while the other two are subscriber nodes.

To run the LIDAR distance estimation Subscribe module:

python3 lidar_getdist.py 

To run the Flash Warning Subscriber Module:

python3 pulse_mqtt.py 

To run the main object detection Publish Module:

python3 adas_cas.py -i 0 -m ssdlite_mobilenet_v2/ssdlite_mobilenet_v2.xml -at ssd --labels ~/open_model_zoo/data/dataset_classes/coco_91cl.txt -d MYRIAD --output_resolution 640x480