Project Logs

Collapse

• Status Report 3

  Nathaniel Wong • 06/11/2019 at 20:05 • 0 comments

  Design Updates

  Our team made process in redesigning the camera security system in these key areas:

  Image transmission. The camera’s Zentri AMW004 wifi module, is able to receive more image frames per second from the Microcontroller Unit (MCU), achieving up to 3-4 frames per second. This was done by increasing the UART baud rate 8x faster at 921600 Hz rather than 115200 Hz, and reducing the size of the data buffer.

  Google Cloud server solution. The cloud server now receives all images and performs person recognition and tracking in real time for a single camera. Once the coordinates and size of the person are determined, a PID controller determines a new angle the pan-tilt base should now adopt and transmits the data via HTTP.

  OpenCV facial recognition. The OpenCV library has been exploited to identify faces (frontal and profile), half body and full body to a reasonable degree of accuracy. Once a region of interest is identified, a CSRT tracker follows the person and updates the PID controller accordingly.

  PWM servo actuators. The PWM servos now respond accordingly to the angles provided by the HTTP connection; this is achieved by updating the duty cycles for each servo.

  Progress made on the project has been acceptable - we are on target to meet the project’s minimum viable product goals, including extending the system to incorporate multiple cameras. however we will not be able to explore our stretch goals of including an LCD screen.

  Planning and Organization

  Figure: Gantt Chart for the project

  Item Desc.	Mfg. Part #	Unit Price	1000 Unit Price	Quantity	In Final Design?	Total Unit Price	Total Bulk Price
  Mini Pan-Tilt Kit	1967	$18.95	$15.15	1	Yes	$62.34	$53.29
  LCD screen	181	$9.95	$7.96	1	No
  Omnivision camera module	2640	$9.44	$7.55	1	Yes
  Silicon Labs Wifi Module	AMW004	$27.69	$25.63	1	Yes
  Microchip Technology MCU	ATSAM4S8BA-AU	$5.26	$4.45	1	Yes
  Custom PCB		$1.00	$0.50	1	Yes

  Figure: Bill of Materials for the project

  Results of Market Research

  Our non-expert interviewees found a product that could recognize specific faces to be very useful in monitoring their private spaces, for example, alerting against front door package thieves or trespassers. On the topic of facial recognition, one interviewee noted that constant face detection alerts could get stale because of false alarms, and suggested using criminal face databases to reduce false alarms.

  We interviewed Prof. Jack Tumblin for his expert input on the computer-vision aspects of the security camera that we are making. Most of his suggested surrounded our efforts to reduce latency on the camera, including using grayscale images rather and RGB for transmission to achieve compression. He also suggested using a more direct form of data transmission rather than UART, or bypassing the MCU directly and using a shared bus between the sensor and wifi chip to transmit images as quickly as possible.

  In terms of image quality, Prof. Tumblin suggested reducing exposure time to reduce motion blur. To compensate, we would have to increase aperture size (if possible) and ISO settings.

  Next Steps

  By the end of the quarter, we will fine tune camera system to ensure that the face detection, tracking, and actuation work seamlessly. Amongst some measures, we will increase the UART baud rate to increase throughput, alter the server and CV configurations to improve latency, and tweak the PID controller to improve responsiveness to the detected person.

  References

  https://www.cnet.com/pictures/security-cameras-with-facial-recognition-tech-inside/

  https://docs.opencv.org/3.3.0/d7/d8b/tutorial_py_face_detection.html

  https://www.adafruit.com/product/1967

  https://www.adafruit.com/product/181

  https://usa.banggood.com/3pcs-Mini-OV2640-Camera-Module-CMOS-Image-Sensor-Module-for-Arduino-p-1452968.html

  https://www.digikey.com/product-detail/en/silicon-labs/AMW004/1586-1012-ND/5269916

  https://www.digikey.com/product-detail/en/microchip-technology/ATSAM4S8BA-AU/ATSAM4S8BA-AU-ND/3593645

• Status Report 2

  Nathaniel Wong • 06/11/2019 at 20:04 • 0 comments

  Design Updates

  Our team made significant process in reconfiguring the camera module into a web client in 4 key areas:

  Image transmission. The camera’s Zentri AMW004 wifi module, as controlled by the Microcontroller Unit (MCU), is now capable of transmitting images to the host server on Google Cloud. This is done by sending HTTP POST requests with the appropriate headers for a image/jpeg Content-Type, and the Content-Length matching the size of the image file. At time of report, the Camera system is able to output about one frame of image output every second to the server in this manner.

  Google Cloud server solution. The Google Cloud compute instance is the workhorse of the security camera system. Currently, a python Tornado web server accepts HTTP requests from any properly configured device on the internet that sends the appropriate headers. The contents of the request are stored as a jpeg device on the disk and a HTTP response indicates that the transaction was successful. A web interface allows a user to watch the image feed ‘live’ conditions, at a 200ms local refresh rate (the camera continues to be the bottleneck at one frame per second). Where appropriate, the response text will include data for x and y coordinate for the location of a detected face (see section on OpenCV).

  PWM servo actuators.  The servos were successfully tested using the breakout boards. We were able to control the servo positions on the pan-tilt base using PWM and altering the value of the PWM duty cycle.

  OpenCV facial recognition. The OpenCV library for python has been integrated into the web server. Currently, we are using a Haar Feature-based cascade classifier for face and eye detection. The classifier works reasonably well on the low pixel count of the images, but only when the face is directed squarely at the camera. It is not capable of distinguishing between faces of different individuals either.

  The overall state of the project is healthy. So far, we are on target to meet the project’s minimum viable product goals. More time will be needed than expected to integrate the PWM servos into the current solution as this will require the wifi module to interface with the microcontroller seamlessly.

  At the current moment, the stretch goals of incorporating and LCD screen and per-user facial recognition may not be feasible given the amount of development time and testing required to complete the features in the current iteration.

  Planning and Organization

  Figure: Gantt Chart for the project

  Item Desc.	Mfg. Part #	Unit Price	1000 Unit Price	Quantity	In Final Design?	Total Unit Price	Total Bulk Price
  Mini Pan-Tilt Kit	1967	$18.95	$15.15	1	Yes	$62.34	$53.29
  LCD screen	181	$9.95	$7.96	1	No
  Omnivision camera module	2640	$9.44	$7.55	1	Yes
  Silicon Labs Wifi Module	AMW004	$27.69	$25.63	1	Yes
  Microchip Technology MCU	ATSAM4S8BA-AU	$5.26	$4.45	1	Yes
  Custom PCB		$1.00	$0.50	1	Yes

  Figure: Bill of Materials for the project

  Results of Market Research

  Our non-expert interviewees found a product that could recognize specific faces to be very useful in monitoring their private spaces, for example, alerting against front door package thieves or trespassers. On the topic of facial recognition, one interviewee noted that constant face detection alerts could get stale because of false alarms, and suggested using criminal face databases to reduce false alarms.

  We will be interviewing Prof. Jack Tumblin for his expert input on the computer-vision enabled security camera product that we are making.

  Next Steps

  By the next presentation, we will have the servos autonomously operating as faces are detected by the camera system and the cloud server. This will require steps to be taken to integrate PWM functionality into the codebase as well as the ability to parse HTTP responses. Should this fail, we will operate the servos either by manually commands from a web interface or on an automated timed loop, panning across the entire room.

  We will aim to improve the OpenCV...

  Read more »

• Status report 1

  Nathaniel Wong • 06/11/2019 at 20:04 • 0 comments

  Design Requirements

  • Camera is configured as a web client
  • Deploy and cloud server to maintain connection with camera modules perform image processing
  • Use computer vision techniques to detect and classify faces seen in camera feeds
  • Cameras track face movements using mounted servo system
  • Alert is sent if the person’s face is not recognized

  Design Description

  Figure: Block diagram of proposed multi-camera system with central cloud server

  Using a hub and spoke model will result in a high latency between sensors and actuators, because a TCP round-trip will be required to process each video frame.

  The benefit to this is that the central server is capable of higher image processing loads. We will use a combination of Tornado, a lightweight python web server, and OpenCV, a powerful real-time computer vision library built on python, to our advantage on the cloud compute instance. This will allow the viewing and autonomous control of multiple video feeds..

  Planning and Organization

  	Action by	Start Date	End Date	Status
  Define project goals	Nat	Apr 2, 2019	Apr 9, 2019	Complete
  Research components	QK	Apr 2, 2019	Apr 9, 2019	Complete
  Prepare order	Nat	Apr 2, 2019	Apr 9, 2019	Complete
  Create new PCB layout	QK	Apr 9, 2019	Apr 11, 2019	Active
  Setup Amazon AWS services	Nat	Apr 9, 2019	Apr 16, 2019	Complete
  Camera client sends images to AWS	Nat	Apr 16, 2019	Apr 23, 2019	Active
  Use breakout board to test LCD&Pan-tilt	QK	Apr 17, 2019	Apr 24, 2019	Upcoming
  Put together breakout prototype	QK	Apr 18, 2019	Apr 25, 2019	Upcoming
  AWS server processes images with CV	Nat	Apr 23, 2019	May 9, 2019	Upcoming
  Make revisions and make embedded PCB	QK	Apr 25, 2019	May 2, 2019	Upcoming
  Put together embedded prototype	QK	May 9, 2019	May 16, 2019	Upcoming
  Make revisions and design revised PCB	QK	May 16, 2019	May 23, 2019	Upcoming
  Pan-tilt base reacts to location of human face	Nat	May 9, 2019	May 30, 2019	Upcoming
  Put together final prototype	All	May 30, 2019	Jun 12, 2019	Upcoming
  Prepare final presentation	All	Jun 4, 2019	Jun 12, 2019	Upcoming

  Figure: Gantt Chart for the project

  Item Desc.	Mfg. Part #	Unit Price	1000 Unit Price	Quantity
  Mini Pan-Tilt Kit	1967	18.95	15.15	1
  LCD screen	181	9.95	7.96	1

  Figure: Bill of Materials for the project

  Splitting of work

  Qiankai will largely focus on the hardware aspects of the design and fabrication process, including PCB design, servo motor control and final integration of different modules.

  Nathaniel will design the central server functionality, modify the camera to accept wifi connections to the server and stream images, and implement computer vision techniques.

  Results of Market Research

  Based on home ownership data, about 40 million middle to upper income urban households own their properties globally, and many of these households face the real threat of petty household theft. To that end, they may find value in an autonomous visual security system that deters thieves while providing homeowners of critical alerts and valuable visual evidence of trespassers.

  List of competitors

  One major brand is SimpliSafe, which provides an array of household security camera products, entry sensors and motion sensors. At a price of US$500 for a complete solution, this may be prohibitive for even that most security-conscious of users. The products also do not include face detection and recognition

  NestCam is another serious competitor that features built-in facial recognition for outdoor cameras, and is able to recognize ‘familiar persons’ with in-app alerts. Cloud storage backs up the video feeds as part of the product package. The camera retails for $349, a hefty price tag for a singular camera, or $600 for a pair.

  Other brands: Arlo, Allianca, Blink, Nest Cam, Zmodo, Dlink, Netgear, Logitech

  Results of Interviews

  We conducted interviews with two potential users, but did not manage to interview an expert. Our interviewees found a product that could recognize specific faces to be very useful in...

  Read more »

View all 3 project logs