Project showcase

Face Tracker Using OpenCV and Arduino © GPL3+

Track your face using OpenCV's facial recognition.

  • 17,183 views
  • 35 comments
  • 72 respects

Components and supplies

Apps and online services

Python 3.8
Ide web
Arduino IDE

About this project

The facial recognition is a very useful tool incorporated in many modern devices to detect human faces for tracking, biometric and to recognize human activities. In this project, I have used the OpenCV's Harr cascade classifiers for detecting human faces and pan/tilt servo mechanism to track the user's face using Arduino UNO.

How it Works?

Facial detection identifies and localizes human faces and ignores any background objects such as curtain, windows, trees, etc. OpenCV uses Harr cascade of classifiers where each frame of the video is passed through stages of classifiers and if the frame passes through all the classifiers, the face is present else the frame is discarded from the classifier i.e the face is not detected.

The OpenCV returns the cartesian coordinates of the image upon detection along with the height and width. From these coordinates, the center coordinates of the image can be calculated using x+width/2 and y+height/2.

These coordinates are passed to the Arduino UNO using the pyserial library when the face is detected. The servo's connected to the Arduino provides a pan/tilt mechanism where the camera is connected to one of the servo. When the co-ordinates of the face is away from the center, then the servo will align by 2 degrees(increment or decrement)to bring it towards the center of the screen.

Detecting the face

I have used 'haarcascade_frontalface_default.xml' which is a pre- trained model for detecting human faces and can be downloaded from Git-Hub(here). Upon downloading, the xml file can be loaded using cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

The function used for face detection is cv2.CascadeClassifier.detectMultiScale() with the 'scale factor' value as 1.1(default) and 'minNeighbour' value as 6. This returns the cartesian coordinates of the image along with the height and width. Increasing the 'minNeighbour' can improve facial detection but sacrifices in execution speeds which would lead to a delayed response from the servo. Thus, the value 6 seemed optimal.

In-order to have a precise facial recognition, a plain background would be recommended as I faced some false detection due to the curtains in the background.

Calculating the Coordinates

OpenCV returns the face coordinates in terms of pixel values. By default, the video resolution is set to 640*480. The coordinates describe the top-left pixel values(x and y) along with the height and width. I have used the center coordinates of the face for reference and can be calculated using x+width/2 and y+height/2 and can be seen as a green dot. These coordinates are sent to the arduino for moving the angle of the camera.

The square in the center of the frame in white describes the region within which the center of the face i.e the green dot must be. If it is outside the squared region when the face is moved, then the servo will align the camera to bring it inside the region.

Sending Serial data to Arduino

I found this part challenging as I tried many ways to send the coordinates sequentially to the arduino but the response was slow. After spending hours figuring it out, I began looking for similar projects online until I found this project(here). It made me aware of the Serial function Serial.parseInt() which takes integer inputs from an incoming serial of bytes(check here). My approach towards sending the serial data is similar to the one used in that project.

The python sends the center coordinates in a single string. For ex: "X100Y200", the value 100 after X represents center x-coordinate and 200 represents center-y coordinate.

Arrangement of Servo

I have attached the horizontal moving servo on the shaft of the vertical moving servo in which the camera is mounted. All the attachments are made using simple rubber bands(I would not recommended it as I made use of existing material available at home).

As I am using 2 servo's for tracking, an additional 9V supply would be recommended (by means of an adapter) to the Arduino to provide sufficient current for both the servo's. In the absence of it, I have noticed some sort of vibration in them without making them move.

Libraries and Installations

This project requires pyserial and opencv libraries which I have downloaded using pip. My current python and OpenCV version is 3.8 and 4.4.0, so make sure you have a similar or a higher version. Also make sure that the XML file for face detection is saved in the same directory which contains the python script.

The python script also requires some modification(in line 9)by entering the correct COM port of your arduino before execution.

Here is a video(gif) captured by the camera tracking my face.

Code

Arduino codeArduino
//Face Tracker using OpenCV and Arduino
//by Shubham Santosh

#include<Servo.h>

Servo x, y;
int width = 640, height = 480;  // total resolution of the video
int xpos = 90, ypos = 90;  // initial positions of both Servos
void setup() {

  Serial.begin(9600);
  x.attach(9);
  y.attach(10);
  // Serial.print(width);
  //Serial.print("\t");
  //Serial.println(height);
  x.write(xpos);
  y.write(ypos);
}
const int angle = 2;   // degree of increment or decrement

void loop() {
  if (Serial.available() > 0)
  {
    int x_mid, y_mid;
    if (Serial.read() == 'X')
    {
      x_mid = Serial.parseInt();  // read center x-coordinate
      if (Serial.read() == 'Y')
        y_mid = Serial.parseInt(); // read center y-coordinate
    }
    /* adjust the servo within the squared region if the coordinates
        is outside it
    */
    if (x_mid > width / 2 + 30)
      xpos += angle;
    if (x_mid < width / 2 - 30)
      xpos -= angle;
    if (y_mid < height / 2 + 30)
      ypos -= angle;
    if (y_mid > height / 2 - 30)
      ypos += angle;


    // if the servo degree is outside its range
    if (xpos >= 180)
      xpos = 180;
    else if (xpos <= 0)
      xpos = 0;
    if (ypos >= 180)
      ypos = 180;
    else if (ypos <= 0)
      ypos = 0;

    x.write(xpos);
    y.write(ypos);

    // used for testing
    //Serial.print("\t");
    // Serial.print(x_mid);
    // Serial.print("\t");
    // Serial.println(y_mid);
  }
}
Python scriptPython
#Face tracker using OpenCV and Arduino
#by Shubham Santosh

import cv2
import serial,time
face_cascade= cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
cap=cv2.VideoCapture(1)
#fourcc= cv2.VideoWriter_fourcc(*'XVID')
ArduinoSerial=serial.Serial('com7',9600,timeout=0.1)
#out= cv2.VideoWriter('face detection4.avi',fourcc,20.0,(640,480))
time.sleep(1)

while cap.isOpened():
    ret, frame= cap.read()
    frame=cv2.flip(frame,1)  #mirror the image
    #print(frame.shape)
    gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
    faces= face_cascade.detectMultiScale(gray,1.1,6)  #detect the face
    for x,y,w,h in faces:
        #sending coordinates to Arduino
        string='X{0:d}Y{1:d}'.format((x+w//2),(y+h//2))
        print(string)
        ArduinoSerial.write(string.encode('utf-8'))
        #plot the center of the face
        cv2.circle(frame,(x+w//2,y+h//2),2,(0,255,0),2)
        #plot the roi
        cv2.rectangle(frame,(x,y),(x+w,y+h),(0,0,255),3)
    #plot the squared region in the center of the screen
    cv2.rectangle(frame,(640//2-30,480//2-30),
                 (640//2+30,480//2+30),
                  (255,255,255),3)
    #out.write(frame)
    cv2.imshow('img',frame)
    #cv2.imwrite('output_img.jpg',frame)
    '''for testing purpose
    read= str(ArduinoSerial.readline(ArduinoSerial.inWaiting()))
    time.sleep(0.05)
    print('data from arduino:'+read)
    '''
    # press q to Quit
    if cv2.waitKey(10)&0xFF== ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

Schematics

Schematic
Connect additional 9V supply to Arduino.
Face tracker bb bova3qa9oa

Comments

Similar projects you might like

Face Tracking Using Arduino

Project tutorial by Harsh Dethe

  • 32,845 views
  • 9 comments
  • 25 respects

Solar Tracker V2.0

Project tutorial by Brown Dog Gadgets

  • 30,336 views
  • 4 comments
  • 93 respects

Test the Effectiveness of Your DIY Face Mask

Project tutorial by Chris Munz

  • 3,718 views
  • 1 comment
  • 7 respects

Smart fire detection using opencv and python

Project showcase by Sagnik Ghosh

  • 1,422 views
  • 0 comments
  • 1 respect

Solar Panel Sun Tracker - Phone Charger

Project tutorial by FIELDING

  • 73,474 views
  • 20 comments
  • 230 respects

Making Roomba Smarter (800 Series)

Project showcase by Sam Baker

  • 11,516 views
  • 3 comments
  • 33 respects
Add projectSign up / Login