Project tutorial
Alexa Controlled Face Recognizing Arduino Door Bell

Alexa Controlled Face Recognizing Arduino Door Bell

„Alexa, who is at the door?“ - A face recognizing Arduino camera using AWS Rekognition for my grandmother

  • 2,491 views
  • 1 comment
  • 12 respects

Components and supplies

Apps and online services

About this project

„Alexa, who is at the door?“ - A face recognizing Arduino camera using AWS Rekognition for my grandmother

Ever wanted to know who rang your front door by asking Alexa?

Imagine it rings at your door and you could just ask Alexa who’s there. Without going to the window or opening the door, you know if it’s a friend or a stranger, because Alexa tells you the name or if it’s an unknown person.

We had the idea to build this solution, since our grandmother is nearly 80 years old and short-sighted. Due to the distance of her garden door, she can hardly see who rings the door when looking out of her window. To help her distinguish people ringing at the garden door, she should be able to ask Alexa, without even the need to go to the window or opening the front door.

Demonstration Video

(Grandma hardly speaks English ;-))

To build the solution, we used:

  • an Arduino Uno with ethernet shield (can easily be ported to an esp8266 for wifi connectivity)
  • a camera (Arducam mini)
  • a switch connected to one of the inputs of the Arduino
  • Amazon Web Services: Alexa Skills Kit, Lambda, API Gateway, S3, Rekognition

We wrote an Arduino sketch that, when the switch is pressed, records an image with the arducam. Due to memory constraints, the image is not saved locally, but read from the camera buffer in chunks and directly posted using a http put request to Amazon Web Services, where a Lambda function (via AWS Api Gateway) processes the uploaded image, stores it in S3 and then passes it to AWS Rekognition. If the face is already known, the name of the person is returned. If an unknown person rings the door, the image is not deleted but stored to be tagged later on.

Using any of your Alexa devices, you can then use the smart cam skill we created.

There are two intents:

- „...who rang the door?“: If the person has been tagged before, Alexa will say e.g. „Peter rang the door“

- „...the person is Peter“: if a previously unknown person rings the door, this will associate the name to the face. Alexa will then confirm the association, and remember the face.

Example (known person rings the door)

  • Someone rings the door
  • Grandmother asks „Alexa, ask smart camera who rang the door“
  • Alexa answers „Peter rang the door“

Example (unknown person rings the door)

  • Someone rings the door
  • Grandmother asks „Alexa, ask smart camera who rang the door“
  • Alexa answers „I did not recognize the person, please tell me who rang the door. For example say: Alexa, tell smart camera that the person ist Peter“
  • Grandma then says „Alexa, tell smart camera that the person is Thomas“
  • Alexa answers „Thank you! I will remember that Thomas rang the door“
  • Next time Alexa will recognize Thomas. (Saving my Grandma from evil people)

Example (camera did not capture a face)

  • Someone rings the door
  • Grandmother asks „Alexa, ask smart camera who rang the door“
  • Alexa answers „The camera could not detect any face“.

Voice User Interface (VUI Diagram)

Hardware Assembly

  • Connect the Arduino Uno and the Ethernet Shield
  • Connect the Arducam as shown in the picture
  • Connect the pushbutton, connect your ethernet cable
  • Install the Arduino IDE and select "Arduino uno" as the board type.
  • Adapt the IP Address to your address range, and hostname (where to send the JPEGs to), and upload the sketch

Software Stack

Software Setup

The project uses several AWS services, and setup guides for the whole project can be found on Github:

In addition, the projects code can be found on Github and is fully documented for your reading pleasure.

Alexa Skill

We have submitted our skill to Amazon however it is currently being certified. The Skill ID is amzn1.ask.skill.fa2a11c4-2a92-4094-978c-05744757c63b. Set up your own skill using the setup guide above!

Code

Arduino SketchArduino
Sketch for Arduino Uno for controlling the Arducam and uploading the JPEG Picture to the AWS
// Smart Camera, Alexa controlled face recognition door bell
// 
// This demo can only work on OV2640_MINI_2MP platform.
//
// When switch is pressed, the camera caputures a JPG image.
// The JPG data is read from the SPI bus in chunks and directly sent out using http put request 
// in order to avoid the need of storing it into ram or on SD Card.
//
// (C) 2018 Sebastian Enzinger & Markus Enzinger

#include <Wire.h>          // library for Arduino ports
#include <ArduCAM.h>       // library for Arducam
#include <SPI.h>           // library for SPI communications
#include "memorysaver.h"   // configures the camera type used
#include <Ethernet.h>      // library for ethernet shield
#define switch 5 // GPIO pin for the door bell switch
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xF1, 0xED };    // Ethernet Mac Address to use for the ethernet shield
char host[] = "d35xafoveji7v.cloudfront.net";           // Where to send the image by http post request

const int CS = 7;         // set pin 7 as the slave select for the cam module's spi port:
bool is_header = false;
int mode = 0;
uint8_t start_capture = 0;
ArduCAM myCAM( OV2640, CS );
uint8_t read_fifo_burst(ArduCAM myCAM);
EthernetClient client;
//-------------------------------------------------------------------  setup() section -----------------------
void setup() {
// put your setup code here, to run once:
uint8_t vid, pid;
uint8_t temp;
IPAddress dnServer(192, 168, 3, 5);     // the dns server ip
IPAddress gateway(192, 168, 3, 5);      // the router's gateway address
IPAddress subnet(255, 255, 255, 0);     // the subnet
IPAddress ip(192, 168, 3, 251);         //the IP address of our ethernet shield
Ethernet.begin(mac, ip, dnServer, gateway, subnet);       // Initialize Ethernet Shield
Wire.begin();
Serial.begin(115200);                                     // Initialize Serial Output for debug messages
Serial.println(F("ArduCAM Start!"));
// set the CS as an output:
pinMode(CS, OUTPUT);                    // Chip select line to arducam
pinMode(switch, INPUT_PULLUP);          // Switch input: set pullup to read "high" when not pressed
// initialize SPI:
SPI.begin();
while(1){
  //Check if the ArduCAM SPI bus is OK
  myCAM.write_reg(ARDUCHIP_TEST1, 0x55);
  temp = myCAM.read_reg(ARDUCHIP_TEST1);
  if (temp != 0x55){
    Serial.println(F("ACK CMD SPI interface Error!"));
    delay(1000);continue;
  }else{
    Serial.println(F("ACK CMD SPI interface OK."));break;
  }
}
while(1){                           //Check if the camera module type is OV2640
    myCAM.wrSensorReg8_8(0xff, 0x01);
    myCAM.rdSensorReg8_8(OV2640_CHIPID_HIGH, &vid);
    myCAM.rdSensorReg8_8(OV2640_CHIPID_LOW, &pid);
    if ((vid != 0x26 ) && (( pid != 0x41 ) || ( pid != 0x42 ))){
      Serial.println(F("ACK CMD Can't find OV2640 module!"));
      delay(1000);continue;
    }
    else{
      Serial.println(F("ACK CMD OV2640 detected."));break;
    } 
}
myCAM.set_format(JPEG);          //Change to JPEG capture mode and initialize the OV2460 module
myCAM.InitCAM();
delay(1000);
myCAM.clear_fifo_flag();
}

//------------------------------------------------------------------ LOOP section ---------------------
void loop() {
uint8_t temp = 0xff, temp_last = 0;
uint32_t len;
bool is_header = false, errorflag=false;
int inChar;
if (digitalRead(switch))   // Switch pressed?
{
    myCAM.OV2640_set_JPEG_size(OV2640_800x600);  // set image resolution in Arducam module
    delay(1000);   
    Serial.println(F("ACK CMD switch to OV2640_800x600"));
    temp = 0xff;
    myCAM.flush_fifo();
    myCAM.clear_fifo_flag();
    myCAM.start_capture();                                   // start image capture
    start_capture = 0;
    while (!myCAM.get_bit(ARDUCHIP_TRIG, CAP_DONE_MASK));    // check if capture is finished
    len = myCAM.read_fifo_length();
    Serial.println(len);
    if ((len >= MAX_FIFO_SIZE) | (len == 0))                 // check for invalid image size
      {
        myCAM.clear_fifo_flag();
        Serial.println(F("ERR wrong size"));
        errorflag=1;
      }
    Serial.println(F("ACK CMD CAM Capture Done."));
    if (!client.connect(host, 80)) {                         // connect to host
        Serial.println("ERR http connection failed");
        errorflag=1;
    }
    if (!errorflag) {
      myCAM.CS_LOW();                                        // switch CS line of camera active
      myCAM.set_fifo_burst();//Set fifo burst mode
      SPI.transfer(0xFF);
      myCAM.CS_HIGH();                                       // switch CS line of camera inactive to allow ethernet shield to use SPI bus
      String response = "POST /prod/smartcamIdentifyPerson HTTP/1.1\r\n";
      response += "Host: d35xafoveji7v.cloudfront.net\r\n";
      response += "Content-Type: image/jpeg\r\n";
      response += "Content-Length: " + String(len) + "\r\n";
      Serial.println("connected to the server");
      client.println(response); 
      static const size_t bufferSize = 512;
      static uint8_t buffer[bufferSize] = {0xFF};
      while (len) {                                         // read chunks of 512 byte and send them out to host, because of memory constraints
        size_t will_copy = (len < bufferSize) ? len : bufferSize;
        myCAM.CS_LOW();
        myCAM.set_fifo_burst();//Set fifo burst mode
        for (int bufptr=0; bufptr<will_copy; bufptr++) 
          buffer[bufptr]=SPI.transfer(0x00); 
        myCAM.CS_HIGH();
        if (client.connected()) 
          client.write(&buffer[0], will_copy);
        len -= will_copy;
      }   
    } 
    myCAM.CS_HIGH();
    int connectLoop=0;
    while(client.connected())                               // Sending complete, now read response from host and forward to serial monitor
     {
       while(client.available())
        {
          inChar = client.read();
          Serial.write(inChar);
          connectLoop = 0;                             // set connectLoop to zero if a packet arrives
        }
       connectLoop++;
       if(connectLoop > 2000)                         // if more than 1000 milliseconds since the last packet
       {
         Serial.println();
         client.stop();                                // then close the connection from this end.
       }
       delay(1);
     }
  }
}
Smartcam Github Repository
Project Code + Setup Guide

Schematics

Arduino Uno / Camera Wiring
Schematic how the Arduino and Arducam are connected
Smart cam wiring rlapcfsxkb

Comments

Similar projects you might like

Intelligent Door Lock

Project in progress by Md. Khairul Alam

  • 15,965 views
  • 19 comments
  • 95 respects

Smart Pool: Alexa Controlled Pool Manager

Project tutorial by Benjamin Winiarski

  • 1,362 views
  • 2 comments
  • 7 respects

Alexa Controlled Door Sign Demo

Project tutorial by 3magku

  • 1,710 views
  • 0 comments
  • 5 respects

Alexa BBQ/Kitchen Thermometer with IoT Arduino and e-Paper

Project tutorial by Roger Theriault

  • 2,385 views
  • 0 comments
  • 9 respects

Dooreo - The Alexa Powered Automatic Door Opener

Project tutorial by Keith Caskey

  • 2,745 views
  • 0 comments
  • 12 respects

Alexa Controlled LEDs Through Arduino Yún

Project tutorial by Ben Eagan

  • 17,509 views
  • 10 comments
  • 24 respects
Add projectSign up / Login