Project tutorial
Audible Vision

Audible Vision © GPL3+

Duffy the mobile robot moves alongside you connected through a "light leash" and alerts you of objects around the path.

  • 2,242 views
  • 0 comments
  • 9 respects

Components and supplies

Necessary tools and machines

09507 01
Soldering iron (generic)

About this project

Updated for the 2019 China-US Young Maker Competition

1)   Background

AudibleVision is designed to alert a person of objects around them. The project’sbrain is the Arduino Uno, which is wired up with two IR Distance Sensors totell you how far an object is. Mounted onto the Arduino is a Bluefruit EZ-linkBluetooth shield that connects to any Windows 10 device, e.g. Laptop or Phone.The device should run the Virtual Shields for Arduino App and use the Windows10 speech synthesis capability to tell the distance. In order to expand thisconcept into a robot, stack a Motor Shield onto the Uno and obtain a mobileplatform to place the sensors and microcontroller. Wire up a Light Sensor and aRemote IR Sensor to the analog pins of the Arduino, and you have a robot thatis controlled by a remote, stays close to a walking person with a leash enabledby a light beam, and warns the person of objects ahead or around theirvicinity. For additional fun connect up a Piezo buzzer and Siren to beep whenchanging direction.

Since Ialready had a Multiplo-based robot, I mounted the Arduino Uno with theBluefruit and distance sensors onto the Multiplo-Bot because of the Uno’sunique Bluetooth compatibility. But for the rest of the items (Light Sensor, RemoteIR Sensor, Piezo Buzzer and Siren) I used the already-existing microcontroller“Duino-Bot”, which drove the motors of the Bot and used a Ping UltrasonicSensor to detect objects. In the future, I plan to connect the twomicrocontrollers. The final contraption, “Duffy”, is a mobile companion that isdirected with a remote, and senses and alerts the user of objects closeby. It took me just about a week’s worthof tinkering in my spare time during my High School Winter break.

Thisproject is simple to create and has a variety of possible add-ons. Instead ofusing a Windows 10 laptop, a Raspberry Pi running Windows 10 can be connected seriallyto the Arduino. A GPS shield stacked onto the Uno can provide the robot withits exact location. Or instead of using a robot, a tiny Arduino such as theMKR1000 can be fitted into a walking cane with IR sensors, which cancommunicate to a Windows Phone via Wi-Fi and inform a visually impairedindividual of objects around them.

2)   Setup

a. Arduino Uno setup

Firstinstall the Arduino IDE Rev 1.6.x on your Arduino Uno from the following link: IDE. Then install thefollowing libraries:

§ Average (https://github.com/MajenkoLibraries/Average)

§ ArduinoJson (https://github.com/bblanchon/ArduinoJson)

§ VirtualShield (https://github.com/ms-iot/virtual-shields-arduino)

§ Tutorial for Windows Virtual Shields for ArduinoApp (http://ms-iot.github.io/content/en-US/win10/WVSA.htm)

     o Install Microsoft Visual Studio 2015: (https://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx)

     o Download the following library: (https://github.com/ms-iot/virtual-shields-universal)

     o After unzipping the Virtual Shields UniversalDevelop folder, copy it into the Projects folder of Visual Studio. Then open itand click on Shield.sln. On the right hand side in the Solution Explorer thefolders “Shield, Shield.Communication, and Shield.Core” will appear as“loading”; right click on one of them and install the NuGet Packages. Finallychoose Build > Rebuild Solution, and then click the green arrow (set toLocal Machine) to run the code, and the Virtual Shields for Windows App shouldopen!

b. Sharp IR distance sensors (see Fig. 1)

Connectthe two IR distance sensors as shown in Fig.1 to the Arduino Analog IN pins, 5Vand GND. It is very simple to get a voltage reading from the IR sensors using “AnalogRead(pin#)”. However, in order to translate the sensorvoltage to an object distance, I used the following tutorial for generalguidance: IR.Then I used my own measurements to find the coefficients of a polynomial fit tothe data (distance vs. voltage) and compute the calibration curve in theArduino code.

c. Bluetooth setup (see Fig. 2)

Solderthe stacking headers on the Adafruit Bluefruit EZ-Link Bluetooth shield. It is advisable to test the Bluetoothcommunication with the Windows 10 Laptop before proceeding to the nextsteps. For this simple Bluetooth pairingand communication test I followed the following tutorials for the Bluetooth Breakoutor Shield. Make sure that the switch is set to theappropriate mode while downloading code into the Arduino or trying tocommunicate with the Bluetooth shield.

d. Light and Remote IR sensors (see Fig.3)

Iused simple “AnalogRead(pin#)” commands to read the voltage from the Light Sensorand selected a threshold value for brightness based on the backgroundlighting. If you want the Robot torespond in bright background light then 2 light sensors can be installed, whereonly one of the sensors receives the light beam from the flashlight. Then the difference voltage can be used torespond to the light beam selectively. For the IR Remote sensing I first recorded the ASCII code that thesensor sends when specific buttons on the remote are pressed, and used thatcode to program the Arduino for forward (UP arrow) and backward (DOWN arrow)motion.

e. Piezo Buzzer and Siren (see Fig. 3)

Toalert the user of objects in the front and change of motion, the Multiplo-Botgenerates sounds through the Piezo Buzzer and the Siren. These are very simply interfaced with theArduino by connecting the positive terminals to Digital OUT Pins and thenegative terminals to GND through a 470 Ohm resistor each. The resistor values can be changed forgetting different sound levels. I usedthe “DigitalWrite(pin#)” command for the siren and the “tone(pin#, frequency,time)” command for the buzzer to create the sound effects.

3)   Bill of Materials

4)    Schematics

Fig. 1) Setup of IR distancesensors with Arduino Uno: Analog IN Pins #3 and #4 are used to detect objectsat left or right of moving Multiplo-Bot. This information is then sent to a Windows 10 Laptop via the Bluetoothcommunication (Adafruit Bluefruit EZ-Link Bluetooth Shield shown below).

Fig. 2) Adafruit Bluefruit EZ-LinkBluetooth Shield for Arduino: The switch is set to “Direct” mode for Bluetoothcommunication with the Laptop’s Virtual Shield for Arduino App. The switch is set to “Soft Serial” mode fordownloading Arduino code via USB cable.

Fig. 3) Setup of Sensors andAudio with Duino-Bot: The Remote IR and Light Sensors are connected to the AnalogIN ports S1 and S4, respectively. The Piezo Buzzer and Siren are connected tothe Digital OUT pins #5 and #6, respectively. The 470 Ohm resistors in the paths to GND are used to limit the currentthrough the Buzzer and Siren and so determine the sound level. The resistor value can be adjusted for adesired sound level.

In this demothe Ping Ultrasonic Sensor connected to Analog IN port S2 is used to stop theMultiplo-Bot when an object is detected in the front. But this is not needed for the Audible Visionproject, since the user controls the robot motion with a light beam.

Code

Arduino Uno communicating with Virtual Shield for Windows AppArduino
Arduino IDE code
/* About Audible Vision: 
 *  Arduino detects the distance of an object in front of a robot using three different IR sensors 
 *  Sends this information to a Windows 10 PC, which uses the Virtual Shields for Arduino app to speak the distance
 *  The libraries ArduinoJson, Virtual Shield, and Speech belong to the Windows Virtual Shields for Arduino 
 *    package that contain a text to speech synthesizer.
 *  
 *  About Sharp IR Sensor:
 *  Some of the following code is borrowed from: http://luckylarry.co.uk/category/programming-tutorials/arduino-programming/
 *  
 *  February 2016
 *  Written by Daniel Martin
 */

//include all libraries 
#include <Average.h>  

#include <ArduinoJson.h>


#include <VirtualShield.h>

#include <Text.h>

#include <Speech.h>

// Instantiate the shields.
VirtualShield shield;

Speech speech = Speech(shield);




int pin2 = 4;     //initializes pin1 as analog pin A1 on the Uno for the left sensor
//int pin2 = 2;     //initializes pin2 as analog pin A2 on the Uno for the middle sensor
//int pin3 = 3;     //initializes pin3 as analog pin A3 on the Uno for the right sensor

void setup() {
  Serial.begin(9600);

  // Begin the shield communication
  shield.begin(9600);

}

void loop() {


  Detect1();    //handles the voltage read from pin 1
  //Detect2();    //handles the voltage read from pin 2
  //Detect3();    //handles the voltage read from pin 3
  
  

}


//Detect1 reads in the voltage of pin 1 and converts it to units of cm.
void Detect1()      
{
  float volts1 = analogRead(pin2)*0.0048828125;   // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
  /*if(volts1 < 0.4)
  {
    volts1 = 0.4;
  }
  else if(volts1 > 5)
  {
    volts1 = 5;
  }*/
  float distance = 65*pow(volts1, -1.10);          // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
  float AA = 0.00000008*pow(distance, 4);         
  float BB = .00003*pow(distance, 3);
  float CC = .005*pow(distance, 2);
  float DD = 0.11*distance;
  float EE = 4.9093;
  float FINAL = AA-BB+CC+DD+EE;                //the final distance, unrounded, is the sum of the terms in the quartic equation
  int fin = FINAL;                             //the final distance, rounded to the nearest cm.

  ///*Serial.print("Object ");
  //Serial.println(fin);
  //Serial.println(" centimeters to your left");*/
  String Fin = String(fin);
  speech.speak("Object " + Fin + " centimeters to your right");
  delay(5000);
}

//Detect2 reads in the voltage of pin 2 and converts it to units of cm.
/*void Detect2()      
{
  float volts2 = analogRead(pin2)*0.0048828125;   // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
  if(volts2 < 0.4)
  {
    volts2 = 0.4;
  }
  else if(volts2 > 5)
  {
    volts2 = 5;
  }
  float distance = 65*pow(volts2, -1.10);          // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
  float AA = 0.00000008*pow(distance, 4);         
  float BB = .00003*pow(distance, 3);
  float CC = .005*pow(distance, 2);
  float DD = 0.11*distance;
  float EE = 4.9093;
  float FINAL = AA-BB+CC+DD+EE;                //the final distance, unrounded, is the sum of the terms in the quartic equation
  int fin = FINAL;                             //the final distance, rounded to the nearest cm.

  Serial.print("Object ");
  Serial.print(fin);
  Serial.println(" centimeters in front of you");

  delay(2000);

}

void Detect3()      
{
  float volts3 = analogRead(pin3)*0.0048828125;   // value from sensor * (5/1024) - if running 3.3.volts then change 5 to 3.3
  if(volts3 < 0.4)
  {
    volts3 = 0.4;
  }
  else if(volts3 > 5)
  {
    volts3 = 5;
  }
  float distance = 65*pow(volts3, -1.10);          // worked out from graph 65 = theretical distance / (1/Volts)S - luckylarry.co.uk
  float AA = 0.00000008*pow(distance, 4);         
  float BB = .00003*pow(distance, 3);
  float CC = .005*pow(distance, 2);
  float DD = 0.11*distance;
  float EE = 4.9093;
  float FINAL = AA-BB+CC+DD+EE;                //the final distance, unrounded, is the sum of the terms in the quartic equation
  int fin = FINAL;                             //the final distance, rounded to the nearest cm.

  Serial.print("Object ");
  Serial.print(fin);
  Serial.println(" centimeters to your right");
  delay(2000);

}*/
Duffy responding to Light leashArduino
DuinoPack IDE code
#include <IRremote.h>



int RECV_PIN = 15;

IRrecv irrecv(RECV_PIN);

decode_results results;

int photopin = 18;
int sirenpin = 6;
int directioncheck = 1;

const int pingPin = 16;

void setup()
{
  Serial.begin(9600);
  pinMode(sirenpin, OUTPUT);
  irrecv.enableIRIn(); // Start the receiver
  //motor0.setClockwise(false);
}

void loop()
{
    // establish variables for duration of the ping, 
  // and the distance result in inches and centimeters:
  long duration, inches, cm;

  // The PING))) is triggered by a HIGH pulse of 2 or more microseconds.
  // Give a short LOW pulse beforehand to ensure a clean HIGH pulse:
  pinMode(pingPin, OUTPUT);
  digitalWrite(pingPin, LOW);
  delayMicroseconds(2);
  digitalWrite(pingPin, HIGH);
  delayMicroseconds(5);
  digitalWrite(pingPin, LOW);

  // The same pin is used to read the signal from the PING))): a HIGH
  // pulse whose duration is the time (in microseconds) from the sending
  // of the ping to the reception of its echo off of an object.
  pinMode(pingPin, INPUT);
  duration = pulseIn(pingPin, HIGH);

  // convert the time into a distance
  inches = microsecondsToInches(duration);
  cm = microsecondsToCentimeters(duration);
  
  //Serial.print(inches);
  //Serial.print("in, ");
  //Serial.println(cm);
  //delay(500);
 /* Serial.print("cm");
  Serial.println();*/
  
  delay(50);
  

  float lightVolts = analogRead(photopin);
  //Serial.println(lightVolts);
  
  if (lightVolts >= 400 && directioncheck == 1 && cm > 35)
  {
    FWD(20);
  }
  else if (lightVolts >= 400 && directioncheck == -1)
  {
    BACK(20);
  }
  else if (lightVolts <= 400 && cm > 35)
  {
    BRAKE(20);
  }
  else if(directioncheck == 1 && cm < 35)
  {
    //motor0.setSpeed(0);
    //motor1.setSpeed(0);
    tone(5, 2000, 500);
    BRAKE(1000);

  }
  
  if (irrecv.decode(&results)) {
    
    digitalWrite(sirenpin, HIGH);
    
    irrecv.resume(); // Receive the next value
    //motor0.setSpeed(50);
    //motor1.setSpeed(-54);

    if((results.value == 32) || (results.value == 2080) || (results.value == 16) || (results.value == 2064)){
       BRAKE(3000);
       directioncheck = 1;
    }
    else if ((results.value == 33) || (results.value == 2081) || (results.value == 17) || (results.value == 2065)){
       BRAKE(3000);
       directioncheck = -1;
    }
    
    //delay(500);
    digitalWrite(sirenpin, LOW);

  }

  

    
  //if(analogRead(1)>300)

}

void FWD(int time)
{
  motor0.setSpeed(50.0);
  motor1.setSpeed(-53.0);
  delay(time);
}

void BACK(int time)
{
  motor0.setSpeed(-50.0);
  motor1.setSpeed(53.0);
  delay(time);
} 

void BRAKE(int time)
{
  motor0.brake();
  motor1.brake();
  
  delay(time);
}

long microsecondsToInches(long microseconds)
{
  // According to Parallax's datasheet for the PING))), there are
  // 73.746 microseconds per inch (i.e. sound travels at 1130 feet per
  // second).  This gives the distance travelled by the ping, outbound
  // and return, so we divide by 2 to get the distance of the obstacle.
  // See: http://www.parallax.com/dl/docs/prod/acc/28015-PING-v1.3.pdf
  return microseconds / 74 / 2;
}

long microsecondsToCentimeters(long microseconds)
{
  // The speed of sound is 340 m/s or 29 microseconds per centimeter.
  // The ping travels out and back, so to find the distance of the
  // object we take half of the distance travelled.
  return microseconds / 29 / 2;
}

Schematics

Fig 1) Setup of IR distance sensors with Arduino Uno
Analog IN Pins #3 and #4 are used to detect objects at left or right of moving Multiplo-Bot. This information is then sent to a Windows 10 Laptop via the Bluetooth communication
Fig1
Fig 2) Adafruit Bluefruit EZ-Link Bluetooth Shield
The switch is set to “Direct” mode for Bluetooth communication with the Laptop’s Virtual Shield for Arduino App. The switch is set to “Soft Serial” mode for downloading Arduino code via USB cable.
Fig2
Fig 3) Setup of Sensors and Audio with Duino-Bot
The Remote IR and Light Sensors are connected to the Analog IN ports S1 and S4, respectively. The Piezo Buzzer and Siren are connected to the Digital OUT pins #5 and #6, respectively.
Fig3

Comments

Similar projects you might like

Adding Bluetooth to Your Arduino (+Light Sensor)

Project tutorial by IoTaLabs

  • 5,082 views
  • 0 comments
  • 20 respects

Persistence of Vision Arduino Code Generator

Project tutorial by Team Chusocreations

  • 5,664 views
  • 2 comments
  • 15 respects

Golf Grip alignment tool

Project in progress by Hunter Mitchell

  • 1,502 views
  • 2 comments
  • 6 respects

Portable Arduino Temp/Humidity Sensor with LCD

Project tutorial by ThothLoki

  • 153,444 views
  • 115 comments
  • 186 respects

Electronic Piano Keyboard With Preset Songs

Project tutorial by Lindsay Fox

  • 78,004 views
  • 60 comments
  • 150 respects

Enter the house like a Sith Lord

Project tutorial by Prasantha Jayakody

  • 20,023 views
  • 10 comments
  • 73 respects
Add projectSign up / Login