Project showcase
VisionSense

VisionSense

An IoT wearable technology to haptically guide users with visual impairments/obstructions while monitoring their environment.

  • 997 views
  • 0 comments
  • 6 respects

Components and supplies

Necessary tools and machines

sander
bandsaw
Hy gluegun
Hot glue gun (generic)
sewing kit

Apps and online services

About this project

Inspiration

Our team was discussing the advances in wearable technology, and how it helps us stay connected while making daily life easier in aspects like health, navigation, and safety. Eventually, we began to wonder how wearable technology could play these same roles in the lives of different user groups. We considered the design specifications and changes for groups that cannot use the sense of sight- primarily people with visual impairment or in occupations like mining/mountaineering that can at times be in pitch darkness.

Functionality

Step (1) Environmental Assessment

The TELUS DevKit contains a Avnet Quectel BG96 Shield, a STMicro NUCLEO-L296ZG MCU Board, and a STMicro X-NUCLEO-IKS01A2 Sensor Board, assembled as show below. Sensory data from the users environment (e.g. temperature, pressure, humidity, latitude/longitude, location, etc.) is outputted and sent over LTE to the Azure IoT Hub; the data is simultaneously stored in Azure Cosmos DataBase (cloud).

Step (2) Obstruction Detection

The Raspberry Pi board is connected to 3 Ultrasonic Sensors, through which it determines any obstructions within a 4 metre range from the user's head. This data is wirelessly communicated to an online server via WiFi, that can then provide input to the Arduino system. The system is also equipped with a help button that sends an emergency text message to an emergency contact with just a simple press of a button by the user.

Step (3) Haptic Direction

The Arduino uses input communicated wirelessly from the Ultrasonic sensors to set custom vibrations that can communicate any obstructions, along with their range and direction relative to the user. Each Ultrasonic Sensor provides input to one motor. Placement of motors is intuitively along the same side as US sensors, so users can easily estimate obstacles based on haptic sense. This component allows users to act upon all the information gathered from our system to seamlessly move around without any hand-held aids.

Challenges

  • Setting up the Python Wireless Connection for Data collection
  • Drawing out (and accordingly building) sound circuitry to ensure all UltraSonic Sensors provide accurate data
  • Connecting Telus Devkit to Azure
  • Stably building the housing components in wearable form for the systems
  • Interfacing Raspberry Pi with Arduino

Accomplishments of Which We Are Proud

  • We did not have actuators or vibrators available, so we really had to understand the inner workings of Stepper Motors and rewire them (as well as format the code accordingly) to have them work as vibrators using input from the Raspberry Pi
  • Setting up the environments for all the systems-on-a-chip was a time-consuming and tedious job with much debugging. Getting through that took many hours, but we managed to successfully interface and integrate all components of our tech
  • The data output by hardware was a bit challenging to work with, especially in the case of the UltraSonic Sensors. However, with much testing and documentation review, we did eventually achieve proper obstacle detection using these sensors

Future Steps and Improvements

  • making the wearable technology smaller and easier to wear
  • using computer vision with an embedded camera that communicates (using audio) any pre-determined tags to user if they appear before the user

Code

GitHub Repository with all main Coding Components
Download files to try out the project; some initial environment set-up would be required by viewers prior to downloading (specific to machine and version of services the viewers are using)
Motor Code and main Azure file
A copy of relevant code

Custom parts and enclosures

haptic feedback motor housing
haptic feedback motor lid

Schematics

arduino_circuit_diagram_PsuTKKygaZ.jpg
Arduino circuit diagram psutkkygaz

Comments

Similar projects you might like

Morse Code Transmitter - Light Signal

Project showcase by Alireza Karkon

  • 1,597 views
  • 1 comment
  • 7 respects

DHT11 /DHT22 Temperature Sensor

Project tutorial by Daniel Porrey

  • 86,065 views
  • 106 comments
  • 137 respects

GPS Datalogger, Spatial Analysis, and Azure IoT Hub.

Project tutorial by Shawn Cruise

  • 18,187 views
  • 4 comments
  • 69 respects

Live Hard, Ship Hard!

Project showcase by SUAI/IHPCNT

  • 8,459 views
  • 2 comments
  • 21 respects

Windows 10 IoT Core: UltraSonic Distance Mapper

Project showcase by Anurag S. Vasanwala

  • 53,288 views
  • 23 comments
  • 143 respects

360 Radar (that encodes data onto an image)

Project tutorial by Alireza Karkon

  • 7,617 views
  • 5 comments
  • 41 respects
Add projectSign up / Login